Wednesday, 31 March 2010

How do you do it? We want to know!

Incase you haven't noticed there's a new poll on the home page of the NGS website (you'll have to scroll down a wee bit) which is proving to be rather popular. We want to know which operating system people use to access the NGS. So if you are a Mac, Windows or Linux user then let us know! Feedback of this kind is invaluable as we can then ensure that our documentation and tools are up to date and relevant for our users. The results so far are somewhat surprising!

Monday, 29 March 2010

NGS training event

The NGS will be holding a roadshow event at the University of York on the 20th of April. As well as the usual roadshow agenda there will also be a hands on training event taking place in the afternoon. The event is open to everyone who is interested in finding out more about the NGS and using our resources.

If you would like to attend the event make sure you register before the 13th of April!

Thursday, 25 March 2010

A matter of trust

With apologies for stating the obvious: when you are providing a service on a grid, you are expecting other people to make use of it.

Before you do, you want to be as sure as you can be that your guests are who they say they are.

The tools we use to establish trust are certificates. We might try to hide the details but we are not going to abandon the technology.

I am not going to explain how certificates work. The basic idea - public key cryptography - is used to establish trust to websites using 'https' and between hosts using Secure Shell. There is some not-exceptionally-scary mathematicsinvolved and some willfully-obscure jargon (X509, PKCS#1, CSR, PKCS#12). This should not hide the key idea:
A certificate is a statement that someone you trust will vouch that the certificate holder is who they claim to be for a limited time.

For example:

  • The UK eScience certificate used by the NGS and by GridPP - and which our users will have buried in a web browser or in *.pem or *.p12 files somewhere - states that the the UK eScience certification authority will vouch for the owner for around 13 months. This is usually the most valuable certificate that our users have and will be protected by some kind of passphrase.
  • A proxy certificate can be created by the holder of a UK eScience certificate. Proxies have a shorter lifetime - usually hours or days- and grants the holder of the proxy the right to act on behalf of the creator. Proxy certificates are the currency of the grid and are not necessarily passphrase-protected.
  • Virtual organisations of the kind used throughout the EGEE project can vouch that a proxy holder is a valid member of that organisation by adding an assertion to the certificate.
So what has this to do with the NGS's ongoing research and development effort?

While certificates are invaluable, no one would claim that they are easy to use.
They can go wrong in interesting ways:
  • You can get bitten by the limited lifetime built into the certificate. If things are running slowly, the proxy can expire before it is needed. Sometimes the assertions associating a certificate to a virtual organisation can expire before the proxy itself.
  • Obtaining certificates is complicated by the different ways web-browsers handle certificates and by the range of existing tools.
So the NGS is focusing its effort on easing the pain.
  • We have developed the certificate wizard: a friendly face to the less-friendly command-line tools.
  • There is a major project underway at STFC to replace the UK certification authority's web-based interface with simpler stand-alone utilities for managing certificates.
  • We provide a certificate trust service which can generate short-lived certificates on request for anyone with a account in a institution within the UK Access Management Federation. These can be used to apply for NGS time, use the NGS portal and make use of selected NGS partner sites.
To be useful to the research communities, the UK grids needed to establish a network of trust that could cover all researchers in all UK academic institutions. We now have our network and we are going to make it easier for you to join it.

Wednesday, 24 March 2010

Read all about it! - VO's, Condor, NGS Summer school and more!

The March 2010 edition of NGS News, the NGS quarterly newsletter, is out now. Featuring articles on the forthcoming "Communicating Science" event, virtual organisations, crystal structure prediction and Condor, Campus Champions, advance notification of the NGS Summer School, user case studies and much more.

The latest edition can be found in the Newsletter section of the NGS website along with all past issues of the newsletter.

Monday, 22 March 2010

Can you get your message across?

At the recent OMMI-UK Collaborations workshop, I was involved in the dissemination and outreach session which is hardly surprising as the NGS outreach person! However one of the topics that came up was that people felt they lacked the necessary skills to do outreach, be it getting their message across clearly to communities outwith their own, writing articles other than scientific papers or even just getting peoples attention!

Plans were already afoot at the NGS to tackle this problem as it was an issue that we had previously identified within the community. We are now pleased to say that registration for the event is now open and further details can be found on the NGS website.

The event is designed to give attendees the skills to communicate their work on a non-technical level, clearly and confidently to a wide range of audiences. The event is aimed at everyone who needs to get their message across be it to research peers, students and potential clients within the university community etc.

The event will be held at STFC RAL, Didcot, on the 11th of May and we hope to see you there!

Thursday, 18 March 2010

Send in the clouds

Today's vague term from the NGS Research and Development effort is 'Cloud'. Everyone wants a cloud.

Large companies have private clouds. The heavyweights of the IT world - Microsoft, Google and Amazon - all offer some form of cloudiness on a commercial basis. The UK Government has announced its own G Cloud project.

One of the major themes within the Research and Development work in the NGS is the provision of an Agile Deployment Environment - which can provide compute services quickly when needed.

This is yet another cloud, the kind usually known as Infrastructure-as-a-service. The idea is to provide virtual machines on request that can be set up in any way that the user wants.

The work is being managed from within the National eScience Centre at Edinburgh and done jointly by staff at NeSC and the Oxford eResearch Centre.

Development has focused on Eucalyptus, an open source implementation of Amazon's EC2 service.

Those involved presented their work at one of the regular NGS surgery meetings held on 11 March. You can find their slides in PDF and Powerpoint form on the NGS web site.

The broad conclusion is that - with current technology - it is possible to build a cloud that works like Amazon's but not (yet) one that scales like Amazon's.

We know that some researchers already use commercial cloud services. Which begs the question, why is the NGS adding another cloud to the crowd?

It partly down to data. Where there are enormous quantities of data to be crunched, or where the data is highly sensitive, a cloud hosted at a University has advantages.

Commercial cloud service charges reflect their costs and even Google, Amazon and Microsoft have to pay for an Internet connection. This is why commercial clouds typically charge for every megabyte of data transfered in-and-out of their service.

If you are dealing in terabytes of data, this can be very expensive. If you host a cloud for academic use in a UK academic institution, you can take advantage of the JANET network linking UK academia and JANET do not charge by the byte.

Some data is made available to academics under very strict rules - which mandate where it can be stored. It is quite common for the owners to forbid the export of data outside the UK or the EU. Again, if you process your data in a UK academic cloud, you can be sure it hasn't left the country.

By building an example cloud service and documenting the cloudy way, the NGS is providing a basis for future institutional and cross-institutional clouds.

Tuesday, 16 March 2010

Have you tried our UI-WMS yet?

It sounds painful but over 50 users since November have proved that it doesn't have to be!

The NGS UI-WMS service has accumulated over 50 users since it was released in November last year, a substantial fraction of the active NGS user base. Users are using the service for all kinds of work from computational chemistry through to bioinformatics.

Their work is automatically distributed by the WMS over the entire NGS, rather than just using one or two sites. If you would like to find out more about how this capability can improve your work flow check the case study in the last NGS Newsletter (page 5-11) or the resources and tutorials linked from the UI-WMS page. Most of the popular pre-installed applications already have examples of how to run them via the UI-WMS.

If you have any questions about the UI-WMS and how it can help you and your research, email our helpdesk - support(at)grid-support(dot)ac(dot)uk.

Friday, 12 March 2010

Let's get together

And so back from the OMII-UK collaborative workshop which took place in Edinburgh Wednesday and Thursday this week.

Like last years event, it was a really interesting and productive 2 days with many actions resulting from the in-depth discussion groups. The great thing about the workshop is that it's not death by PowerPoint. The sessions are discussions sessions where delegates chose from a range of topics, disappear off into groups in various parts of NeSC and then after an hour everyone reconvenes and reports back their findings. Importantly it's not just findings but short-term and long-term actions that are reported back!

Unsurprisingly as the NGS outreach person, I attended several sessions on collaboration and was the "report back" person for the session on "assistance with publicity, outreach and dissemination"which had a healthy audience (obviously a pertinent topic!).

There were many findings over the past two days and these can be found ordered by session on the event website. However the one that really stuck in my mind was something that was raised by several of the researchers who attended the meeting.

They have things they need done which they can't do themself eg code written, advice on optimising or tweaking existing software. They need computer scientists for this but they don't know how to "hook up" with them for want of a better description! The things they need done quite often aren't large projects and may only take a few weeks or less but they don't know where to go for help, who to approach and who is best suited for the job.

Several possible ways of tackling this problem were suggested at the event and if you are interested in the ideas that came up, the slides from this particular topic can be found on the website under - "Long-term researcher-driven collaborations" and "Collaboration 2". This was such a popular topic that we ended up with 2 discussion sessions.

So the question is for those research scientists out there who work with computer scientists - how did you hook up? How did you get together and form a collaboration?

Monday, 8 March 2010

OMII go visual

Our colleagues at OMII-UK have recently launched the OMII-UK YouTube channel which lets you view the latest software demonstrations and videos from their staff and collaborators.

The channel will be grown over the next few months but currently it contains software demonstrations of Campus Grid Toolkit, Middleware interoperation, OSCAR, Rapid and JSDL Applications Repository. However it's not all demonstrations as they also have videos showing the sustainability lecture recently presented at NeSc in Edinburgh by Neil Chue Hong (OMII-UK's Director) and interviews with OMII-UK's PIs and some of their partners.

The OMII-UK collaboration workshop will take place in Edinburgh this week so look out for blog posts from the various NGS staff attending including myself.

Wednesday, 3 March 2010

GlobusWorld 2010

Globus is one of the most prevalent middlewares in use for Grid computing and is used extensively in the NGS. GlobusWorld is the main shindig for globus developers and users to share their experiences of the software. This year's GlobusWorld is being held at Argonne National Laboratory, on the outskirts of Chicago.

The NGS presentation on "Authentication in the NGS" went down well with interest in MEG and the Certificate Wizard. It was also interesting to find out that GSI-SSHTerm is now embedded into the US Teragrid portal.

Certificate ease of use issues were echoed by other institutions who all seemed to be doing their own integration of identity provider and grid software so users can use their institutional credentials to get short-lived certificates.

There have also been many interesting talks about the latest advances in globus-related Middleware, in particular version 5 of the Globus Toolkit (gt5) and their cloud-like service

Tomorrow is the final day of the conference with tutorial sessions on the Globus Toolkit and

Further information on the above NGS tools/services can be found under Use the NGS / User Tools on the NGS Web site.

Tuesday, 2 March 2010

Blowing your trumpet!

So two NGS roadshows last week down south and both went very well! It's always nice to see our users present their work as it's the best way to really find out what you are all up to out there! Quite often the research presented is literally hot off the machine so the roadshow audiences are getting a truly up to the minute view of research being performed on the NGS.

As the liaison officer it's my job to let the world know what the NGS and it's users are up to so we can advertise your research and our resources through a wide variety of publications, events etc. We have recently had some of our users research featured in SCW (the role of geographic isolation and dispersal limitation in generating high endemic plant species diversity) and in iSGTW in an article on "Supporting the arts and humanities with e-science".

We also feature our users research on our website in our case studies section, posters, research papers etc. If you have anything you would like added into these sections (particularly the research paper section) then please contact me at support@grid-support(dot)ac(dot)uk.

Monday, 1 March 2010

The M-Word

There is one term that you really can't avoid in Grid computing - as much as you might want to.

It is the M-word - Middleware - a term that apparently dates back to the late 60s and has been gathering new definitions ever since.

As far as the current work of the NGS is concerned, Middleware is the software that you install on an existing computer system to hook it into the grid.

For sites associated with GridPP project, this software is gLite.

If you are outside GridPP and do not wish to use gLite, we currently recommend packages from the Virtual Data Toolkit (VDT). VDT is a project from the US Open Science Grid to collect together and package as much grid software as practical. There are more than enough packaged applications within VDT to get an NGS affiliate site on the grid.

Full NGS partners - who might need something gLite-like and capable of supporting many virtual organisations - can extend VDT with another set of seemingly random letters LCAS/LCMAPS (Local Centre Authorisation Service/ Local Credential Mapping Service).

To simplify the job of selecting the relevant packages from VDT, and building LCAS/LCMAPS - NGS staff have developed the 'NGS VDT Installer scripts'. These are maintained via the National eScience Centre's NeSCForge service (

The scripts started as a way of collating and documenting the knowledge of NGS operations staff.

Three years ago, the original NGS 'core' sites at RAL (STFC), Oxford, Manchester and Leeds had all deployed VDT but had set the service up independently.

So we got together, agreed on what packages were needed, how they should be configured and what local tweaks needed to make things work. Rather than simply documenting this information, we turned it into a set of executable scripts - the 'NGS VDT Installer' scripts - which could produce a consistent VDT-based service on a host.

The scripts can be thought of as runnable documentation. Someone who needed to reproduce a standard NGS installation could either run the script or read it as a guide to what to do.

More recently, following work done at Manchester, we scripted the process to building LCAS/LCMAPS. There is now a set of scripts that can take a site to full NGS partner status.

Maintaining and developing the installer scripts is one of the jobs of the NGS research and development group and we released the latest version last Friday.

Bug reports permitting, this will be the last of this generation of install scripts.

So where do we go from here?

That depends on what happens to the M-word over the next few months.

We do know VDT is still being developed. We also know there are plans to produce a pan-european Unified Middleware Distribution incorporating gLite.

Whatever happens, we'll be watching.