NGS at SC'11 - round up

The UK presence was fairly significant this year at SC'11 with attendance by David Wallom, NGS Technical Director and a significant number of the leaders of research computing centres from around UK universities. This included NGS member sites @ Bristol, Leeds, Oxford, Southampton and EPCC.

In this blog post, David gives us a round up of NGS activities at this major computing event.

Having got off of the very long flight from Heathrow
to Seattle we settled onto the metro to get us  downtown. After passable sleep the following morning we headed over to the SC’11 venue – the Washington Convention Centre to collect our badges and then visit the workshop on HPC in Smart Grid, where there was UK interest from the EC FP7 HiPerDNO project being presented by Dr Stef Salvini, OeRC. Following a very productive day where we learnt the state of the art in US Smart grids, how they intend to utilize knowledge developed through the national e-infrastructure for research. We then met up with the EGI team who had an exhibition stand at the conference.

The Monday workshop, Many-Task computing on Grids and Clouds 2011,  started off with an interesting keynote from David Abramson (Monash) a long term friend of the NGS through support for their Nimrod tool which is popular with several of our biosciences users. After this there was a panel session which went slightly off topic to talk about exascale more than Many task but it still attracted several questions around the need for exascale, when we are still struggling to get a significant user base onto smaller HPC systems. Overall a good workshop though having the panel first did mean that a number of people didn’t hang around for the rest of the papers. This workshop was operating in a very competitive market with other sessions on cloud and data management which also attracted significant crowds.

The first full day of the conference allowed for the first good look around the exhibition floor alongside several interesting birds of a feather sessions There was also the  first of a number of conversations with different groups and vendors, including Microsoft, Mathworks and Adaptive Computing.  To give an idea of scale this picture is down one of the main aisles in one of the 5 rooms that were all about this size!!



Pretty impressive stands by a number of people
but the coolest was the multi projection globe on the NOAA stand.

We of course also announced
our activity with Globus Online which created a lot of interest and ended with us having a number of interesting conversations with NSF regarding future collaboration between our national e-infrastructures.





During the meeting the EGI booth was continually visited by a reasonably large number of people, we had the Real Time Monitor showing as normal (having seen a lot of 3D screens this needs to be done in 3d now for next year!). They did though give away a pretty large number of t-shirts as did a lot of stands, so I ended up as the moving poster board around down town  Seattle from 6:30-7am every morning on my morning run!

E-infrastructure summer school - registration open now!

A brief hiatus here on the NGS blog as several of us are / have been on holiday. Back to normal service now hopefully!

A lot of my time since I returned from holiday has been devoted not to the NGS but to another project I am currently working on. Catchily named SeIUCCR (pronounced "sucker"), the project was funded by the EPSRC "Crossing the Chasm" call which called for networks and "advocates" to promote the wider uptake of UK e-infrastructures by researchers in engineering and the physical sciences.

Part of the SeIUCCR project is an e-infrastructure summer school which is due to take place in Abdingdon near Oxford in September. The residential summer school will offer an introduction to e-infrastructure including Clouds and Grids to UK PhD students and postdocs over 4 days. The summer school is fully funded including travel expenses and applications are open now.

If you (or anyone you know) would like to apply then be quick as applications will close at 9am on Monday 15th of August. Details of the summer school can be found on the SeIUCCR website and a detailed agenda is available from the registration site.

NGS TV channel?

I'm beginning to think that the NGS might need it's own TV channel as there is another video featuring NGS staff available for your viewing pleasure.

This time it is the videos from the UCISA Cloud Computing seminar which was held in February in Loughborough, UK. This was a one day event focusing on a Cloud Computing which is becoming more and more popular in the UK academic sector.

The event was billed as "a perfect opportunity to take a look at the innovation and challenges with Cloud Computing and to reflect on the benefits it can bring to your organisation". The sessions were designed to give a flavour of Cloud Computing possibilities as well as case studies from early adopters.

The presentation videos are all available on the event website and include a presentation from the NGS Technical Director, David Wallom on the FleSSR project - Flexible Services for the Support of Research.

Taverna and Cloud on the NGS website

Incase you missed it (and we wouldn’t want that now!), a quick heads up of some new articles on the NGS website.

First up is an article on running Taverna workflows on the NGS. Mike Jones (NGS) and Donal Fellow (RCS) from the University of Manchester have been tackling this issue and are making rapid progress. The success of this project could result in an increase in NGS usage by the large Taverna user community.

The second article is an update on the NGS cloud prototype which has been running since last summer when we asked for users willing to try this new service. We were inundated with volunteers and this article by David Fergusson (NGS, NeSC) looks at the user communities taking advantage of this prototype service.

Looking for some reading material?

If you are looking for some reading material in the run up to Christmas then have a look at the latest edition of NGS News, the quarterly newsletter from the NGS.

This quarters edition contains features on -

• Using the NGS Cloud Protoype in teaching
• Running Taverna Workflows on the NGS
• Research Communities on the NGS Cloud Prototype
• Quantitative genetic analysis on the NGS
• ... and much more!

To download your copy of this quarters newsletter go to the dedicated NGS News webpage where you can download copies of all past NGS newsletters.

Dead. Again.

Apparently 'Grid Computing is Dead'.

Again.

It wasn't Colonel Mustard, with the lead piping, in the library. It was David De Roure, with a posting, on the Nature eResearch blog.

To be fair on David: he is an eyewitness - not the perpetuator to the dastardly deed. He was highlighting a panel discussion at the IEEE eScience conference in Brisbane entitled "Grid Computing is dead: Let's all move to the Cloud".

That title looks like another round of that popular panel game: my vague terminology is better that your vague terminology. As Simon Hettrick has pointed out - Clouds computing has one big advantage over Grid computing - the Name. Clouds sound nice and fluffy; grids sound hard and rigid.

You cannot really discuss Cloud computing in general. You need to talking about the the various Somethings-as-a-Service.

Most cloudy discussions concentrate on IaaS - Infrastructure-as-a-Service. Through the wonders of virtualisation, imaginary computers are conjured up on a magic box somewhere on The Internet. You can ask for an imaginary computer and use and abuse it just like a real computer under your desk. This has changed the way computing is delivered.

It is not the only option. Some Research Institutions, and commercial companies, are offering access to High Performance Computing systems and calling it 'Cloud Computing'. More accurately, this is  SaaS - Software-as-a-service - and PaaS - Platform-as-a-Service. This is good when you need access to a particular application or the messy bits needed to build an application. 

A few years ago, they might have been called such an offering 'Grid Computing'.

The Grid is a Platform. We are offering it as a Service. 

It might be a slightly-rickety platform but we have used it to support many applications and enable new research.

The Grid isn't dead. It has PaaS-ed over to the other side.

And breathe - it's over for another year

So the NGS Innovation Forum is over for another year and, although it seemed to consume most of my time over the last month, I’ll miss it!

This years event was well received by all those who attended according to the feedback I was given both at the event and afterwards by email. Always nice to know we’re doing the right thing!

The event kicked off on Tuesday with a day focused primarily on our users and I’m glad to say there were some present in the audience. Steven Young gave a brief summary of the Campus Champions who are our eyes and ears in institutions – ready to help users and to feedback comments and suggestions to us. We then moved onto a series of talks about user tools. The aim of this session was to talk through some of the tools that we offer in order that users could head home from the event and actually apply them in their research. The tools covered were –

• UI/WMS
• Application hosting environment (AHE)
• Databases
• Working with communities (in particular ccp4)
• NGS Cloud Prototype
• New Certificate Wizard

The day also featured three presentations from users who make a great deal of use of the NGS resources. We had presentations from a variety of research areas to demonstrate just how widely used our resource are. Luke Rendell from St. Andrews University talked about simulating learning strategies, Zhongwei Guan talked about modelling composite structures and Narcis Fernandes-Fuentes talked about using the NGS for early stage drug discovery. A bit of a range of uses!

Day 1 was really good with lots of questions and discussion which continued right the way from the last session through the drinks reception and poster viewing until the end of the event dinner!

Wednesday was aimed primarily at IT staff, sys admins etc so there were a few new faces on this day. In order to bring everyone up to speed, David Wallom re-capped the discussion from findings the day before. We then kicked off with a presentation from the University of Westminster who have been a NGS member for some time before moving onto a discussion session about how the NGS can help to facilitate collaboration between researchers and institutions.

Presentations on two NGS projects followed – accessing the NGS with Shibboleth and updates to the NGS accounting provision. The last session was dedicated to the EU with an update from the EGI Director, Steven Newhouse followed by presentations from two ESFRI projects – CLARIN and ELIXIR.

An exhausting couple of days but well worth it!

From an outreach point of view I’m now busy organising a couple of new roadshow events that people requested during the IF, I’m gathering the presentations from the event to go on the NGS website (watch this space!) and announcing the winner of the best poster at the event.

Congratulations to Jarmila Husby from the School of Pharmacy, University of London whose poster “Molecular Modelling Studies of the STAT3β homodimer:DNA complex” was voted the winner by the delegates. Jarmila won an Amazon voucher which is very handy with Christmas coming up! All the posters from the event will also be on the website soon.

If you missed the event there are a number of ways to catch up – the Twitter posts are available, a blog post from Catherine Gater of EGI, an article on Cloud computing from Simon Hettrick at SSI and photos from the event are available on the NGS Flickr account.

Thank you once again to all those who attended and hopefully we’ll see you all next year!

NGS Cloud at the NGS Innovation Forum

We're on the home straight for the NGS Innovation Forum. Registration closes this Friday (12th Nov) at 4.30pm sharp so if you want to attend and haven't yet registered, you had better do so soon!

All our previous NGS innovation forums have featured breakout sessions with groups reporting back to the meeting as a whole. This year we will have our breakout sessions as usual but we would like to flag one up early!

We will be having a break out session on the new NGS Cloud prototype so this would be an ideal opportunity for current users of our cloud service to feedback directly their experiences to NGS staff. It would also be an excellent opportunity for anyone interested in the NGS cloud prototype to find out more.

Remember you don't have to attend both days of the event, delegates are welcome to attend either day as a single day.

Splendid Isolation

I wandered lonely as a.... typical - I come back from a week's leave with a post on cloud computing to discover that every cloud related title has already been used.

The last few weeks have seen what can only be called a storm of cloud activity - the release of the Openstack cloud software; Amazon adding a cloud tuned for High Peformance Computing applications to their collection; the publishing of two reports on academic use of clouds and, or course, the public launch of the NGS's own prototype cloud service.

Some commentators have claimed the launch of the Amazon service means that The Grid is Dead. Obviously, those of us in the grid world disagree - as Craig Lee of the Open Science Grid explains.

Alternatively, we are now all officially zombies - mmm.. brains.

People who see the cloud as a computional grid with all the nasty authentication stuff ripped out are misunderstanding the point of nasty authentication stuff in a world where not all data is open and access control and audit trains are vital.

The people who say that the cloud is the greatest thing since sliced bread are also missing the point. Most clouds are quick-and-easy ways to get your hands on a virtual machine. It is virtual machines that are the greatest thing since sliced bread.

Why? It is all about keeping things separate.

Many modern applications are bad neighbours. They do not sit happily on a server but need plumbing into web service containers such as Tomcat or Glassfish and databases such as MySQL or PostgreSQL. It is not uncommon for the applications to require specific versions of their supporting software.

Trying to persuade two such applications to co-exist on the same server is the stuff of system administrators nightmares - sitting on the sysadmin scale of horror somewhere near an operating system upgrade or running out of coffee.

Virtual Machines let you dump these applications and their hangers on in an isolated environment comparatively cheaply and easily. Isolating applications isn't a new idea - it was implemented in the mainframe systems of the 1960s - and system admins have been using techniques such as chroot jails for decades to provide some level of separation.

With apologies for borrowing great poetry to make a cheap point: Wordsworth's poem Daffodils - the one that starts 'I wandered lonely as a cloud' - mentions the 'the bliss of solitude'. It is the solutude of computing clouds that makes them useful.

NGS clouds take off!

On Tuesday we announced that the NGS cloud prototypes were ready and that we were seeking users. Well I'm glad to say that we have had an excellent response and we have identified several use cases to kick off with. NGS staff are now in discussions with users to take these forward and get them up and running ASAP.

If you would like to be amongst the first to use the new NGS cloud prototypes and to receive assistance in getting started then let us know by contacting the helpdesk with details of your research. Contact details are available on the NGS website.

Cloudy skies

There has been a lot of news about clouds recently, and looking out of my office window, I can certainly see plenty! I thought this was supposed to be summer?

Here at the NGS we are talking about clouds in a good way with the announcement that our clouds prototype is now live and we are looking for NGS users who would be interested in using the resources.

We have two cloud infrastructure prototypes available - one based in Edinburgh and one based in Oxford. Both sites have staff on hand to help you get started using the resources. If you are interested in finding out more please see the announcement on the NGS website.

Head in the clouds

That's certainly where my head has been for the last 2 weeks as I've been on holiday! A big thank you to Jason and Jens for keeping the blog ticking over whilst I've been away.

However the title doesn't just refer to holidays (even though that's what most people are thinking of at this time of year) but also cloud computing. When I was away JISC released 2 reports on Cloud for Research.

The first report is entitled "Using cloud computing for research" and aimed to

• document use cases for cloud computing in research for data storage and computing;
• develop guidance on the governance, legal and economic issues around using cloud services for storage and computing in academic research;
• make recommendations to JISC on possible further work in the area for data storage and computing.

The second report is entitled "Technical review of cloud computing for research" and focused on the following areas -

• the current status of cloud computing in research communities;
• state-of-the-art cloud technologies in academic, commercial, and industrial domains;
• technical guidance on the use, adoption and migration to cloud computing for research;
• recommendations for future technical and standardisation work to JISC.

The reports and the resulting recommendations make for some interesting reading.

Keine Hexerei

It was good to see demos at the recent Open Grid Forum (OGF) in Chicago.

There was an OpenNebula deployed platform with associated cloud storage resources. The former used OGF Open Cloud Computing Interface (OCCI), and the latter used the Storage Networking Industry Association's (SNIA) Cloud Data Management Interface (CDMI). If I understood it right. It certainly looked nice. SNIA now have a reference implementation of CDMI - open source, too.

The grid filesystem demo worked fine - until they tried to show the resilience features . But they had a recording of a successful one. Seeing is believing.

In today's griddy and cloudy world, your work is somewhere else. To demonstrate your work to your colleagues, you need to access it. But conference networks may be flaky. Your laptop may suddenly decide to upgrade or virus check itself. You may expect to have a power socket nearby but you don't, and then you run out of battery. Of course you test it the day before but when it really matters, it doesn't work.

Jens' third law of computing: ``A succesful demo is indistinguishable from a rigged one.'' This is well known, of course: Arthur C. Clarke said "any sufficiently advanced technology is indistinguishable from magic."

Of course I am not accusing the successful demonstrators of cheating. They were probably just lucky.

Back in the old days, prestidigitators had to say "Keine Hexerei, nur Behändigkeit" (no witchcraft, only dexterity), to avoid accusations of witchcraft. These days in computing demos it's the other way around: no amount of dexterity alone will make it work, you need some magic or luck as well.

Cloud Computing Users and the NGS

The Belfast e-Science Centre (BeSC), which is a member of the NGS now offers a hosting-on-demand service within the NGS for UK academic users. The BeSC already have several commercial partners which take advantage of the service and would now like academic users to make the most of this service as well.

The service allows a remote user to deploy software into servers within the BeSC domain and to manage these deployed services remotely. The BeSC service hosting cloud can be accessed via a web UI and using an API BeSC are developing (currently called libcloud); a Europar 09 paper on libcloud can be found here.

The API is intended to provide a provider neutral interface to remote resources such as those provided by Amazon, Flexiscale etc and the BeSC hosting cloud; plugins for all of these providers are part of the library. If you have an interest in using this library in your development and/or helping its development please contact Terry Harmer (t.harmer@besc.ac.uk) for more details.