Versions of version control

Earlier this month, the developers at Manchester released a new version of the NGS accounting clients - with Condor and limited support for Sun Oracle Grid Engine.

Those readers who stumbled across the earlier R+D posting `Who, where and how much?' may recall that the accounting clients translate the records from local batch management systems into a format suitable for accounting on a Grid.

The code - like other software developed within the NGS - is being distributed via the NGS project at NeSCForge.

NeSCForge is a version of the GForge Collaboration Toolkit hosted at the National eScience Centre. In addition to using it to releasing packages, we also make some use of the revision control service.

A revision control service allows all changes to the files making up a package are recorded in a repository - with associated log messages - allowing the history of a piece of software to be tracked. When bugs bite, the ability to identify when a change was made is invaluable.

The revision control service is based around the venerable Concurrent Version System
(CVS). When CVS development started, back in 1993 according to CVS's own ChangeLog, CVS was the bees-knees if you needed to keeping track of large numbers of files.

The bees-knees aren't what they used to be. Many developers have moved on from CVS to Subversion or to distributed revision control systems like GIT. We use Subversion at Leeds, my colleagues at Manchester prefer GIT.

Some NGS software development, such as that for the VDT installer , is done via the NeSCForge CVS service. A lot is done via private repositories and other revision control software.

When code is developed in private, people outside lose the ability to spot when and where the bugs appeared. If you are developing security-sensitive software which can be used by anyone, it could be considered good manners publish these in a public repository.

If you use NeSCForge for that public repository, you need to feed everything to CVS.

This is easier than it sounds...

GIT comes with a git-cvsexportcommit command - that will translate GIT changes into CVS-ish commits. Subversion users can download and install svn2cvs.

We use svn2cvs at Leeds to mirror changes to X509runsetgid - mentioned in last week's 'Licensed to Grid' posting - to NeSCForge. For this to work, you need to give NeSCForge a SSH authorized key and use somewhat verbose command line...

env CVS_RSH=ssh svn2cvs.pl \
$SVNROOT/x509-setgid/trunk \
:ext:developerid@forge.nesc.ac.uk:/cvsroot/ngs x509runsetgid

While it can be a little embarrassing to leave a record of every bug you created in the public domain - you can console yourself with the knowledge that you are doing other developers a favour. While it is good to learn from your own mistakes - it is much better to learn from somebody else's.

Let's get together

And so back from the OMII-UK collaborative workshop which took place in Edinburgh Wednesday and Thursday this week.

Like last years event, it was a really interesting and productive 2 days with many actions resulting from the in-depth discussion groups. The great thing about the workshop is that it's not death by PowerPoint. The sessions are discussions sessions where delegates chose from a range of topics, disappear off into groups in various parts of NeSC and then after an hour everyone reconvenes and reports back their findings. Importantly it's not just findings but short-term and long-term actions that are reported back!

Unsurprisingly as the NGS outreach person, I attended several sessions on collaboration and was the "report back" person for the session on "assistance with publicity, outreach and dissemination"which had a healthy audience (obviously a pertinent topic!).

There were many findings over the past two days and these can be found ordered by session on the event website. However the one that really stuck in my mind was something that was raised by several of the researchers who attended the meeting.

They have things they need done which they can't do themself eg code written, advice on optimising or tweaking existing software. They need computer scientists for this but they don't know how to "hook up" with them for want of a better description! The things they need done quite often aren't large projects and may only take a few weeks or less but they don't know where to go for help, who to approach and who is best suited for the job.

Several possible ways of tackling this problem were suggested at the event and if you are interested in the ideas that came up, the slides from this particular topic can be found on the website under - "Long-term researcher-driven collaborations" and "Collaboration 2". This was such a popular topic that we ended up with 2 discussion sessions.

So the question is for those research scientists out there who work with computer scientists - how did you hook up? How did you get together and form a collaboration?

Advanced Distributed Services Summer School 2009

The NGS in conjunction with the Training, Outreach and Education team at NeSC are running an Advanced Distributed Services Summer School (ADSSS 09) on the 7th – 11th September at Cosener’s House , Oxfordshire.

The sessions will be taken by many of the leading researchers and technology providers in the field of Distributed Computing in the UK so it is a chance for students to take part in a unique learning experience in this field (including a large component of hands on tutorials) but also to spend a week in a small group with the leaders in the field!

The aim of the school is to help develop the skills of those involved in providing computational support for research in a wide range of disciplines.

In particular the school will focus on the use of, provision of interfaces to and the development of services based on employing the composition or aggregation of computational or data services.
Topics covered include how to compose a variety of services into bioinformatics work flows which can be used to support biomedical research processes, how to use and develop lab or department scale clusters of computers to run simulations, how to work with the NGS to compose protein simulation models for running on UK or international super computers, developing a portal to support legacy applications.

Further information on the summer school can be found on the ADSSS website or email adsss09@lists.nesc.ac.uk.

Registration is now open and the cost of the summer school including accomodation and meals (except Tuesday and Thursday) is £276.

Roadshow talks available online

If you are considering hosting a NGS roadshow at your institution or are simply curious about what goes on at these events, then have a look at the NGS website where the presentations from our latest roadshows (Aston and Bath) are now online.

The Aston roadshow was a 2 day event with a "normal" NGS roadshow held on the first day for a few hours before lunch with a full day of "hands on" training on the second day organised by the NeSC training, outreach and education team. Presentations from Aston are available here.

The Bath roadshow was a more traditional event with a series of presentations followed by lunch. Presentations from Bath are available here.

Both events were full organised by the NGS and NeSC leaving local organisers with the single responsibility of booking a suitable room - oh and some help with local advertising never goes amiss! If you would like to publicise NGS resources in your institution with very little effort required then please contact myself (gillian.sinclair@manchester.ac.uk) for further information.

NGS on the Road!

Earlier this week the NGS roadshow team travelled to Aston University to host a roadshow event for staff and students. The event was well attended with 25 people in attendance ranging from PhD students to ITS staff.

The talks covered topic such as what is the NGS?, getting started with the NGS and the NGS portal and job submission. There was also a presentation from one of our users – Jonathan Mitchell from the University of Leeds bravely stepped in after our original presenter couldn’t make it at the last minute. Jonathan talked about using NGS resources to understand the folding of DNA and showed some rather impressive animations.

The presentations created a great deal of discussion which carried over into lunch. There was plenty of NGS associated staff on hand to answer questions including David Wallom (NGS technical director), David Fergusson (NeSC Deputy Director of Training, Outreach and Education) and Hamza Mehammed (NeSC Senior Trainer). I was also there in my role as NGS Liaison Officer.

The feedback collected from the day was very positive with the majority of people planning on using NGS resources in the future as well as the majority of people planning on telling their colleagues etc about the NGS.

The NGS roadshow events are free to host as the NGS covers the cost of catering and staff travel. All the local institution has to do is book a room and help to advertise the event locally. The roadshows can be held just for the host institute or people from surrounding institutes can be invited as well. If you’d like to know more about the roadshows then please see the website or contact Gillian Sinclair, the NGS Liaison Officer (Gillian.sinclair(at)manchester.ac.uk).

What do people want from ICT? – Part 2

The second presentation at this JISC conference session was given by Prof. Rob Procter, Research Director of National Centre for e-Social Science (NCeSS).

Rob was talking about the e-Uptake project which looked at barriers and enablers to the uptake of e-infrastructure. Again interviews with 50 people from a wide variety of research areas were conducted as well as intermediary services such as ITS units at universities and national services. This resulted in over 25 hours of audio tape to be transcribed which must have been a daunting task!

The second phase of the project was of particular interest as this involved 25 “established and inspirational” HEI’s as well as small specialised research institutes. Two interviews were performed at each – one at a strategic level and the other at the direct user support level.
This was of particular interest to me as often we deal directly with the end user but it is important to have the support of everyone in the institution from the strategic level down to the “coal face”. This is particularly important to the NGS as we try to encourage more HEI’s to become members of the NGS. If people at a strategic level are not aware of the services and benefits on offer then it makes our job much harder.

The results from these interviews will eventually be available as the researchers are currently developing an online database of their findings. However some of the initial findings were presented. I’ve listed them here along with some comments on what the NGS is currently doing to tackle these problems.

1. People weight up the cost of adoption – is it worthwhile learning to use e-research?
Through user case studies and user participation in our roadshow and innovation forum agendas, we showcase how using NGS resources has helped real-life research take place. There is no better advertisement than our major users! We also showcase users research in many of our outreach mechanisms such as user case studies, the NGS newsletter, website etc. All examples to potential new users of how the NGS can help research.

2. What’s the long term sustainability?
The NGS has just received funding for another 2 years (until March 2011) and discussions are already taking place for long term security after this date.

3. Enablers are needed such as roadshows, consultancy, collaboration and the need to follow up potential leads
The NGS hosts roadshows at any interested institution and these events are organised by NGS staff ensuring as little burden as possible on local organisers. The events are free to host and attend. All you need to do is let us know that you’d like one!
The NGS has a helpdesk staffed by experienced grid users who are there to offer help in the immediate term but also longer term advice to institutions and users.

4. Lack of support infrastructure
The NGS has a helpdesk as mentioned above and also offers a great deal of support to new institutions wishing to join the NGS by providing them with a “buddy”. We also offer a wide variety of training which is free through the training team at NeSC so new users can get up and running as quickly as possible on the NGS and also training for more experienced users in order to hone their skills.

5. Awareness raising
Well this is what I do as the NGS Liaison Officer. It’s my full time job to make sure that the NGS reaches out to as many people as possible through every means available! We have a wide variety of Web 2.0 tools such as this blog, Twitter, Facebook etc. We attend a large number of research-based conferences in order to reach out directly to users as well as conferences held by organisations such as JISC. We go out to institutions with our roadshow events in order to reach a wide variety of people at an institution from PhD students to IT directors. We produce a large range of literature from user case studies to newsletters demonstrating how the NGS helps researchers and how NGS resources are used. And much much more!

Of course there is always more the NGS could be doing but with the resources that we have I’d like to think that we are tackling many of the issues that Rob raised and that our users do feel supported and informed. We are always looking for suggestions from users for things that they feel we could do more of or things they feel that are missing from the NGS. All comments are welcome!

NGS Roadshow - Aston in April

The NGS are hosting a roadshow and training event at Aston University on the 7th - 8th of April. The event is open to anyone at any institution including PhD students, researchers, academics, team leaders, ITS people - anyone who has an interest in using the NGS!

The first day comprises of a series of short presentations covering topics such as how to get started on the NGS and how to submit a job as well as presentations on the kind of support that is available to get you up and running.

The second day is a training session run by the TOE team at NeSC who will give a hands on session so participants can submit actual jobs to the NGS.

Participants are welcome to attend only one or both days of the event. For further details and to register for this FREE event please see the event page on the NGS website.

All set for Sicily

Preparations are well underway and indeed almost complete for the NGS presence at OGF25 / EGEE User Forum event next week in Catania, Sicily.

The NGS has organised the UK and Ireland Federation stand which is number B08 and located just outside the Michelangelo meeting room. Several UK and Irish organisations will be represented on the stand including GridPP, OMII-UK, NeSC, Grid-Ireland and the NGS.

Several NGS staff and associated staff will be present at the conference and will be helping to staff the UKI exhibition stand over the week. If there is anyone in particular you would like to speak to then please drop by and we can arrange a meeting. Staff present includes NGS directors, technical staff, helpdesk manager, liaison officer and many more!

Now that all remains to be done is hope that the packages for the exhibition stand arrive safe and sound and to pack the suitcase!

RA Training at NeSC

A bit short notice so apologies for that, but we have just been informed that NGS RA training will be taking place at NeSC next week (18th). So if you are interested in being able to dispense grid certificates at your institution then please see the webpage for further details.

Please note that it does say by "invitation only" but this is untrue! The event is open to any interested parties.

Live from the NGS Innovation Forum '08

Posting live from a very packed NGS Innovation Forum at the Museum of Science and Industry in Manchester. We have a total of 127 delegates here over the next 2 days from all over the UK.

Currently Dave Fergusson from NeSC is speaking about the training opportunities available from NeSC and the NGS which range from helping new users get started to helping more advanced users make the most of the wide range of services available from the NGS.

Other talks today have included a keynote presentation from Dan Katz about TeraGrid and Michael Wilson speaking about EGI and how the NGS is involved in Europe.

For those of you who can't make the event all the presentations are being videoed and will be available online. Watch this space!