ARC quirks and keeping track

Another week, another apology for having very little new to say.

Our excuse is that we have all been busy preparing for a meeting of NGS Collaborators (today, 6 July), a workshop on Moonshot, Grid and High Performance Computing (on Thursday) and for a Town Meeting on the future of e-Science and HPC Infrastructures and Applications in the UK (Friday).

On the plus side, when we have all recovered, there should have plenty to write about.

There has been a small amount of time available to work on Leeds' ARC grid software deployment - concentrating on the dull-but-useful task of tracking down and reporting minor bugs.

One such quirk appears when logfiles are rotated - that is renamed and compressed at regular intervals to conserve disk space. ARC continues to write to the original - now renamed - file rather than to a new one. We found the bug, and reported it and discovered that the developers were already aware and it will be fixed in the next release.

Which gives me a chance to opine...

In the 4-years-or-so that I have been involved with The Grid, I have needed to contact software developers all over the world.

With very few exceptions, the developers have been capable, helpful and responsive.
(and I am not going to to identify those very few exceptions)

The open nature of much of the development work, with bug databases and source code repositories readable by anyone, gives bug-hunters from outside the development team as much information as those inside. If used wisely, this information means better bug reports and faster fixes.

Unfortunately, bug-hunting is becoming harder. It is an unfortunate side effect of
the European Grid Infrastructure and European Middleware Initiative - and their remit to co-ordinate development activity from many disparate teams.

The individual development teams have their own development processes and tools....

• The WLCG Nagios developers at CERN use installations of Jira for bug-tracking, Confluence for documentation and and Fisheye for viewing the source code
• The gLite developers use their Savannah service for bugs and feature requests and provide ViewVC for source code browsing. [Addition 8-July: As Brian Bockelman has kindly pointed out - the ViewVC link above no longer takes you to the latest versions as components of gLite have moved to a TRAC instance at CERN.]
  
• ARC has a bugzilla for bugs and uses TRAC for source code browsing.

On top of this is the all-seeing EMI GGUS tracking system that sends bug reports to the individual teams and the EMI wiki.

There are different interfaces for different tools, sometimes you need a certificate, sometimes you need an account. As someone slightly outside - but with an interest in - the development of grid software, I know how hard it can be to check if a bug has been reported, whether it has been fixed and when the fix will be available.

There is clearly work being done to improve the situation and no-one would claim that distributed, international software development is easy to do. What the grid really does not want to do is weaken whatever connections there are with the system administrators who deploy the software and the external developers who use it.