Ravioli code

The ngs-vo-tool - a utility for organising your virtual organisations - was released to the NGS's area on NeSCForge earlier this week.

Previous postings have described the reasons why the tool was written and how we worked out what it needed to do. This one is about developing and testing the program code itself.

So if you thought the 'Rough guide to the User Account Service' was dull, look away now.

The ngs-vo-tool was meant to be an example of 'software as documentation'.

If you are running complex software, which needs configuration information to be scattered around multiple files in different formats, and want to describe how you set it up then you have two options...

• Write a long and detailed description of every stage in the process and expect people to read and follow the documentation.

or

• You write a program to do the dirty work and expect anyone who wants to know how it works to read the program.

We are doing the latter. This is why ngs-vo-tool is written in Python - a language designed to be readable by people who don't actually know the language. Take this example from the code:

class LcmapsMapper(BaseMapper):
 "LCMAPS gridmapfile and groupmapfile entries"

 ... snip ...

 def add_mapping(self, vo, acinfo):
     self.assert_mapping_args(vo,acinfo)

     gridmap_e,groupmap_e = mapfile_entries(vo,acinfo)

     if gridmap_e not in self.gridmap_entries:
         logger.debug("Adding <%s> to gridmapfile" % gridmap_e)
         self.gridmap_entries.append(gridmap_e)

     if groupmap_e not in self.groupmap_entries:
         logger.debug("Adding <%s> to groupmapfile" % groupmap_e)
         self.groupmap_entries.append(groupmap_e)

 ... snip ...

As long as you can wrap your brain around Python's way of using indentation to group related code together, that is, describes the process of adding VO information to the contents of a LCMAPS gridmapfile and groupmapfile.

The ngs-vo-tool was also an attempt to apply the kind of software engineering techniques that our colleagues at the Software Sustainability Institute want to encourage in academia. In particular, we tried to use something approximating to Test Driven Development (TDD) to keep the bugs under control.

in TDD, for each bit of program code, there is a corresponding bit of test code to put it through its paces. A test harness is used to run all the tests whenever the code is changed.

If you have the source code, run:

  python setup.py test

to watch all 57 tests fly past.

In proper TDD, the test is written first and run before any attempt is made to write the thing being tested. I'm afraid that I was more pragmatic and wrote the tests and code at the same time.

This approach encourages the developer to split the code into largely-self-contained modules with well defined interfaces simply because these are much easier to write tests against. Some people refer to this as ravioli code.

I would be a fool to claim that the program is bug-free - although the testing did shake out bugs early in the development process and I do think that the code is cleaner and easier to read because of this approach.

Someone at eScience centre at Southampton once came up with the phrase 'making the useful usable' as a way of describing their work. I hope that someone will find the ngs-vo-tool useful in making their services to the grid usable.