welcome to the Cobalt Skyway

I’ve been writing Quantum for nearly 5 years now, and I enjoy the regular challenges it presents. About 3 years ago however I had a slight lull in my workload, so decided to use the time to try and learn something a bit different. My first thought was to take a look at sed or awk, as I had heard several people talking about them being useful tools, but then a colleague of mine suggested that I learnt something called PERL (Programme Extraction and Report Language) as it had, in his opinion, overtaken awk in terms of what it could do. So I picked up a 'Learning Perl' book and have been a huge fan ever since! It is always in the forefront of my mind whenever I am doing anything mundane or repetitive - 'I could write a Perl to do this for me - why am I doing it by hand?!'

It’s something that I think often doesn’t occur to people. Given complex multi-layered weighting, twisty hierarchical data, or complicated recodes, they will find clever ways to deal with them. However when it comes to pesky mundane tasks (I’m talking about things like adding significance testing to cross breaks, manipulating text, comparing data from more than one source etc.), too many people seem to take for granted that these things just have to be done by hand. Maybe it’s because most of these tasks take less than half an hour - and in some cases may only take 5 minutes, that we seem to just grin and bear it. But ever since I started using Perl, I take personal offense at having to do any repetitive tasks without writing something clever to do it for me! And I can’t count the number of times that I have done something by hand, thinking that I am never going to need to do it again, only to be faced with exactly the same situation a week later and then wishing that I had written something automatic for it!

A good example of this is setting up the DP on projects where our colleagues on the other side of the office have done the CAI. They provide us with a '.qax' file, which we then manipulate to build our axes. Quite often the questions that are asked in online surveys are set up in the form of grids, so that the respondent can answer several questions on one page. This is great if you want to show it in your tables as a grid, but if you want to split the parts of the question up into individual tables, then you have to spend time manipulating your axes and splitting it out into a series of lines and include files.

The main point here is that the input that we get is always the same, due to the naming conventions that our scripter’s use. The end result I want to get to is also always the same, so I see no point in spending 10 minutes every time doing the same task, when you could write a perl which will take the grid, split it up, and write your files for you. I then took it one step further and added some extra commands to the Perl so that if it is a rating scale question (Strongly agree - Strongly disagree for example) then it will also add factors, means and nets, as well as writing the axes and include file for a mean summary table for you. Not only is this all now saving me time, but as it is all automatic it cuts out any chance of me accidentally putting the wrong column number in the wrong place and mixing the data up.

This is just one example of what can be done using Perl. The main issue seems to be identifying areas where a Perl could be applied - to be fair it is easy to get carried away and spend half a day writing a Perl which will only save you 5 minutes and you will only use once. But in my opinion, if there is a task that you are doing more than once, and you are doing the same set of things each time, then there is no reason to do this by hand - no matter how tempting this is! Even if it starts off as only saving 5 minutes you never know where it will lead - every little helps!

Senior DP Consultant, Rebecca Cole works as a Senior DP consultant and heads up the Innovations Unit at Cobalt Sky