In this article I talk with Trish McNally, a Senior IT Quality Assurance Analyst at Optus. Earlier this year Trish used two different self assessment methods for ITIL quality assurance,here she describes the valuable lessons learnt from both methods.

CASE STUDY: Conducting ITIL self assessments

Trish, you conducted two ITIL self assessments, one in a classroom setting and the other with individuals completing the assessment remotely, what were the pros and cons of the two formats?

With the classroom session they’re a captive audience. They have immediate access to having questions answered. The only limitation is the number of people you can fit in. So if you’re going to use the group situation it’s useful to think about booking two or three sessions, rather than one.

With the remote session I was able to get more people to participate and everyone to complete within the allotted time, but it did involve some chasing up.

How many people were involved in both assessments?

As I was restricted by the size of the room for the first one, there were fifteen people. With the remote method I had thirty-three participate. The change from classroom to remote had no particular methodology behind it other than I could increase the number of people involved and collect more data.

Did you notice any difference in data quality from one method or the other?

I think the process being assessed has more influence than the method. For example, in the classroom we assessed the Incident Management process. If these people were doing the assessment at their desk they could be called away at any time. Having them in the classroom helped them focus without that distraction. Where with financial management and capacity management I don’t think it would make any difference.

It seemed more comments were recorded in the classroom session. All though I think it comes down to the individual.

How did you select people for the self assessment?

We already had initiatives underway to improve Change Management and Incident Management. The people involved in those were invited participate in the assessment. Managers were also asked to nominate two to three people from each area. I also involved a couple of customers.

How did you engage people?

The approach was couched in “we’re trying to improve these processes and this self assessment will help support that”. When you get people that are familiar with the processes, the majority of them love to give you the benefit of their knowledge if they’re invited to so.

Did you do anything different moving from the classroom to remote?

Based on feedback from the first assessment in the classroom, I put together a more comprehensive hand out sheet. When people did the assessment remotely, the hand out covered off a lot of the questions they would’ve asked. I found if people have questions and can’t get hold of you immediately they lose momentum and put off completing it. The more information you can provide them up front the better.

How long did you give the remote assessment group to respond?

I gave them two days rather than a week. My thinking was if you give them two days they more likely to do it straight away rather than leave it to the end of the week. For the remote assessments I did a number of conference calls, keeping the numbers small, making sure everyone could log into measureITIL and run them through some of the questions. What I found is if people logged on and selected the process they were going to assess, they often just continued and filled in the assessment while they were there. If they couldn’t log in, or they had some other issue, they were the ones I had to follow up.

How many people completed the task?

Everyone in the classroom setting completed the assessment while they were there. Out of the thirty three people that did the remote assessment, only one person didn’t complete it. I did have to do some chasing up though.

Any changes you’ll make next time round?

I’ll stick with the remote format because of the ability to include a larger number of people in the assessments. As there is no restriction, I would increase the numbers even more. I would still give people only two days to respond, as the tight timeframe gets their attention. I would also start chasing up people sooner.