Why WPBA aren't working.

'Clare, you need a blog' - well Anne-Marie, here is my first attempt!

Curiously, as a medical educator who champions the use of blogs as a pedagogic strategy, I have seldom used one myself. However, recent twitter-conversations about workplace based assessment in medicine, lead me to extend and share my thinking about the ongoing debate about the value, or otherwise, of these tools.

I have spent the past six years, post MMC, engaging in rich, often challenging, conversations with educational and clinical supervisors, many of whom decry the learning value of WPBA and resort to meaningless, tick box practices to meet regulatory requirements. Others, wishing to use them as intended, raise anxieties about using the 'full range' of performance judgments, on the basis that the mean is 'above expectation' unless the trainee is experiencing difficulties. It is hardly surprising then, that Collin's (2010) evaluation of foundation training describes assessment of trainees as 'excessive, onerous and not valued'. The move away from WPBA to 'supervised learning events' is poignant, given that the original intent of WPBA was to capture learning arising from everyday working practices. WPBA were meant to 'script' in time for trainers and trainees to come together, to discuss observed practice and to ensure that feedback was an integral part of working life. I wonder then, what went wrong?

It seems to me that WPBA embodied a fundamental shift in the culture of medical training, a shift from time-served apprenticeship to time-measured training. Assessments, historically, were significant, loaded affairs, based around high-stake summative judgements allowing entry into the profession, or otherwise. WPBA in their fledgling foundation form, were designed to be something very different; a 'diagnostic' tool, illuminating profiles of performance, that could form the basis of a shared 'developmental' conversation. They were designed as formative tools, capturing everyday working practices, tracing development over a trainee's working year. For this reason, the foundation WPBA were individually formative (for learning) and collectively summative (of learning). They used both criterion (pre-determined descriptors of performance) and normative referencing, with the norm being what it was reasonable to expect of a trainee at the END of that stage in training. They were valid because they were designed to sample authentic practice (not assessed performance) and reliable in that they were based on multiple assessments, on multiple occasions, by multiple assessors.

In my experience, the lack of faculty development activity to support the introduction of these tools, meant that trainers defaulted to known assessment practices. i.e. WPBA were viewed as summative hurdles to overcome, and trainees baulked at any judgement below 'at the level of expect ion'. Boxes were ticked; feedback sections left empty. All too soon, WPBA started to shape working practices ('we better do all your assessments today, as your time here is nearly up') rather than respond to working practices. All too often, trainers unaware of the normative benchmark, used the tools to judge performance relative to the stage of training, rather than end point. All too soon, these skewed assessment practices diluted any potential learning value embedded in the tools.

So where next with WPBA? Given that WPBA are being held onto during specialty training, at least for now, are their ways to retrieve their learning value.

1. Assessors to be very clear about the intended purposes and parameters of the tools they are using. Formative or summative? Norm or criterion referenced? Understanding their purpose will lead to more purposeful use.
2. Royal Colleges should be explicit about the design and development of the tools they are advocating. How has their validity and reliability been assessed? What constitutes 'best practice' in their use. All assessments involve some kind of 'trade off';being clear about limitations of tools is just as important as knowing their strengths.
3. Trainers and trainees should have opportunities to familiarise themselves with tools before they are used; where merited this may include some shared faculty development activity so the learning value of tools can be enhanced.
4. Rather than seeing WPBA as a driver for learning, see them as a way of capturing and fostering learning. If you are going to use them, use them authentically and in so doing, add meaning to their use.
5. Feedback is key; not the rather instrumental 'good bad good' routine, rather a rich, developmental conversation which explores the ways in which performance might be developed, with 'what happens next' being a far more important topic than 'what just happened'.

Clare.