Have you ever completed 360-degree feedback in which you select colleagues who then complete a questionnaire about you, their responses are averaged and then presented back to you? Many of us have as part of leadership development programme, some of us may even have experienced 360 as part of our performance appraisal.
In our ongoing research into the Top Management Programme (TMP), we have developed and piloted a different approach to 360-degree feedback to help us evaluate the impact of the programme. We have called this approach, ‘Story-based Evaluation’.
Evaluating leadership development is difficult. The ‘Story-based Evaluation’ is an attempt to consider leadership as asocial process whereby a leader influences others to accomplish shared objectives (Day, 2001). By engaging with these ‘others’, we hoped to uncover what they perceived as leadership in action. We sought to uncover beliefs, world views, and alternative forms of data. In response to Jarvis et al (2013) we sought an approach that gives attention to the “everyday experience” of participants while also providing insight into the “return on investment”. Most of all, we wanted to move away from asking for ratings of leadership characteristics to an approach that captures the context within which leaders find themselves (Bryman and Lilley, 2009) as well as the forms of leadership that emerge.
For this new approach, a sample of TMP alumni selected three colleagues. Rather than provide ratings against a list of leadership behaviours, each colleague is invited to relate three leadership stories concerning the TMP alumni in the last twelve months. Leadership stories are real situations where a colleague has observed the TMP alumnus: working to improve institutional performance; shaping institutional strategy; bringing about change; and/or working collaboratively for the benefit of the institution.
The stories are then presented back to the TMP alumni to consider their recollection of them and to reflect on whether the TMP influenced their practice in these situations.
So, what have we found?
Firstly, all of the alumni involved in the ‘Story-based Evaluation’ found the process beneficial to them:
- “I found hearing the stories extremely humbling actually… you never have these conversations really, everybody's busy and getting on with the day job… to get this level of in-depth feedback about what people valued about particular situations I think is very helpful.”
- “It’s a learning for me as well because it's very difficult to understand what people think about what you're doing… I didn't realise this was going to be this useful.”
- “Very enlightening… it's actually really helpful to hear what resonated with people.”
We also found that the colleagues providing stories found real benefit as well:
- “It is quite reflective for me actually, about what I like about [their] leadership style and maybe what parts of that I should try and sort of learn from myself.”
- “It made me think of some things and then in thinking about them and writing some notes about them, you are also engaging in reflection yourself. It makes you do a thing that you might not find time to do otherwise.”
- “The process of me engaging with you is helping me to learn part of the leadership style that I'm interested in.”
Thirdly, we are learning about what is perceived as leadership within HE right now. The stories provided examples of innovative leadership and ideas that challenged convention. There was evidence of the TMP alumni promoting collective leadership and working across boundaries. Change featured highly, both in terms of implementing change but also in addressing poorly thought-through change. In addition, many stories focused on communication and leadership-building.
Lastly, and for us most importantly, we asked what influence did TMP have on the leadership practice evident within these stories?
This will be considered more fully in our year two report but several findings are worth considering. The title of our research is “Leadership Journeys” and it was clear in these findings that TMP alumni did not start their leadership journey on TMP nor did it end there. Indeed, many of the stories were linked back to experiences both pre- and post-programme rather than TMP itself.
However, it was clear that the experience of TMP did play an important role and much of the practice was directly related to TMP through either, specific sessions and learning experiences, or linked to feedback shared by other participants. The influence of the impact groups (the participant-driven element of TMP) and TMP coaches was also discussed. Some of the leadership practice was a direct reaction against sessions in TMP e.g. group discussions where the alumni disagreed with the consensus, or a guest speaker advocating a model of leadership that didn’t feel right to the alumni. In a few cases, alumni shared that the stories mentioned by their colleagues were initiatives actually generated during TMP. Ultimately, a lot of the leadership practice was related to the confidence developed and reassurance offered through TMP, as well as through being encouraged to experiment and having a safe space to reflect.
As our project continues, we will be conducting ‘Story-based Evaluation’ with leaders from several of the TMP cohorts to build on the findings from this pilot.
Mark McCrory is a Lecturer in Management and Martin McCracken is Research Director at Ulster University. They are part of the research team working on ‘Leadership Journeys: Tracking the Impact and Challenge of the Top Management Programme’. You can find out more about this research project.
Jarvis, C., Gulati, A., McCririck, V. and Simpson, P. (2013)
Leadership matters: Tensions in evaluating leadership development. Advances in Developing Human Resources, 15 (1). 27-45.
Bryman, A. and Lilley, S. (2009). Leadership researchers on leadership in higher education. Leadership, 5 (3), 331-46.