What works? … Just ask

Sometimes we are so caught up in the demands of practice, and the continual search (and pressure) for enhancement, we actually forget to stand back and ask ‘What do we do that helps students to learn and thrive?’. The ‘we’ here is my university and not me as an individual. After an extensive discussion with students these summary results (in the form of the graphic) provide a useful aide memoir to celebrate that which is good and/or it offers a list to help remind us what students find valuable.

Here is a copy of the graphic with a white background for anyone wishing to use or adapt this. If you need a different format, please just make contact.

The trouble and joy of action research in Teaching and Learning PgCerts

Over the last four years I have been leading a Postgraduate Certificate in Teaching and Supporting Learning in Higher Education. This programme is very similar to others up and down the country that have alignment to the UK PSF. Within this course we have a module that focuses on action research. Participants are required to undertake a small project, with the aim of improving and developing their own professional practice, while also strengthening their claim for recognition. I have recently evaluated the place of the action research project in this programme of study. I was overwhelmed by the extent to which colleagues wanted to keep the project component. They told me that they valued it because it enabled them to focus on an area of interest, it helps them to actually make a difference to the practice, and it helps them build relationships with different colleagues and students. It also helps staff to empathise with their own dissertation students and recognise their support needs. Other colleagues, who were not undertaking the programme, have reported how they have benefited too: some suggested that the evidence generated by these projects can really inform decision-making, while others suggested that the project elements makes new staff empowered to make change happen. After hearing all of this, it would be ludicrous to drop action research from our new staff development programme. Nevertheless, including action research in a postgraduate certificate of this type does not come without problems. The challenges that are noticeable, and which were cited by those involved in feeding back, included

– Some new researchers face real difficulty in understanding action research. For colleagues with a scientific background who have not previously undertaken social science research, moving to work with action research represents a massive paradigm shift. This can be disturbing and uncomfortable. It can mean that a good number of weeks grappling with what is and isn’t permissible in the style of research, and that a sense of bewilderment can take over. As an associated point, I believe that the language of action research is alienating to those from other traditions and those who support action research must try to use clearer language, or at least be explicit about the terminology at every turn.

Timing can be a real challenge, and it is important that action research is not rushed. There is a need to plan to work with the academic year cycle if change is to be meaningful.

– Deciding how to present the research developed through the project can be a point of difficulty. Within our programme colleagues are encouraged to use a format which fits with their audience’s needs/preferences to maximise impact, but there is a tendency to ‘play it safe’ and use a written ‘report’ format. In future iterations of the module dissemination will be added as an essential feature, not as a ‘nice to have’. It maybe necessary to provide dissemination opportunities through internal conferences and/or offer support for journal writing.

Choosing a focus can be difficult and can cause delay in getting started. Colleagues often have multiple ideas for research, some fit ideas their interests, some match with their managers preferences, some fit with a development need and some foreground the priorities of the organisation. Often there is no clear convergence between the ideas and the new researchers first job is to navigate through competing priorities to make a choice. It’s also important to help researchers critically consider the value of any pet projects; it is easy for passionate individuals to get engrossed in a topic without stopping to ask ‘ what are the benefits?’ or ‘so what?’.

– ‘How much action?is a question that I am often asked. Within a project does the action need to be substantial, or is it ok to research a topic to prepare for action? It is necessary to provide explicit guidance on this point.

Keeping focus can be tricky as projects develop. One question leads to another and the cycles of research in reality are often messy and don’t look like the text book. As a facilitator it’s important to reassure that the sense of chaos will pass, and to support the researcher to find a way through.

Keeping action research in our PgC is a ‘no brainier’ but much needs to be done to address these and other challenges. In response to some of these points, Lin Norton and I have penned a guide for first time action researchers to try to address many of the issues that we have seen as troubling. It is out on 1st March, published through the Higher Education Academy. Watch this space.

In the meantime here are the slides from my January presentation on action research in the context of a PgC, given to the HEA Scheme Leaders Network. Some of the student quotes contained within this really get to the heart of the benefits of lecturers and support staff undertaking action research.

Writing up your action research

I’ve been doing a lot of reviewing of action research project reports recently, and it strikes me that there is so much action research work taking place across the Higher Education sector, particularly in learning and teaching related postgraduate certificates, yet much of this work is not shared. Lessons learned by practitioners in one institution can often resonate and inspire others. We should be sharing this work in a way that is digestible and accessible.

This work also reminded me of some of the common difficulties for presenting action research. I’ve pulled together 8 tips for anyone considering how to present their work: 

1. Write in the first person. Don’t use third person to make the research appear to be separate from your own practice, because it shouldn’t be. Ultimately action research is about you and your practice, so don’t shy away from this. This is not a weakness. When your research is placed in the context of other research and literature, your own position is framed and can still be scrutinised.

2. Be clear what practice based problem you are trying to solve. This will provide a thread to help structure your research story. If the problem is not well defined, it is likely that the account that follows will also be fuzzy. Having a clear statement of the problem can provide an anchor for the research which can focus the literature, guide methodological choices, and frame the conclusion.

3. Set out why the issue being researched actually matters. Why does it matter to you, to colleagues, to students, and to the wider sector. Being clear why your research matters evokes interest amongst stake holders, after all if it doesn’t matter to you, why would others be interested?

4. Know the difference between action research and thoughtful action. Action research must use research methods (though any are permitted) and must involve the collection of data. Having an idea about improving your practice, making that improvement and then making general observations about its success doesn’t constitute data. Data should seek more than a simple endorsement to a change in practice, it should dig deeper.

5. Action research is a cyclical process by definition. Define your cycles. Every piece of action research should be a part, full or multiple cycle story. Often the clarity of the cycles gets lost amongst findings and reflections. Take time out during the research process and the research write up (or presentation process) to articulate the cycle(s) and steps in your particular research. I encourage my students to draw the research as a cycle. This ensures the research remains true to the approach and it can help the researcher manage the messiness of action research by seeing how steps fit together.

6. Time goes by quickly. In my mind 2004 was two minutes ago. But much has happened in Higher Education since the early 2000’s, with marketisation, technological advances, diversification of entrance requirements, the growth of metrics and so so much more. It is therefore very important to use up to date literature. It’s easy to rely solely on classic texts, but the world moves on. It may seem to be an obvious point, but failure to attend to recent evidence weakens research.

7. Action research demands a level of reflection on process and findings. Nevertheless it is critical to show your findings and analysis. Reflection is not a substitute for research, so be transparent about what your findings were before you reflect on them. It is almost impossible for others to read research critically if there is no transparency of findings.

8. The collaborative aspects of action research are widely written about, but sometimes literature makes collaboration sound and feel grander than it need be in reality. Collaboration can take many forms including working with students to understand an issue or to sense check the possible ways forward, it might mean working with colleagues to scrutinise your research design or findings, or as co-researchers. Collaboration generates a multiplier effect as it is a vehicle for spreading and sharing ideas. Engagement with others can help moderate and challenge deeply held assumptions which would otherwise limit the possible ways forward through the challenges and problems which are at the heart of the research issue. So the 8th tip, is be open to others and draw the learning from collaboration in to both the process and the reporting of the research.

Conference Reflections: Harper Adams University Learning & Teaching Conference 2017

This slideshow requires JavaScript.

Last week was the third Harper Adams Learning and Teaching Conference. This annual event brings together colleagues from across the institution, alongside colleagues from partner organisations in, and beyond, the UK. The conference was jam-packed with awesomeness! Although I couldn’t get to everything, the sessions that I did attend were informative and motivating.

Professor Tansy Jessop started off by inspiring a ‘nuclear climb down’ on assessment, where together teaching staff agree to summatively assess less. The shift away from too much summative assessment, Tansy reminded us, only succeeds if we collectively mean it. If some tutors are stealthily continuing to over-assess, then inevitably students will direct their attention to these activities at the expense of others. She talked about relinquishing assessment driven pedagogies of control, to a curriculum driven more by learning.
The keynote also brought some pragmatic suggestions of what staff can do by way of formative feedback strategies. I sensed a quiet wave of ‘Oh yeh’ moments around the room as the ideas were really workable. Suggestions included:

  • A policy approach of less assessments (the coordinated climb down)
  • Use of public spaces, like blogs, to collate ongoing learning and reading (the audience drives meaningful engagement)
  • Asking students to design multi choice questions
  • Asking students to bring along sources to class and then through group discussion arriving at the ‘best’ sources
  • Working with journal papers to write abstracts or deduce ideas in papers based on abstracts.

My own ‘aha’ moment was to rename every formative assessment, simply as activities that drive learning. I think I knew this already, but it’s easy to drown in terminology and metrics that cloud definitions and purpose. The keynote also highlighted how we might make the most of formative feedback. Humorously critiquing some well used feedback structures (like the feedback sandwich), Tansy suggested that, essentially, we need to become more dialogic around feedback. We need to find ways to have conversations, find out what feedback is useful, encourage students to solicit the right types of feedback and to take control of their learning.

In one of the workshop sessions the brilliantly enthusiastic Professor Kay Sambell encouraged us to consider how we use exemplars. Some sharing around the room threw up some different practical approaches, including using exemplars to: demonstrate the underpinning features of academic writing (e.g. What is involved in making an argument); take the stress out of understanding a task to free up headspace for more detailed and creative aspects of the task, essentially this is about demystifying the task; provide a process of socialisation in to the academic requirements of assessment; and, to provide a starting point. We also discussed some of the limitations of using exemplars, which included: Triggering worry in students who may believe standards set to be unachievable; stifling creativity as students might only see one way to complete the task; and, risking students believing the exemplar to be the finished article rather than a finished article. Moving on from our evaluation, we identified different things to do with exemplars. We were united in agreeing that just giving examples would do little in itself to help students. Active use of exemplars was shown to include such things as:

  • Peer marking to familiarise with task requirements
  • Discussion of different exemplars
  • Rank ordering exemplars
  • Analysing ‘fail’ grade work to help understand what should be avoided

Decisions about how to use exemplars included whether to annotate, whether to provide full or partial exemplars, and whether to use student work only or to consider tutor generated work too. By the end of this session my ‘note to self’ was that looking at weaker work in depth was a valuable step in working with exemplars. It provides a window in to the assessment process for students, it can help them avoid common pitfalls and it can massively raise awareness of issues of academic practice.

Rebekah Gerard’s poster was a great complement to Kay’s session. Bex shows how we can really use live exemplars in a workshop session to improve exam technique. She used a technique called ‘pass the problem’ and her PgC action research showed how students experienced this strategy.  Her poster shows the technique she used for ease of replicability:

Dr. Jaqueline Potter, from Keele University, shared her analysis of teaching excellence award nominations which had led to a better understanding of what qualities students value in staff. The overwhelming message was about kindliness. Whilst students want constructive, joined up and useful feedback, they really want it as a personal, kindly interaction. How to be kind is quite a different matter, but presumably remembering what it was to be a student would go a good way to help to keep an empathetic mindset. After completing our in-house PgC in Teaching and `supporting Learning many colleagues report that their best learning is in the process of being a student again and gaining an understanding of the stresses, strains and liminality of this process. Perhaps to embody the kindness that Jackie’s research has highlighted, we should all be eternal students. My note to self here is to follow Jackie’s lead and analyse the scheme data I hold on teaching excellence – or ask what do students value?

Jane Headley and Rebecca Payne’s session on exemplars was a great lot of fun! By offering a task to us (the task was – getting your team through a piece of A5 paper) and giving each group a different experience with an exemplar, we were able to feel and experience the use of exemplars. Our team had an exemplar in full, but as a team who wanted to be original (I was just happy to pass, but others wanted to excel) we decided to ditch knowledge of the exemplar and add our own twist. The result was redefining team (after all it didn’t say a human version of your team) and to create a stop motion video. This first hand experience showed me that exemplars can show students that a task is possible and that it can then free up the creative mind to do the task differently. Working in a team, and with an enjoyable task, simply added to the creativity. This point too is something we would do well to remember!

For posterity I have retained a conference programme.

LT_Conference_2017 programme

The only bad thing about the day is not being able to get to all of the sessions. Luckily I have previously heard the other speakers and they are all awesome!

Making digital exemplars

In addition to my usual classroom use of exemplars as a means of familiarising students with the assessment requirements of a specific module, this year I have created a video walk through of an exemplar. Initially this was to enable those who missed the relevant class to catch up on the session, but the approach was welcomed by students who attended the exemplars activity session, as well as those who did not. 

How to create a digital exemplar walk through: 

• Bring up the exemplar on screen after selecting a ‘good’ piece of work

• Read through and use comments in word to annotate the work with points which would be surfaced in feedback, particularly comments related to the assessment criteria (of course!). Comments include things done well, things done less well which could be avoided and opportunities for further detail and development. This tagging process acts only as an aide memoire so that as I create  feedback video I am aware of what I wanted to include. 

• Open Screencast-o-Matic to very easily screen record the work as a video as I re-read it through and talk to each of the tags. ‘This work includes this … which is useful because…’ ‘this work used literature in this way …. It might be better to …. Because ….’. None of this is rehearsed; that would be too time consuming. The resultant video is a commentary on performance.

• The video is uploaded and made available to students.

After using the resource there was some consensus amongst my students that the value was ONLY in listening to the assessment commentary and not specifically in looking at the work. One student described how they listened but did not watch. They then recorded notes about what they should include, remember and avoid. They avoided looking at the work for fear of having their own ideas reshaped. If assessment judgments are ‘socially situated interpretive act[s]’ then the digitised marking commentary may be a useful way of making that process more transparent for students, and indeed for other staff.

I will definitely be including this in future modules.

Handley, K., den Outer, B. & Price, B. (2013) Learning to mark: exemplars, dialogue and participation in assessment communities. Higher Education Research & Development Vol. 32 , Iss. 6.

Action research and its link to UK PSF

How does action research link to UK PSF?

Action research can be a useful strand within a learning and teaching staff development strategy.

How specifically though does this type of practitioner research link to UK PSF?

This is useful to articulate so that

i) Staff developers can be conscious of how to advise colleagues
ii) Those undertaking action research can link their work to the national recognition framework
iii) Colleagues undertaking action research may use the links with UK PSF to further enhance their reflections.

The ways in which I think action research and UK PSF are linked are as follows:
1. A self-review using the Dimensions of Practice can help to inspire topics for focus in an action research project. Effectively the framework can help identify areas of practice or understanding that could be usefully progressed. While self-development is a potential motivator for action research, care must be taken that this aim does not trump the needs of students.
2. Areas of Activity (A1-A4) can be directly enhanced through action research (and with direct benefit to students) e.g. a project to develop inclusive online learning spaces (A4)
3. The act of action research can be a way of contributing to professional development (A5) – although this does require a degree of openness in the outlook of the researcher. Without this open mindedness the project simply becomes a problem solving exercise, rather than something that really impacts individuals and their development.
4. Knowledge ‘about’ practice can be developed through reconnaissance phase of the action research project. This stage is the initial scoping research that helps determine an effective course of action once an issue is identified. It may involve a literature review, collegial discussion and student engagement.
5. If action research is collaborative, as ideally it should be, then the process can facilitate a better understanding of the needs of individual students (V1); this can ultimately challenge personal beliefs about what we think our students like or need.
6. Action research is a direct contributor to ‘V3’ which relates to pedagogic research and scholarship.
7. Sharing action research can start to contribute to the process of influencing others, as is a requirement of Senior Fellowship. This might be through internal institutional events, papers, or other forums (although if you do this, don’t forget to gather evidence of impact as you go … what effect did your research have on others?).
8. Given the link between theory and practice in action research, this form of scholarship is one way to demonstrate point V on Descriptor 2 (Fellowship) and Descriptor 3 (Senior Fellowship).
9. The exploration of ethical dilemmas related to action research can relate the research to the wider context (V4). For example questions about data usage and data protection have origins and implications beyond a practitioner research project.
10. Action research can assist with progression in UK PSF. Themes and issues that might be considered in a small project can be built up in to a bigger body of work with more impact and influence. A small project can sow seeds for something greater.

Bitter sweet TEF reflection

Yesterday I avoided comment on social media about TEF, because I had such mixed feelings about the exercise. We should celebrate great teaching and if TEF helps with this then that’s a good thing, but we need to take care that TEF does not divide a community which is rich in collaboration, mutual encouragement and shared mission. Despite the well reported limitations of the TEF data, universities and individuals have been excited with some outcomes. Being a party pooper I’d question whether we should be so celebratory when good colleagues, in good institutions, doing important work, and in many cases providing excellent teaching on the ground, are now labelled as ‘third tier’. I struggle with that. It seems the institutions who faired less well have been left alone to point out the limitations of TEF, which in the aftermath of the results, makes those speaking out like sore losers. I don’t believe that is fair when we are all aware of the limitations of the measure. In our immediate celebratory reactions we are potentially signaling uncritical acceptance and alienating colleagues in the process. Some collective sensitivity may be part of the answer.

Another thought as the results came in was about what TEF does to existing alumni. I have a degree from bronze, silver and gold rated institutions. By far the most impactful, rewarding and engaging of these was my ‘bronze rated degree’ (from Liverpool, by the way). Does TEF devalue the sense of worth for existing qualifications? I fear it might. As a ‘bronze alumni’ I feel outraged at this label and am clear that it absolutely does not reflect my experience. As we start to categorise universities in this way, we need to think about the consequences for past as well a future students.

Unfortunately the celebration of TEF results is bitter sweet.