Over the last six months I have been looking in to the Undergraduate Viva. Asking questions such as what are the benefits? What makes a good undergraduate viva? and, How can students be prepared for their undergraduate viva? One of the results of this is a guidance document on how to conduct a viva of this type. It may be of interest to others.
I am aware others are grappling with how to define graduate attributes, so I thought it helpful to share the approach that we took. As part of a whole university curriculum review, and a strategy review, we set about trying to identify what it was that the curriculum should achieve. Essentially we asked, what was our goal? Unless we know this any curriculum initiatives would be tinkering. So we asked a very fundamental question, what should a Harper graduate be? This goes beyond simply asking what they should be able to do, and incorporates a sense of self that is needed to deal with a fast changing external environment and this is needed to be resilient for the future. This idea is underpinned by Ron Barnett’s work on working in super complexity. It’s a huge question but one that we answered, I think, in a creative way.
We gathered as many staff as were able to attend to join a room with huge pieces of card printed with a giant graduate. In course teams staff were then asked to build a graduate in their discipline. Using the card as a focus for thinking, prioritising, debate and discussion each team built their own graduate. Of course this informed course level thinking before more detailed discussions got underway about course content. Using post it notes to stick on to the graduate allowed rearrangement, re-prioritisation and change as the group discussions evolved. The views in the room were not formed in isolation since colleagues were involved in both student and industry engagement.
After each team had spent several hours identifying what they graduate would look like in a perfect world, we collated all of the words used by all of the teams. These were then collated and put in to a word cloud creator. The commonality in the lists showed itself as the larger words were repeated across different course areas. After some sorting and filtering it became clear that we did have a collective and common vision of what the graduates of the future should be. This exercise became the foundation of the new graduate attributes. The build a graduate exercise was also undertaken by course teams with students and industry contacts. The word cloud produced is shown below.
The word cloud gave students and staff a visual connection to the exercise that we had taken, and a constant reminder that the definition of ‘our’ graduateness was a collective exercise.
The headline attributes helped to ground the Learning and Teaching Strategy; they provided clear direction as to what our activity should be pointing to. It provided one of the key cascading ideas for strategy and operational policy.
For the curriculum aspects, once we have the broad terms for what a graduate should be, we interpreted each attribute, skill area of understanding for each level of study. This involves some word-smithery and some external scoping to see how others level their outcomes, but it also required an eye on the future. We ended up with was a breakdown of each of the graduate attributes, and a description of what should be achieved each level in this area. A snapshot of the attributes are offered below.
It’s one thing articulating the graduate attributes and specifying them for each level, it is quite another to deploy them as the beating heart of the real curriculum. The first thing that we did was ask course teams to develop programmes that addressed each area at the correct level. Course level engagement forced deeper conversations about ‘what does digital literacy mean in our context?’ ‘where are the opportunities for global perspectives?’ and this sparked the attributes into life. Each programme then mapped where the attributes were met, but this one way mapping was deemed insufficient, as once it is complete it can, in reality, be committed to a top drawer and dismissed as a paper exercise. So we went a step further and requested that modules were individually mapped against the graduate outcomes. This makes it much clearer to students and staff, what skills the module should address. Through validation and scrutiny each module was checked to ensure it really was enabling the development of these attributes, through its content, pedagogy, assessment or independent activities. The next step is to get student to actually consider their progress against the graduate outcomes in a meaningful, rather than tick-boxy way. I’m sure others have taken different approaches to developing graduate attributes, but this sought to be pragmatic and inclusive.
In September 2015 I committed to deliver a thirty-credit module, called The Teaching Practitioner, using A flipped classroom pedagogy. The module is the first of two in a PgC Teaching and Supporting Learning in HE; it is associated with Associate Fellowship of the Higher Education Academy.
My motivation for flipping the classroom was three fold:
- My contact time was limited and therefore moving ‘delivery of content’ outside of the classroom was an answer to a specific timetable challenge.
- In learning and teaching provision of this type I wanted to actively avoid ‘preaching’ or appearing as the ‘authority’. Everyone, without exception, on a work-based programme brings experience and the class dynamic is much more about guiding equals and facilitating mutual learning.
- I would rather place my energies in to discursive, challenging and unexpected contact time, rather than repeat sessions of transmitting content, which can be accessed in other ways.
The pattern of delivery was simply that each week I shared materials to work through, including narrated presentations, videos (commissioned and existent), reading, reflective tasks and then we would gather to discuss. The discussions varied in formality, structure and style as the module progressed. Over the course of the module I learnt a great deal, the key points from my mental list of lessons are shared below.
Essential to do list: Each week I published what needed to be done in advance of the face-to-face class. Importantly the list split out what was essential and what was optional. Participants reported that this was a helpful organizing distinction and allowed better management of their activity. This is something that I would definitely adopt in future modules of any type to act as a pacesetter. Simple, perhaps obvious, but actively encouraging participants to make choices about the level of engagement they can make is a pragmatic way of supporting work based practitioners who have so many competing demands on their time.
Slides not videos: I experimented with the media format of presentational material (pre-class content). The staple across most weeks was the narrated PowerPoint. I found more editing control by using Audacity to record the audio and then drag and drop in to PowerPoint, compared to recording direct in to PowerPoint. Audacity gave me opportunity to edit out any major interruptions with ease (phone calls, door knocks etc). I included some video lectures of studio production quality however participants found them relatively less engaging, with a preference for visuals and audios mixed in together with the ability to more easily navigate the presentation. I was surprised by this preference, but there is no doubt narrated presentations are easier to create.
Don’t force theory: We took a discursive approach to our face-to-face time (which was usually two hours per week). I provided questions and starters and then tried to guide the discussion. At first the conversation was loose, multi-directional, on and off-topic. I worried that we were not being ‘very level seven’ and the participants shared some of these concerns. However an under the surface, a process of sense making was going on; each person, in their own language and terms, through sharing and reflecting on their own experience got chance to reconceive, affirm and evaluate their practice. The explicit linking to theory was a more private activity, which seemed to occur in response to assessment. It was only obvious that this had taken place at the end of the module as discussion and theory were fused. Perhaps the discussions were a shared liminal space in which we muddled through difficult issues, then we went away to individually reflect and make clear.
Facilitation skills matter more than online production skills: My role can be linked to all the activities of a facilitator, including:
- Providing occasional expertise
- Sharing anecdotes
- Collating the issues that we couldn’t solve and referring them to other forums, or mentally ‘parking them’ as knowingly messy
- Archiving ideas (e.g. photographing shared lists and posting them online for future reference)
- Providing clarity as needed
- Providing confidence
- Managing the group dynamics
- Modeling active listening
As we progressed through the weeks, methods for each of these aspects became more developed e.g. creating graphics for summaries, defining the discussion purpose to keep us mainly on task. One thing I did from time to time was add a summary of the discussion as a resource for reference so that everyone had opportunity to revisit key points. This involved simply using my mobile phone and talking through the diagrams that we had created in class such that everyone had a record. This was not onerous at all if done straight after the session while fresh in memory.
Quick and dirty production process: If the model of delivery is going to be sustainable then resources need to be produced within a realistic time frame. By taking a quick and dirty approach to development, those on the programme see the approach as achievable and replicable; it provides accessible modeled practice. For me there is also a really clear sign in this approach that the value of the learning experience is the interaction and not a resource. To avoid perfectionism I never listened to my own presentations after they were recorded other than for a quick sound check.
Shared endeavor: While new roles were not formally defined, we fell in to a more even relationship. I sensed that we were co-researchers (in to the effectiveness of the pedagogy) and co-learners (about all aspects of the programme). We were facilitators and facilitated, rather than ‘teacher and student’. To reinforce this role equality, I tried to be very open about when I was learning too.
Allow choice about levels of engagement: As grown ups, participants face a simple rational choice about whether to engage or attend; sometime this choice is made in light of personal life and professional workload. In the weeks where individuals had not done the preparation for class, no action was taken or penalty applied. This approach relies on a commitment to engage and the rewards are implicit in the design. It also reflects the idea of running a community of equals. The group dynamic needs to be honest about the need for preparation, but pragmatic when this slips. If the facilitation works well then even those who have not prepared should be invited and able to contribute experience, and hopefully then inspired to retrospectively visit the online class.
A human process not a technical one: Flipped classroom may evoke thoughts about complex online tools and an unfathomable methodology of teaching promoted by centres of e-learning and academic development, but for me the experience of flipped classroom is a fundamentally human process which involves a respect the opportunity to explore individual experience and knowledge. It allows social learning and creates space for the discussion of any issues arising that matter to the group. I hope the language around this practice, and the identity of the learning model as slightly exotic, does not take away from the collegial simplicity, which resonates with traditional seminar based learning.
Support for the flipped approach from participants was demonstrated in three distinct ways: i) the adoption of flipped classroom by some group members ii) protest when classes are not flipped iii) outstanding, highly personalized, deeply connected assignments to demonstrate the culmination of meaningful engagement (though I am a little bias on the last point).
If I had a point nine on my list, it would be to keep faith that the approach will pay off, even when there is angst about its effectiveness. That said, when I saw in the module assessments that we had reached our destination (albeit a fleeting one on the way to the next module) I was very relieved!
In considering how to support curriculum design for new programmes I have developed a question framework for course design teams to use to help them to deliberate and discuss the shape of new programmes. It tries to encourage a balance of looking back at what has worked before, and looking forward at how designs could be improved. It encourages discussion in the context of the discipline, but focuses on the underpinning structure of the design rather than specific content.
Reflecting upon what makes a university operationally ready to engage with employers in collaborative provision, I have pulled together a list of “good conditions for employer engagement“. Its purpose is simply to provide prompts for discussion about how institutions can prepare to engage with employers to meet the development needs of the workforce. This list assumes that there is already a strategic level commitment in place and it is of course not exhaustive.
Today I received confirmation that my Senior Fellowship application to the HEA had been accepted. I thought it helpful if I shared a few insights in to the application process with others who might be considering individual entry route.
There are three ways to achieve recognition with the HEA –
- Individual entrance route
- In-house accredited CPD schemes
- Through recognised qualifications (e.g. a PgC which leads automatically to Fellowship)
In 2011 the HEA launched new levels to their recognition scheme (there are now four levels of recognition: Associate Fellow, Fellow, Senior Fellow and Principal Fellow). I suspect many institutions will take a little while longer to develop internal CPD systems for the two new levels of recognition and so for now the individual route may remain the favoured approach for some colleagues.
I completed a Fellowship by individual entry in 2007 and have taken the same route for the Senior Fellowship. The application comprises a 6-7000 word reflection on practice and two ‘case studies’ of practice. It needs to demonstrate all of the areas of activity, all the professional values and all areas of knowledge as described by the Professional Standards Framework. It is not enough to list how you meet each requirement – there is a need to show how you apply the knowledge and values in practice. The biggest challenges in the application process were:
- Managing the time needed (which is significant)
- Going beyond description in the account to ensure sufficient reflection
- Selecting areas of practice on which to reflect
The process requires some detailed planning. The approach I took was to begin with the activities. I simply plotted out what I did against each of the headings listed. So, against ‘assessment and feedback’ I located my roles, projects and practices (present and past), likewise I asked myself what had I done in the area of learning design and developing student guidance and so on for each activity area. Simply creating a list provides the raw material for the reflection.
The next thing I did was to place the ideas in to chronological order so they made sense in terms of my personal progression – of course doing this showed up duplications and sparked additions. Initially I planned to create matrix to ensure I had all values and areas of knowledge covered, but this was quite limiting and made a tick box exercise of the process. Instead I took each activity on my list and reflected by asking a series of questions around each point, including:
- What did I do? (description of the activity)
- Why did I choose to work this way? What shaped the decision? (was it the influence of a colleague, a particular belief, a policy, an engagement with a particular academic idea or theory or case studies from elsewhere). What knowledge and understanding informed this way of working? **
- How was this way of working beneficial to students, colleagues and/or others (including industry partners)?
- What was the impact? How do I know this approach was working well?
- What was learned about working this way? Are there things in future that need to be done to refine this approach further?
**these were the most important questions as they gave opportunity to review both values and knowledge
I then tagged the emerging narrative against each of the framework requirements by adding “(v1, k3)” – these are the labels given to the framework requirements (k = knowledge, v = value). These tags were added where I believed my reflection demonstrated the criteria. By doing this I was able to see where the gaps were. My original draft was lacking in v4 for example and so I was able to track back and ask where, in my activities, did I draw upon this this value?
The case study elements (perhaps these should be re-named since they are more like illustrations of practice) are a thicker, and more focused, description of things that you have done. I considered these to be a zoom lens on two areas of my practice. I could adopt the same approach as for the general narrative but had more space to provide more detailed description and reflection.
During the process it was really important for me to have a critical friend who could chat through the sticking points and offer me feedback. I had hoped the application would take a day if I chained myself to my desk but in the end it was much longer. Overall the process has been valuable (if not a little intense) – it provides a useful opportunity to look back at what has been achieved and I was particularly pleased to see that my values had not waned too much! It was quite motivational to retrace my steps over years of practice and it was also helpful in informing planning for new CPD. While I was frustrated by the time commitment needed, without this dedicated ‘thought space’ the benefits of the reflective process would not have been realised.
After a inspiring Learning & Teaching Forum lead by Professor Chris Rust of Oxford Brookes, I pledged my post session action would be to capture my best bits from the day. So … Some take away points from today’s session ….
- Authentic assessment is an excellent way to encourage engagement by students as it helps to personalise the student’s approach to the task and generates buy in. So rather than offering abstract tasks, like produce an essay on leadership styles, frame it to have a sense of audience and so that it emulates real world situations that the student may encounter. This could be as simple as changing an essay on business planning principles into a presentation of a business proposal to a prospective funder.
- Spot review class activities and feedback to the group – a major efficiency saver and a good way of making feedback a routine. So in a class of 40 students pick out five pieces of work to review and send group feedback based on those which have been seen.
- Getting back to first principles of constructive alignment guarantees some variety. If the learning outcomes are sufficiently varied and, if the assessment lines up to ensure that the actual outcomes are being assessed, this should in itself offer a degree of variety.
- Diversity is good, but care needs to be taken to ensure that the student journey does not become disjointed by variety. Having some repetition of assessment approaches at the programme level ensures that there is opportunity for students to make use of feedback. A balance is to be struck between variety and the opportunity to facilitate growth in students.
- Tutor time is disproportionately spent on housekeeping feedback – Are headings present? Are tables labelled? Is evidence offered when requested? etc etc. A super simple tip might be to have a housekeeping checklist that students complete before submission to deal with all of these aspects, thus allowing time to be better spent on more substantial points of feedback. It works on the principle that answering ‘no’ to any of the housekeeping question prompts a response such that issues are dealt with before submission. Using such checklists, as a routine, ensures that the student takes greater responsibility for their own learning and tutor feedback can deal with more substantial issues. It must be an integral part of the assignment, i.e. will not mark without it, or else it will not be adopted.
- Less is more. With a greater number of summative assessments the opportunity to give feedback which can feed forward is limited by processes, effort spent on the justification of grades and administration. Instead, lose an assessment and gain the opportunity to utilise feed forward on a piece of work. One assignment, but constructed drawing upon feedback along the way. Simple, but brilliant (and a bit more like real life where review on a document would be an entirely sensible step).
- Reviewer is king! It is the act of reviewing more than the act of receiving feedback that can spur interest, new insights and leaps in understanding. Getting peer review embedded within courses is an excellent way of raising the presence and effectiveness of the feedback process. To buy into this we need to lay aside fears around peer feedback meaning a lack of parity in the quality and quantity of feedback received (which may be inevitable), and appreciate the value of the experience of reviewing as where the learning is really at. Liken this to being a journal reviewer – how much is learned by engaging with a review whether good or awful? (Analogy courtesy of Mark).
- Group vivas. Liking this lot and not something I had previously encountered. So simply a group project and attribution of marks depends in some part on a group viva where honesty is, in theory, self-regulating.
- 24 hours to act. In considering the value of formative quizzes, computer aided or class based as an opportunity to engage with knowledge received, we were reminded of the benefits of engaging sooner rather than later. Engagement with formative quizzes (or indeed reflective processes) within 24 hours of a class is much more effective that if left.
- Use audio feedback – it doubles feedback and makes production smoother.
- Future proof feedback plans. Think how SMART devise ubiquity will play out in future. Formative in-class tests may be more efficient on paper for now, but insure efforts by taking a dual pronged approach (online and paper).
- Pool efforts. Whether across course teams, departments or with colleagues nationally, look for efficiency gains in providing formative question banks. Open educational resource banks (e.g. Open jorum), subject centres and commercial textbooks with CDs of instructor question banks may all be sources to consider.
- The uber simple approach of asking students what the strengths and weaknesses of their assignment are can focus minds. Additionally asking them on which aspect they would like feedback creates a learning dialogue and ensures feedback is especially useful.
While all of these points matter it remains that it is most important to review the bigger picture. A major barrier to diversifying assessment and capitalising (in learning terms) of feedback opportunities can be modular structure to programmes. The TESTA Project revealed that “the volume of feedback students receive does not predict even whether students think they receive enough feedback, without taking into account the way assessment across the programme operates”. Volume of feedback or assessment will not improve student perceptions of feedback. Point 4. Above leads to the assumption also that timeliness alone is not enough either. While it is good for module level assessment and feedback to be considered in relation to the ideas above a holistic look at the programme level helps us to understand the assessment journey of the student. How can feedback feed forward? Is their sufficient variety across a programme? Is their repetition? All questions worth asking beyond the module level.