A UK PSF compatible framework for professional reflection

One ongoing challenge I have is around how to increase the depth of reflection on teaching practice (or indeed other professional practices) within the context of formal development programmes. Sometimes we use models of reflection to assist, including Gibbs, Johns and Greenaway’s models. However existing models, and even free flow writing, have not always yielded in-depth reflections. Based on my own experience of supporting reflection across different professional groups I have summarised three limitations of existing models of reflection.

1) A tendency to focus on iterative improvement with less emphasis on validation of practice

Models tend to steer the reflector to assess any issues that require a change in approach (plan-do-review and variations thereof); change is king. Based on experience, sometimes colleagues find that they don’t need to change but instead they can take value from affirming their practice and recognising what they do as effective or good. Affirmation and confidence in practice are as important as identifying points for change and development.

2) A limited engagement with the idea of governing variables

Reflection models can tend to encourage single loop learning as critical incidents are located and considered. I always encourage anyone reflecting to consider what is within their remit and control, and to focus their attention accordingly rather than locating issues within the practice of others, particularly when this leads to a sense of blame or the shifting of responsibility for personal practice. Nevertheless, it can be very useful for some attention to be given to the constructive consideration of challenging the status quo and the operational norm. New(-ish) practitioners can often assess the context in perceptive ways as they have not necessarily been acculturated and institutionalised. To encourage a focus on the constraints and context of practice is very different than shifting the focus of a micro reflection to others because it may be easier than examining one’s own practice. It means standing back and asking what are the things around me that I need to challenge? (challenge is key here, and the answers may not be to hand, challenge – not change). Possible areas to challenge include policy and established ways of working. Whilst senior staff may be able to act on these realisations, new lecturers (or practitioners in other fields) may be less empowered or confident to take action. However, if institutional staff development is joined up, then the issues raised through these reflections can filter through course leaders and assessors for discussion elsewhere.

3) A tendency to focus on incidents rather than wider periods of personal transformation and growth.

A third issue with existing models of reflection is that they tend to focus on an incident by incident basis i.e. take a critical incident and consider it in depth, resulting in a learning or a change. This approach can be simplistic and fail to make connections between a range of events and practice. The resulting reflection therefore tends to be overly descriptive and sometimes forced. Instead I am now encouraging a ‘compound reflection’ – to look back over a series of events or a time period and consider the resultant personal and professional growth. This is especially powerful for identifying personal learning about practice, and the recognition of evolving beliefs and values. It should also provide a chance to review meta-learning, asking what happened across this period to assist my learning? I am not convinced that this depth occurs on an incident by incident basis.

I am proposing an alternative reflection model to capture some of the points above. Essentially this encourages the focus on either an event/incident or a period of time identified by the reflector (e.g. across one term, or after a CPD programme). In the model focus is drawn to three areas, which align to the UK Professional Standards Framework Dimensions of Practice. Individuals should separately  attend to their activities/practice, knowledge and values/beliefs. The actual dimensions of practice can be used to further frame thinking. For EACH of those three areas stimulating questions can be asked to encourage external or internal conversation. Affirmation, challenge and meta-reflection are all evident.

 

EXPERIENCE Single events, multiple events, or reflections on

career phases

FOCUS 1.     Knowledge 2.     Activity 3.     Beliefs
QUESTIONS (to be applied to Knowledge, Activities and Beliefs) •       Affirmation: What knowledge, practice or beliefs have been reinforced? Where is the evidence that gives me confidence? How can I  share this practice?

•       Question: What questions have been generated? Which should be prioritized?

•       Change: What changes have I made? What is the impact?

•       Research: What information do I need to move forward? Why do I need this? How might I get this?

•       Challenge: What forces constrain my knowledge, practice or beliefs? Which of these can I change? Which might I alert others to? Which must I accept?

•       Meta-reflection: How do I learn at my best? What role to colleagues play in my learning? How can I ensure that I keep learning? What learning routines should I maintain or change?

Of course this is an early attempt at shaping up a framework to assist reflection, so any thoughts by reply are very welcome.

 

Posted in Uncategorized | Leave a comment

Effective feedback through GradeMark

This week I’ve been asked to share my own thoughts on how GradeMark can be used effectively. After some recent student interviews on this topic, here are my top ten points, some of which apply outside of GradeMark too.

  1. Give sufficient emphasis in feedback to content and context as well as structural issues in student submissions. Students sometimes report back that they have too much emphasis on grammar, layout and spelling and not enough on content. Some tutors may have a preference to feedback on what is easier to see.
  2. Avoid annotations e.g. exclamations, highlighting question marks, without explanation for students – they don’t know what short hand annotations mean. Give ideas on what students can do to improve their work in future e.g. you might use a range of literature sources and compare the different ideas, rather than only providing one perspective.
  3. Use the full features of GradeMark to help with feeding forward on how to improve and to provide links to further advice. This is almost impossible to achieve by hand, so use the full range of the tools features.
  4. Wherever possible make sure that feedback can help with the next assignment – know what students will study next, so you can make these links.
  5. Capture4

    Relating comments to criteria

    Use the criteria explicitly in the feedback so that students can see how decisions about their work have been made. You could even use the colour feature within GradeMark to relate comments to different criteria e.g. Pink for analysis, Green for Evaluation, Yellow for structure and writing style.

  6. Avoid using the term “I think ….” While we all know feedback has a degree of subjectivity to it, making links to the criteria should counter the expression of personal preferences to determine marks. Students often feel that their marks are about playing to the preferences of different tutors. This can be countered by consistent use of criteria.
  7. Use summative comments as well as annotation to draw attention to the most important areas for future development and/or any specific issues arising. Students may not always be able to prioritise amongst many annotations.
  8. To speed up your workflow, consider using voice to text facilities alongside GradeMark. This is especially simple on Apple devices where the built in ‘speech to text’ is usable and accurate.
  9. Use common errors and issues from one year to pre-prepare comments for use in feedback in the following session (assuming adjustments in teaching don’t address everything). Because these are prepared in advance the advice can be more detailed and helpful and they can be used in teaching so that students can absolutely see what they should and shouldn’t do.
  10. Capture3

    Personalising a generic comment

    Wherever possible personalise comments; students appreciate the interaction or dialogue through feedback when it feels personalised to them or, when anonymous, to their work – “I suppose because we are all in the same boat, I suppose it wasn’t  really personalized because I suppose a majority of us get the same things wrong” (Student quote 2017). So if you aren’t personalising, advise the students why this is the case. If you do want to personalise there are many ways to do this including:

  • Using an audio comment in addition to generic text comments
  • Adding additional specific comments on to generic comments
  • Including a summary comment in addition to annotation

I’m sure there are other points, and some of these are already well rehersed in literature, though particularly point 1.  is little explored elsewhere. 

 

Posted in Uncategorized | Leave a comment

The tricky issue of word count equivalence

The challenges of managing media rich assessments, or managing student choice in assessment, has been evident in higher education for as long as I have been employed in the sector, and probably a lot longer. Back in 2004, when I worked on the Ultraversity Programme, the course team had an underpinning vision which sought to: enable creativity; encourage negotiation of assessment formats such that the outputs were of use; and, develop the digital capabilities of students (a form of assessment as learning). We encouraged mixed media assessment submissions for all modules. At this time we debated ‘the word count issue’ and emerged with a pragmatic view that alternative media should be broadly equivalent (and yes that is fuzzy, but ultimately this helps develop judgment skills of students themselves).

In the HEA accredited PgC in Teaching and Supporting Learning that I now manage, we assess using a patchwork media portfolio. Effectively there are five components (including an evaluation of assessment and feedback practices, a review of approaches used in teaching or supporting learning and a review of inclusive practices used) plus there is a stitching piece (a reflection on learning). The assessment brief describes what the students should show, but it is not prescriptive on the precise format. Each element has a word guide, but this should be used by those working with alternative media as a guide to the size of the output and the effort they apply.

wordcount

Where students opt for media rich formats, they are asked to decide on equivalence. Close contact in class sessions provides a guiding hand on judgment, critically with peer input (‘yes, that sounds fair’). Techniques to assess equivalence include taking a rough ‘words per minute’ rate and then scaling up. I have had other items such as posters and PowerPoints, again, I ask them to use their own approximation based on effort. Because the students in this particular programme are themselves lecturers in HE, there is a degree of professional reflection applied to this issue. We don’t ask for transcripts or supplementary text when an individual submits an audio or video format, because it can add considerable work and it may be a deterrent to creativity.

Media experimentation within this programme is encouraged because of the transformative effect it can have on individuals who then feel free to pass on less traditional, more creative methods to their students. I asked one of my students to share their thoughts having just submitted a portfolio of mixed media. Their comments are below:

My benefits from using media were;

  • Opportunity to develop skills
  • Creativity
  • More applied to the role I have as a teacher than a written report would have been
  • Gave ideas to then roll out into my own assessment strategies, to make these more authentic for students
  • Enjoyable and I felt more enthused to tackle the assignment elements

But I wouldn’t say it was quicker to produce, as it takes a lot of advanced planning. And, it was tricky to evidence / reference, which is a requisite for level 7. This is where I fell down a little.

I judged equivalence with a 60-100 words per minute time frame for narrative, and / or, I wrote the piece in full (with word count) and then talked it through. I think the elements that I chose to pop into video were those that were more reflective, and lent themselves better to this approach. With the more theoretical components, where I wasn’t feeling creative or brave enough to turn it into something spangly, I stuck with the written word. The exception to this was the learning design patch, where I wanted to develop particular skills by using a different approach.

This student’s comments very much match up with comments made back in 2009, by Ultraversity students who reported “without exception, felt that they had improved their technical skills through the use of creative formats in assessment” (Arnold, Thomson, Williams, 2009, p159).   Looking back at this paper I was reminded that a key part of managing a mixed picture of assessment is through the criteria, we said “In looking at rich media, the assessor needs be very clear about the assessment criteria and the role that technology has in forming any judgments, so as to avoid the ‘wow’ factor of quirky technology use. At the same time he/she must balance this with the reward of critical decision-making and appropriateness in the use of technology. Staff and student awareness of this issue as well as internal and external quality assurance guards against this occurrence” (p161). This is exactly the approach taken within the PgC Teaching and Supporting Learning. Tightly defined assessment criteria have been very important in helping to apply consistent assessment judgments across different types of submission.

If we want to receive identically formatted items, which all address the learning outcomes using the same approach, then of course mandating a single format with a strict word count is the way to go. But if we want to encourage an attitude to assessment which encourages creativity in new lecturers, and which acts as a development vehicle for their own digital skills, then we must reduce concerns about word counts and encourage junior colleagues to develop and use their professional judgment in this matter. The student quote above shows the thoughtful approach taken by one student to address the issue for themself.

Frustratingly, even by using word count as the reference point for parity we may ‘other’ some of the more creative approaches that we seek to encourage and normalize, but ultimately wordage has long been the currency of higher education. It is good to see some universities being pro-active in setting out a steer for equivalence so that individual staff do not feel that they are being maverick with word counts when seeking to encourage creativity.

Posted in Academic Practice (PgC Teaching & Supporting Learning), Patchwork text & Patchwork media, Technology, Uncategorized | Tagged , , , | Leave a comment

Towards Inclusivity

A recent HEFCE blog post reminds of the need to continually consider inclusive practice in HE. Many universities are responding to the need for inclusivity with a range of policy approaches, guidance documents, suggestions for best practice and the internal publication of student data to further make the case for change. In looking at the recent blog post, and the recently published Inclusive Teaching and Learning in Higher Education as a Route to Excellence, the focus of inclusivity remains predominantly with disability. Of course this dimension of inclusive practice is enormously important; as explained in the blog, there are more students with disabilities entering HE, the achievement of disabled students is below peers when there is no additional funding in place, and that funding is being cut. Morally this is wrong and action is needed. Nevertheless there are other complex dimensions in the inclusive landscape. Groups of students that could under perform on their potential include specific socio-economic groups, students from an educational background that is ‘different’ than the majority (especially students on BTEC routes), and BME students.  My main concerns when I read about inclusive practice are i) the full spectrum of issues associated with inclusivity are not getting coverage ii) a sense in the public discourse that the sector is addressing inclusivity as a result of a funding change, and not out of a moral responsibility to all students iii) the deep-seated sense of othering that occurs as a result of ii).

In considering how to approach inclusivity, the philosophy of universal design is appealing; essentially in this, teaching and learning is established to enable all students (or as many as possible) through anticipatory approaches. In a universal design approach all education should be set up to encourage all people to access provision and reach their real potential. While this is a wonderful vision, the reality of retro fitting design principles on to established curricula is hugely complex. 

Recognising the pragmatic limits of universal design, I remain concerned that  a superficial approach to inclusivity is emerging in pedagogy. Yes we can get assignment briefs out earlier as required, yes we can post material on the virtual learning environment if that is university policy, and yes we can make sure reading lists are up to date. These points are important, but they are but small pieces of a large jigsaw. For a sustainable, deep routed and sincere approach to inclusivity, a more holistic approach is needed. I am proposing four levels of action to make for real inclusivity. These are summarised below. This is not exhaustive of course. 

Ways to progress inclusive practice:

  1. Rules

Rules and guidelines can relate to a whole manner of aspects of inclusive practice including: Posting lecture notes or slides in advance of classes to help orient students who may wish to read through the content before class to address any areas of underpinning knowledge that their own education did not afford them time to explore; allowing students to record classes on their personal devices; ensuring that assessment briefs meet the required standards of accessibility; using appropriate instructional design layouts for online spaces; and, using minimum font sizes for visual presentations. All of these and many more rules can be implemented. However, focusing only on this type of approach to bringing about inclusivity feels like an extension of a deficit model where staff must behave in a certain way to accommodate groups of students. While the rules are important, and these types of practice are essential, they can be received as yet another bureaucratic thing to do. This approach in isolation can work against fostering a deeper culture of inclusivity. It encourages a surface approach to the issue. A much more holistic and deeper approach is required to make a real, long lasting difference to student learning.

  1.  Developing student learning skills

Students can be supported to self-help and to develop skills that empower them in their own learning. If students learn how to learn, even when they are lacking a specific set of skills to help them thrive in higher education they will not be phased and will be able to progress. Well-formed personal development programmes, and an attention to the skills of learning alongside taught ‘content’ are essential to empowering students with the study skills necessary to overcome any barriers to learning. Supporting students to develop skills in note taking, critical reading, listening, writing in different genres, revision, exam technique, project planning, making the most of their learning routines etc etc. is an significant component of creating an inclusive learning environment.

  1. Developing inclusive mindsets through open and honest engagement

Staff in HE are at times challenged by the diversity of the student body. The recent Times Higher Staff Survey gives some insight in to some lecturers’ frustrations, with a sense of weakening standards, ill prepared students and a lack of student work ethic. Perhaps these concerns are inevitable and are not so different to the ones I heard sixteen or so years ago when I began a career in HE, but they do sit awkwardly in an age of inclusive practice.  There is a complex academic psychology around inclusivity. Many/most lecturers will have academic excellence behind them; they survived and thrived in a system of learning that allowed them to ‘come good’. When these systems change, some teaching staff experience the disruption of genuinely held views, for example about standards, and about the balance of effort between lecturer and student. A rift emerges between privately held personal beliefs, histories and values, and the expectations for teaching practice. Even those who champion inclusivity may still have repressed  concerns about some specific issues. What is said publically, may still be in conflict with some inner feelings. There is then a need to provide a frank, challenging and respectful dialogue in higher education institutions if private theories are going to work with public discourse. Rules and skills training programmes will otherwise be undermined by the occasional, but damming, careless comment, or the unquantifiable look of exasperation. If inclusivity is to become more embedded, then the dual discourse (under and over the radar) needs to come together. Mindsets can’t be forcibly steered towards embracing inclusivity; but conditions of openness mean that deep-seated beliefs can be aired and held up to debate. Hocking, back in 2010, noted “the need for shifts in negative beliefs about, and attitudes towards, student diversity that currently inhibit the development of inclusive learning and teaching”. I’m suggesting here a new level of openesss about the reasons that some may appear negative, some empathy towards these deep seated views and dialogue to engage with some of the underlying issues which prevent a real mindset shift.

  1. Praxis

Praxis is used here to mean values driven, living, self-reviewing, sincere practice; it is more heartfelt than just practice! If inclusivity is to be a way of working, rather than a set of steps, a process model or a policy discourse, then it needs to become a way of working and thinking. I don’t think this is as much of a change as might be assumed. Most lecturers that I have encountered want to put students first. They want them to succeed and reach their potential. That is it. That is the cornerstone of inclusivity. The debate and politicisation and the connection to our own beliefs stemming from personal history falls second to the simplistic aspiration to help others do well. When we go back to first principles, the praxis of inclusivity is very simple: How can I assist all students to succeed?

Posted in HE, Inclusivity, Learning to learn (meta-learning), Uncategorized | Tagged | Leave a comment

9 Things to do with Assessment Rubrics

I’ve used rubrics in assessment marking since I first held an academic role some fifteen-ish years ago. For me, rubrics are an essential tool in the assessment toolkit. It’s important to recognize that they are not a ‘silver bullet’ and if not integrated in to teaching and support for learning, they may have no impact whatsoever on student engagement with assessment. I am therefore trying to collate a list of the ways in which rubrics can be used with students to enhance their performance, help them grow confidence and to demystify the assessment process. My top nine, in no particular order, are as follows:

  1. Discuss the rubric and what it means. This simply helps set out expectations and requirements, and provides opportunities for clarification.
  2. Encourage students to self-assess their own performance using the rubric, so that they engage more deeply with the requirements of the assessment.
  3. Encourage students to peer assess each other’s performance using the rubric, leading to further familiarization with the task, as well as the development of critical review and assessment judgment skills. This also allows the seeding of further ideas in relation to the task, through exposure to the work of others.
  4. Get students to identify the mark that they are aiming for and re-write the criteria in their own words. This sparks discussion about the requirements, flushes out any issues needing clarity and can result in students raising their aspirations (as the ‘assessment code’ is decrypted there are moments of “If that’s what it mean’s … I can do that”).
  5. Facilitate a negotiation of the rubric. Where full student led creation of a rubric is impractical, or not desirable, a tentative rubric can be presented and negotiated with the class. Students can have an influence on the coverage, the language, and the weightings. As well as familiarizing with the requirements, this allows a sense of ownership to develop. In my own experience rubrics are always better for student negotiations.
  6. Undertake a class brainstorm as the basis for the rubric design. Ask what qualities should be assessed e.g. report writing skills, then identify what this means to students themselves e.g. flow, use of literature to support argument. Then use this list to develop a rubric. It is a form of negotiation, but specifically it allows the rubric to grow out of student ideas. By using student language, the criteria are already written in a form that is accessible to the group (after all they designed the key components).
  7. Simply use the rubric as the basis for formative feedback with students to aid familiarity.
  8. Use the criteria to assess exemplars of previous students’ work. This will have the benefits of familiarity, developing assessment judgment as well as sparking new ideas from exposure to past students work. Of course this can be further developed with full sessions or online activities built around exemplar review, but the rubric can be central to this.
  9. A rubric can be partially written to offer space for choice. Leaving aspects of the rubric for students to complete leaves room for students to show their individuality and to customize tasks. Rubrics don’t box-us in to total uniformity. Recently I created a rubric for a research project and left space for students to articulate the presentational aspects of the criteria. Some students filled in the rubric to support the production of a report, others a poster and others a journal article.
img_20160908_152105

Using a class brainstorm to form the basis of a rubric with criteria relating to reflection 

I have only included approaches that I have used first hand. I’d like to build this up with the experiences of others; if you have additional suggestions please do let me know.

Posted in Academic Practice (PgC Teaching & Supporting Learning), Assessment, HE, Uncategorized | Leave a comment

Undergraduate Vivas

viva

Access the guide by clicking the image above

Over the last six months I have been looking in to the Undergraduate Viva. Asking questions such as what are the benefits? What makes a good undergraduate viva? and, How can students be prepared for their undergraduate viva? One of the results of this  is a guidance document  on how to conduct a viva of this type. It may be of interest to others.

Posted in Assessment, curriculum, Uncategorized | Tagged , | Leave a comment

The process of defining graduate attributes

I am aware others are grappling with how to define graduate attributes, so I thought it helpful to share the approach that we took. As part of a whole university curriculum review, and a strategy review, we set about trying to identify what it was that the curriculum should achieve. Essentially we asked, what was our goal?  Unless we know this any curriculum initiatives would be tinkering. So we asked a very fundamental question, what should a Harper graduate be? This goes beyond simply asking what they should be able to do, and incorporates a sense of self that is needed to deal with a fast changing external environment and this is needed to be resilient for the future. This idea is underpinned by Ron Barnett’s work on working in super complexity. It’s a huge question but one that we answered, I think, in a creative way.

BAG.png

Resources from the ‘build a graduate’ workshop

We gathered as many staff as were able to attend to join a room with huge pieces of card printed with a giant graduate. In course teams staff were then asked to build a graduate in their discipline. Using the card as a focus for thinking, prioritising, debate and discussion each team built their own graduate. Of course this informed course level thinking before more detailed discussions got underway about course content. Using post it notes to stick on to the graduate allowed rearrangement, re-prioritisation and change as the group discussions evolved. The views in the room were not formed in isolation since colleagues were involved in both student and industry engagement.

 

After each team had spent several hours identifying what they graduate would look like in a perfect world, we collated all of the words used by all of the teams. These were then collated and put in to a word cloud creator. The commonality in the lists showed itself as the larger words were repeated across different course areas. After some sorting and filtering it became clear that we did have a collective and common vision of what the graduates of the future should be. This exercise became the foundation of the new graduate attributes. The build a graduate exercise was also undertaken by course teams with students and industry contacts. The word cloud produced is shown below.

The word cloud gave students and staff a visual connection to the exercise that we had taken, and a constant reminder that the definition of ‘our’ graduateness was a collective exercise.

wordle.PNG

A first workshop output on defining graduateness

 

Capture2.PNG

The final version of the graduate attributes 

 

The headline attributes helped to ground the Learning and Teaching Strategy; they provided clear direction as to what our activity should be pointing to. It provided one of the key cascading ideas for strategy and operational policy.

 

For the curriculum aspects, once we have the broad terms for what a graduate should be, we interpreted each attribute, skill area of understanding for each level of study. This involves some word-smithery and some external scoping to see how others level their outcomes, but it also required an eye on the future.  We ended up with was a breakdown of each of the graduate attributes, and a description of what should be achieved each level in this area. A snapshot of the attributes are offered below.

Capture2.PNG

It’s one thing articulating the graduate attributes and specifying them for each level, it is quite another to deploy them as the beating heart of the real curriculum. The first thing that we did was ask course teams to develop programmes that addressed each area at the correct level. Course level engagement forced deeper conversations about ‘what does digital literacy mean in our context?’ ‘where are the opportunities for global perspectives?’ and this sparked the attributes into life. Each programme then mapped where the attributes were met, but this one way mapping was deemed insufficient, as once it is complete it can, in reality, be committed to a top drawer and dismissed as a paper exercise. So we went a step further and requested that modules were individually mapped against the graduate outcomes. This makes it much clearer to students and staff, what skills the module should address. Through validation and scrutiny each module was checked to ensure it really was enabling the development of these attributes, through its content, pedagogy, assessment or independent activities. The next step is to get student to actually consider their progress against the graduate outcomes in a meaningful, rather than tick-boxy way. I’m sure others have taken different approaches to developing graduate attributes, but this sought to be pragmatic and inclusive.

Posted in curriculum, Uncategorized | Tagged , , | Leave a comment