Preliminary findings – Lecturer experiences of choosing and using technology in assessment feedback

Part way through my thesis research I have stood back to ask what is all this data saying? To this end I have produced a pause for thought document about the emerging findings. This is not the finished output, but in creating it I managed to consolidate my thoughts, and in sharing it I hope for any comments that may help refine further analysis or additional data collection.

Please feel free to leave any questions or comments in response.

And thank you to those who have assisted me in setting up the next round of interviews after a recent plea for help.

Preliminary Research Findings available here.

 

Marrying mixed methods and critical realism

As  I edit through my thesis I needed to cut out a short section that I had written about how mixed methods can be highly compatible with critical realism. As my work emerged I did not, in the end, use mixed methods.  This section was hard won and so simply deleting it felt like a waste. Therefore, attached to this post is a short piece about the compatibility of mixed methods and critical realism, in hope that it may be useful to someone else.

Download the short article here: Marrying mixed methods and critical realism

Efficiency, technology and feedback

In considering staff experiences of choosing and using feedback technology, one of the emerging themes has been the differing views on feedback technologies and efficiencies. While the jury is still out on the data and the process is incomplete, my observations are that efficiency can be conceived in different ways in the negotiation of technology. For some efficiency is a primary driver in the decision making process. The search for technology and the refinement of its use is motivated and shaped by the quest for efficiencies. For others efficiencies are a welcome benefit of technology – they are almost an unexpected gift – welcome, but not necessary. Efficiencies also appear to be conceived relatively; rarely are efficiencies discussed without a reference to the relative enhancement gains that can be made through a technology. Wherever there is a time saving there is a tendency to ‘re-spend’ the saved time making still more enhancements to the feedback – adding detail and depth for example. In this way efficiencies become difficult to identify as they are theoretically achievable but in reality they are trumped by the possibility for improvement. Efficiency also seems to be a veto concept for some; it is not a particular concern in the run of practice but is triggered only when a particular technology is likely to encroach other activities or provide an intolerable stress.

Empathetic validity and action research in educational development

As part of my doctoral studies I have recently undertaken an action research project relating to strengthening approaches to feedback practice. Informal reconnaissance led me to believe that feedback practice is very siloed. At the same time in the planning process I encountered a paper by Ball  (2009) who showed that collaborative practitioner centred action research  in itself can bring about the questioning of ones own practice, put simply, discussing the feedback practice of others shines a light on the way that each of us works and we then ask questions of ourselves and review how we might act differently.

Informed by this, my action hunch was that the development of an electronic sharing resource for good practice could be a mechanism for modelling good practice and bringing about transparency. Influenced by Ball, I envisaged that ensuring ownership of this resource by those who would use and populate it could act as a catalyst for critical dialogue around practice.

In seeking exemplary practice to populate the resource it became clear that there were some issues to address first. The project got messy in the way described by Cook (2009).  The intended action therefore was put on hold, and became a second project phase. This was clearly emergent work in progress.

The action research methodology permits these off-piste directions and in my search for good practice to populate the resource, I generated four spin off cycles to explore what is good feedback in this context? How can feedback practice be developed?  What are the barriers to developing good feedback practice? What conditions might be needed for the benefits of practitioner sharing to be realised?

I have take away learning about all of these points but by far the greatest learning from this research has been  the development of empathy with those engaged in the process. I have a much better sense of their experience and in staff development terms this is important for productive ways forward. My data was not vast and my conclusions didn’t add a lot to the already overflowing pool of literature on this topic, but it felt valuable. Trying to justify your research in terms couched in feelings is something that even I, as a self-confessed navel gazer, am not used to doing. In reading around this I was drawn to the work of  Dadds  who described a phenomenon called empathetic validity which refers to “the potential of the research in its processes and outcomes to transform the emotional dispositions of people towards each other, such that more positive feelings are created between them in the form of greater empathy” (2008, p208)

Whether empathy can really be incorporated a project aim I am not clear, I imagine it either happens or it doesn’t, but its benefit for me has trumped any of the the intended consequences of this project.

Ball, E. (2009). A participatory action research study on handwritten annotation feedback and its impact on staff and students. Systemic Practice and Action Research, 22(2), 111-124. doi: 10.1007/s11213-008-9116-6

Cook, T. (2009). The purpose of mess in action research: Building rigour though a messy turn. Educational Action Research, 17(2), 277-291. doi: 10.1080/09650790902914241

Dadds, M. (2008). Empathetic validity in practitioner research. Educational Action Research, 16(2), 279-290. doi: 10.1080/09650790802011973

Apps 2012

Image

As I have progressed through my EdD my ways of working have got a little smarter. There are four apps that have served me well in 2012 for supporting my studies …

1. Reminders (so in the hours where I have too much to do I can remember what they were!)

2. Good Reader – managing my online library downloads and annotating my reading without reams of paper. By far the best reading app I have found (still).

3. Good notes – high levels of functionality, a great jotter and annotator – good for generating diagrams and mapping out thoughts.

4. Splashtop – allows my desktop (including Endnote) to be fully functional from my ipad or phone. Excellent when not wanting to be stuck at my desk.

Two more fab apps (not study related)  for 2012 have been

5. Screenchomp – Jing for the ipad – great for audio visual feedback for students and again this means there is no need to be desk bound.

6. Spelling – my best parenting app! So the kids can input the spelling list for the week and then run the tests until the spellings stick. Very motivational for kids who hate spelling.

Strategy and mission

Having poured over some fairly hard going documentation and policy text books for a few days for assorted reasons I was pleased to stumble upon a refreshing approach to writing mission statements, which I’m sure would work equally well for policy and strategy documents!

OK, so not in a million years will HE documentation ever take this flavour, but wouldn’t it be better if it did!

C-Map

I am asked increasingly about concept mapping software. I have previously favoured iThoughtHD; however, while this is very intuitive it is not so good at enabling inter-label links (something only realised after a little time and intensive usage!). C-Map was recommended to me as an alternative. Though not native to the ipad, it has a greater focus on the links rather than the labels and in turn this helps the author to think about structure, more than the brain dump. It forces the user to clarify: Why is x connected to Y?

“A concept by itself does not provide meaning, but when two concepts are connected using linking words or phrases, they form a meaningful proposition”. (Villalon and Calvo 2011 p18)

C-map is downloadable for Windows and Mac and wonderfully, is free.

Below is my own mind map to demonstrate C-map (though I am confident that there are better examples!!). Click to view.
Lydia's map of learning theory

Villalon, J. and R. A. Calvo (2011). “Concept Maps as Cognitive Visualizations of Writing Assignments.” Journal of Educational Technology & Society 14(3): 16-27.