Category: Education Research

This is research that I, or colleagues do about education. I mainly research veterinary education, but am interested in broader higher education research as well.

Standard Setting Non-Existent Exams

I wrote a blog about those #SQAresults, #covid, the Scottish education system, and tried not to panic about what’s going to happen next academic year …

JillyLeave a comment

The Scottish Qualifications Authority (SQA) released its exam results this week to huge uproar. This is a fascinating, and horrifying glimpse into what might await us in higher education, and the rest of the UK. Let’s talk about it. 

The SQA Higher exams are sat in 5th (15-16 years old)  and 6th year (16-17 years old), and are typically the exams that get you into university. Typically, the maximum you can sit in a year is 5. I have 6 Highers, because I sat 5 in 5th Year and 2 in 6th Year (and failed my Higher Psychology – a discussion for another day). There are also Advanced Highers, which we won’t discuss here. 

To get in to do Zoology at the University of Glasgow, you now need 5 A Highers at the end of 6th Year (it was easier back in my day!).

You sit a preliminary exam around Christmas time (the prelim) which are, to the best of my knowledge, set by the individual school based on what has been taught so far. The actual Higher paper, sat in May, is held at the same time and same place with the same paper across the country. This year, students could not sit their Highers, and so the SQA asked teachers to estimate what they thought students would get instead. 

The majority of students who sit these exams are aged 15-18 years old. Over the past four years, 76.8% (+/-1.2%) of students aged 15-18 have achieved an A, B or C grade in their Highers. This year, teachers estimated that 88.9% of this age category would achieve an A, B, or C grade. 

The SQA had a problem. 

The teachers estimates would have meant a 12% point rise in the number of students across the board who received a A-C Higher grade. Why were the teachers estimates so high? What was the SQA going to award students? Could the SQA use the students’ last exams, a prelim that wasn’t standardised across material or paper, to fairly discriminate between ‘excellent’ and ‘satisfactory’ students? What were they going to do?

Option A: Use the teachers estimates

Teachers were told to guess at what their student could do on their best possible day. This, I think, is a crucial mistake in the story, because best possibly days are rare, and have big impacts on performance. I rarely ever had my best possible day on my exams (see my failed Higher Psychology). I expect teachers also felt very sorry for students, and I expect they wanted to support students through this. I would not be surprised if a few schools leant on their teachers to whisper “hey, we could do with some better results this year”. The result? The estimates were far out of line of normal exams. 

If the SQA awarded the estimated grade, they would devalue the exam and the accreditation. This is complicated because the SQA is also the first national examining body in the UK to release grades, thanks to Scotland’s early summer. Every year, we get stories about how grade inflation is making exams easier to pass, and their results harder to trust. These stories arise from creeps of 2 or 3% points. 12% points would have been scandal.  Scottish students would have found it difficult to use those grades to demonstrate their ability, and access to university may have been a challenge. The SQA may have feared that other examining bodies would take a different line, and they would disadvantage Scottish students by being perceived as lenient, who knows? Certainly the rest of the UK is watching Scotland right now. 

Option B: Use the prelim grade

The next solution may have seemed logical – why not use the last exam the students sat? The one that they would have based any appeals on in a better year? (My Higher Psychology prelim was a C if I remember correctly. The appeal went nowhere). 

Exams are a pretty poor way of assessing students. The one thing we can agree on is that you can be broadly sure the right student is sitting in the right seat (ehhhhh), and that every student is seeing the same paper at the same time. At a national level, that requires a massive amount of coordination. It is a phenomenal amount of work to ensure that the Higher Psychology paper I sat in C201 in Park Mains in 2004, is the exact same paper that the other 2778 students were sitting. That when I left, the first moment I could, enough time had passed that I wasn’t likely to be texting my pal in Stornoway the answers. That me and the other 826 students who failed that paper were all fairly marked. It is an exercise in logistics that prelims, which are taken from past papers (in fact, I think I knew exactly which past papers were being used in my Psychology prelim), and are dictated at the level of the school, cannot match up to. 

Again we come back to standards. If the students didn’t all sit the same exam, how can we be sure that these 2020 grades are the passport to the future our schooling system is built on?

Option C: Standard set

And so the SQA took a third road. If about 77% of students usually achieve an A-C grade, then we have no real reason to assume that in a normal year, about 77% of students wouldn’t achieve the same.

But therein lies the rub. The SQA did not take the average of everyone – it took the average of your school, perhaps hoping to smooth over that prelim issue a little. Unfortunately . . . exams are a really, really terrible way to assess students, and consistently students in lower Scottish Index of Multiple Deprivation (SIMD) categories, perform poorer. If you’re in the poorest 20% of the population, you are probably going to a school in a deprived area, with other poor students. Historically, your school will do poorly . . .

And this is what the data shows. 

Most peoples scores were inflated above the usual. Most peoples scores were brought back in line with what their school would likely do. Some very bright students in poor areas have probably done very poorly. Some middling students in very good schools may have benefits. There has been a lot of anger about this: 

And some more big picture observations

https://twitter.com/mgshanks/status/1290650545758371847?s=20

Model Answer: So what do we do?

The Scottish Greens have issued a ‘no detriment’ petition, which I have signed. This petition proposes that students should at least achieve the grade they achieved at their prelim.  But I actually don’t think this is a good answer either. 

The Scottish Government have assured students normal appeals procedures will go ahead, taking prelims into account, but I know from personal experience this doesn’t always get you the result you want, and up the page we just said prelims weren’t standardised, so . . . what do we do?

These exams didn’t happen. Even if they had, they would have been as shit as they always are in terms of equity, diversity and inclusion. COVID will disproportionately affect students in deprived areas, so why are we trying to pretend that four or five letters besides someone’s name, plucked from the aether, can tell us anything about these students abilities?

If I was in charge of university admissions, or had the ear of parliament and the SQA, I’d be advocating for “NULL” in those fields. I’d be advocating for more holistic assessment of incoming students to uni, much like Multiple Mini Interviews in medicine and veterinary medicine, and I’d be advocating for Scotland to take the lead here, because we need to fix this issue. We could be Finland, but we playing.

Exams are shit at assessing anything but whether a student can sit an exam. I don’t set exams in my courses at uni for this very reason, instead I set skills-based assessments wherever I can. I’m not perfect at this, and I could do better. I’ve recently had interesting conversations on twitter about whether we in the UK have an overly aggressive quality assurance approach when it comes to exams, and flexibility in QA this year is something I was firm we had to raise in our 10 Simple Rules paper. But I do like the Scottish Credit and Qualifications Framework, I like what it tries to standardise in terms of assessment throughout all levels of Scottish education.

I just don’t think we should pretend students have sat exams that they haven’t.

Covid fucking sucks. 

You can find my visualisation code over on github.

Chronicles of Athena: Chatty Theenie

I’m training Athena how to use Augmented and Assisted Communication tools (talk buttons!) Want to know more?

JillyLeave a comment

Have you heard of Hunger 4 Words? Christina Hunger is a speech pathologist who has started using Augmented and Alternative Communication techniques to help her dog communicate. You can read about Christina and her lovely dog Stella over on Christina’s website here.

Needless to say, I was immediately fascinated.

Christina has a great introductory post here which I recommend you read to verse yourself in AAC. Athena certainly is able to express herself, often with different vocalisations, but often with behavioural cues. And she’s pretty good at understanding what we mean with our vocal and behavioural cues. Things like ‘food’, ‘play’ and ‘bedtime’ are all easily communicated between us.

We know that Athena will learn a behaviour – she has a whole host of ‘tricks’ that she will do (despite me being a terrible trainer). So why am I interested in button pushing? Well I want to know if Athena can generalise and predict.

If Athena knows what ‘ball’ is, can she press ‘ball’ to tell me that’s what she wants, even if she can’t see it? Would she ever be able to press ‘food’ + ‘ball’ to tell me she wants her puzzle-feeder filled? Would I ever be able to say ‘food’ + ‘ball’ and would she be able to say ‘no’ + ‘food’ + ‘hedgehog’?

To be honest, I don’t believe she ever will. I think that’s a step too far for a cat. I don’t think I’ve ever seen scientific evidence that a cat can conceptualise of two different futures and choose one or the other. I think they’re more reactive than that. But one of the things I love about social media is how it can give us access to larger sample sizes, to more information, to more examples of what’s possible.

So I’ve started Theenie on her AAC journey. If you’re interested in following her, she has her own Instagram now!

https://www.instagram.com/p/CDOwdtcnv0L/?utm_source=ig_web_button_share_sheet

Reflecting on Values

If nothing else, the impact of COVID19 is going to make people think about how they teach. Now’s a good time to think about how we work to the UK Professional Standards Framework

JillyLeave a comment

By now you will have read and no doubt commented on our 10 Simple Rules for an Online Pivot, after all it’s been preprinted for over a week, and time moves fast these days.

You have now sought out my blog for more of my intelligent wisdom on all things pivoting.

Sadly, my coauthors are not here to edit me down or cover for my ineptitude, so you’re stuck with me today.

Something we gently hinted at in our 10 Simple Rules, but didn’t have the time or indeed the energy to go into detail on, was the idea that monitoring and evaluation of courses was going to be exceptionally challenging over the next academic year. I think this also feeds into what we mean by ‘evidence’ in higher education, which is a conversation I have with my mentees for the Advance HE Fellowship Accreditation Scheme a lot.

Absolutely everyone teaching through this experience should be able to go for one of the Advance HE Fellowship levels after this. Associate Fellow, Fellow, Senior Fellow or Principal Fellow. The four different roles aren’t so much a progression up the ranks, but rather a reflection on the different ways you can teach. At R(D)SVS for example we’ve gone big on getting people to Associate Fellow level, particularly our clinical staff including vet nurses, and lab and teaching techs. The scheme is flexible, and allows our staff to get recognition for example as to how they manage their learning environment (you ever tried to manage 20 UGs in a cow shed?), even if their teaching role doesn’t include aspects like assessment and QA. Fellows are likely lecturers, Senior Fellows are supporting others to teach, and Principal Fellows supporting the sector and the development of their colleagues. 

As clinicians and STEMM practitioners, my mentees often take a narrow view of evidence, in that it has to be somehow quantifiable and testable. Perhaps we’ve spent too much time teaching null hypotheses. When my mentees are writing their reflective accounts of practice, I often spend time asking them to broaden their definition of evidence of teaching. It can be, I assure them, your  reading of free text comments in course evaluation questionnaires, your student feedback, even your feelings of how the course went.

The reflective aspect of Advance HE Fellowship follows the UK Professional Standards Framework. In the reflective part, you often focus on the Values of the framework, which are:

  • V1 Respect individual learners and diverse learning communities
  • V2 Promote participation in higher education and equality of opportunities for learners
  • V3 Use evidence-informed approaches and the outcomes from research, scholarship and continuing professional development
  • V4 Acknowledge the wider context in which higher education operates recognising the implications for professional practice

You demonstrate these across all areas of activity that are relevant to you, and with the core knowledge you have. People going at different levels of the award will use them differently, but I often find that people struggle to identify how they use the values. Well today I taught my first pivoted class. I think this one probably counts as ‘non emergency pivoting’ as we had a bit of time to think about it. I thought it would be useful for folks to see an example of reflection here for teaching. You can see where I highlight how the values affected my choices, and how I use ‘evidence’ in this context. 

This class was an SCQF Level 11 tutorial on data literacy skills, and was the first time I had taught this material as I was new to the course this year. It is very similar to an on-campus course I teach at SCQF Level 8 and an online MSc I teach at SCQF Level 11. In addition, this teaching is being split across two staff members, so needs to be consistent. I teach this kind of material quite differently on campus and online, so it was challenging to receive a new tutorial set with limitations on what I can change and how I can influence the classroom. I know from both other courses that this subject is sometimes challenging and frustrating for students, partly because students are often at very different stages with these skills, and being able to check with each individual where they’re at is a key part of the teaching. I have seen this in course feedback, and challenges in teaching data literacy is often discussed in the literature (Kross & Gau, V3). In my preparation I found myself wondering how many students would be learning with only one screen, and so balancing the computer task at hand with the handout and e-learning software. Having had several one-to-one interactions with students during this period, I’d noticed that workspaces were often not ideal, in bedrooms, or in shared workspaces with family (V1). I decided to mirror the handout on the e-learning software presentation, and when screen-sharing I would share a small window to ensure detail was preserved when viewing on a small screen (sharing my large monitor for example would result on very small text if viewing on a laptop screen, I had discovered this during a run through with a colleague, V2, V3). During the session I was keen to manage expectations and model the practice I wanted to see, for example I highlighted to the students that this was not ‘typical’ teaching and I would appreciate their feedback on what worked and didn’t work, and I told students that as I was broadcasting my mic a lot during the session, they may hear some background noise as I was working in a shared space. I wanted to ensure that reasonable levels of background noise, such as pets or children engaging with the students, was not something to be worried about during the session. There is a lot of debate at present regarding what a good learning environment looks like, and I have seen people take stricter approaches to effective working environments (V4), however in my previous experience with teaching this topic, particularly with online learners, it was more important for me to create a relaxed and welcoming environment, particularly for these students who are having difficulties learning online. In future I might want to re-evaluate this approach, particularly as some students may find their working style evolves with more experience at distance learning, and I will continue to monitor the informal feedback from classes and colleagues’ experiences to make that decision. 

I hope that example of a worked reflection, mapped to the values, will be useful for anyone worked towards Advance HE fellowship. What I did leave out was a feeling of intense sadness after I was finished. I felt homesick for my campus, my home, in a way I wasn’t expecting. What’s coming up is going to be hard for all of us – but if this is the push you need to finally recognise the active choices you make when you teach, well I’ll count that as a win.

Behind the Paper: Discipline Based Education Research for Animal Welfare Science

I’ve been working in education research longer than I worked in animal behaviour. Who knew?

JillyLeave a comment

Note: This has been sitting in my drafts for weeks and isn’t getting any more written, so here it is!

I’ve been meaning to do more ‘behind the paper’ posts whenever an article gets published, but I keep forgetting. Oops. One of my recent papers, Discipline Based Education Research for Animal Welfare Science has motivated me to start up the practice again.

This paper is somewhat of a Dear John letter to my former career. You see, as of 2020, I’ve been working in veterinary education research longer than I worked in animal behaviour and welfare. Moving disciplines from ethology to veterinary education was more than a little scary, and committing myself to work in a truly interdisciplinary space was a dive into the unknown.

DBER for Animal Welfare Science marks my sixth first author paper in education research, versus my four ethology papers and my one human-animal interactions paper. In terms of time and experience, I’ve been working in education longer than I was an ethologist. 

I have a couple of reflections on changing disciplines from your PhD studies that I thought others contemplating a similar move would find useful. 

The Deficit

There’s no way of getting around it – when you jump disciplines you have a massive knowledge/reputation deficit that you need to take time to recover. I’ve been exceptionally lucky in the team I ended up in, and I was supported during that year where my productivity dived and I was getting my teeth into some gigantic projects. I have two particular projects that I think of as my second PhD, one of which has also recently been published (here) and the other I’m currently writing up. The sense of scale on those tasks felt comparable to writing a PhD, not least because you’re doing them on top of the day job. You just don’t need to do all the ‘learning to research’ part of the PhD. You already  have your workflows and your skills, and its just translating them to a new context. 

Another aspect of the deficit though is progression. I ‘only’ made it to the Lecturer position in August 2019. My friends who stayed in the same field outpaced me in earning capacity relatively quickly. I’ve had many people assuming I’m further up the academic ladder than I am. Again, I’ve got a very supportive team around me who were supportive of me taking on responsibility before I was necessarily ‘supposed’ to. I did encounter the odd person who thought a Research Fellow shouldn’t be doing whatever thing I was off doing because I wasn’t ready yet. And I am the kind of awful person who gets bothered by that. I like being recognised for what I do. Know your value, and don’t be afraid to fight for recognition as an interdisciplinary researcher. 

The Culture

Something I didn’t expect when I switched disciplines was the need to take a long hard look at myself. The cultural biases I had as an ethologist only became clear to me when they were smashed up against the norms of educational research. This was really useful for me in a lot of ways, but it was also humbling. I now sometimes find myself gritting my teeth when I hear ethologists repeat those biases. Although by the same token, I see what education researchers ‘think’ they know being spouted forward when I’m able to say “actually, in this other field we do …”

Of course you soon develop new biases and ways of thinking. Something that gets my goat very badly is how ‘interdisciplinary’ is bandied around very freely.  I don’t think you’re interdisciplinary until you’re jumping across a ‘purity’ level. Two forms of biology no longer cut it for me. This makes me very fun at departmental meetings and wins me lots of friends. 

The Goods

I have been able to take things from one field into the other. For example, we have a really interesting PTAS project looking at human behaviour using an ethology lens. And of course DBER is part of it.  

I’m also so much happier in DBER than I was in ethology. This field always appealed to me, but felt like such a strange jump after I started investing in my career path. I started my PhD, not really out of any great love of the subject, but because it was 2009 and a paid job during a recession. I came to love it, and I don’t regret it at all, and more importantly I’m better at what I do now because of that time I spent in another field.

I truly believe that academia is going to experience some big changes soon. I believe that the old postdoc model is dead and careers like mine will become more common place. If you do find yourself staring at a frightening opportunity in the next few months, I can’t tell you what to do. But I can say that I have no regrets, and being interdisciplinary is the happiest I’ve ever been

Pivot!

So you’re pivoting to online teaching.

JillyLeave a comment

This is a blog about pivoting face to face university teaching to online – in case that turns out to be useful in the forseeable future.

Some of this may be Edinburgh specific. Its mostly based on my experience, and is meant to be about designing your teaching activities and managing your teaching environment.

Take home messages:

  • Online environments are different ‘spaces’. You will need to make a few adjustments to how you teach
  • You must set expectations at the start about how you expect students (and yourself!) to behave in this space
  • Remember that this will be an agitating experience for many, and you/your students may not be comfortable. A little leeway helps.

There are three broad types of teaching online

Asynchronous discussion

Examples: Learn discussion boards, email chains, Twitter(ish)

Characteristics: Discussion moves at broken pace. Sub-threads/topics can appear easily and the ‘point’ can get lost. Can be difficulty for students to articulate what their ‘question’ is if they don’t understand where they’re going wrong

My experiences: This form of teaching often has issues with clarity, and its probably where I make the most ‘mistakes’.

Consider what ‘tone’ you want to set. I like emojis and GIFs because in person I am quite jokey. Others like to maintain distance. Neither is wrong, but try to aim for consistency. Remember that if you’re pivoting online things will feel unfamiliar, exciting and scary. You need to manage your class. I would encourage you to make full use of the medium to communicate clearly. If GIFs aren’t your thing don’t be afraid to literally type out “I was making a joke there”. Written communication has a myriad of ways of expressing what Face to Face communication would rely on body language for. Make use of them.

To that end – I often make the mistake of responding too quickly in this context because I’m desperate to help. Take the time to think about your response, and accept that your students working from home may be struggling with kids, ill housemates, distractions, and the point of an asynchronous board is that you can take your time. Consider setting a ‘class hour’ where you’ll respond to all comments so students are aware when they might get a response and don’t need to repeatedly check back.

Looks like:



Synchronous discussion

Examples: Instant messaging apps (Slack, Skype chat, WhatsApp), webinars, Zoom, Skype Screen Sharing, Blackboard Collaborate

Characteristics: Often involves (but does not require) a live presentation of slides or screencast of activity. Participants can chat by voice which is often noisy and challenging. Participants can chat in a text box which can be challenging for slow typers.

My experiences: While this feels the most familiar, this is my least favourite way to teach at distance. Moderation is vitally important. You should mute all participants unless you have a small group you know well and you want collaborative working (if so – consider giving them separate asynchronous discussion boards). Not all participants will have the IT literacy/attention span/hearing ability needed to distinguish lots of speakers on a bad connection and interruptions will be distracting

Further, some participants, like me, are chatty by nature. Do not let me bulldoze discussion because I’m loud and clear enough to be heard with a crackly mic and I never learned to think if others wanted to talk. As always, set up the environment that you want to teach in.

It’s tempting to use the text chat function. Again, fast typers like myself thrive here, whereas others will be left behind. Consider your choices carefully here. I like to encourage discussion in the chat when I’m presenting and then keep a note of what questions to come back to, but this takes a huge mental load and is not for everyone. Its easier to do with a moderator.

If you’re uncomfortable moderating (you don’t know the tech, you find it too hard to multitask) ask colleagues to step in. Learning techs might be keen (although overloaded) your academic colleagues might welcome a chance to observe your style. Finally, possibly the best option is to ask students to nominate one of their own as moderator so they are all taking responsibility for the learning environment.

A very quick word on live chats/ back channels in these contexts …  I have only seen students use these respectfully. Staff seem to see text chat as an opportunity to be mean, and staff on live chats have made me cry as a presenter. Again set your expectations at the start.

If you really want a live discussion – I’m a big fan of Google Hangout style pieces and they could be a good drop-in/tutorial replacement. You could run something similar with most e-collaboration resources.

Looks like:

Pre-Recorded Content

Examples: Previous years’ lecture recordings, YouTube videos, Articulate lectures, MediaHopper lectures

Characteristics: Broad range of presentation styles. Can be a bit experimental. Can be very simple. Students don’t have facility to ask questions so needs to be paired with discussion of some type or activity.

My experiences: Paired with asynchronous discussion boards, this is absolutely my favourite way to teach at distance. It gives the learners a lot of control, and you can actually experiment quite a lot with format and message.

Pre-recorded lectures can be super short – want to demonstrate something that you keep getting asked about? Do a short video! They can also be as long as a regular lecture (or longer!) because students can decide how to parcel their time. Although don’t go overboard! Record yourself with a phone camera if you’re trying to demo something more practical, or screencast a program.

If you are screencasting or recording a presentation, do some PC hygiene before recording, especially if you’re going to be demoing anything online. Don’t run unnecessary programmes in the background it will make your laptop chug. Invest in a good usb mic that is YOURS and can sit in your home, especially in these times! Use incognito browsers so people don’t see how much toilet paper the cookies are advertising to you.

Looks like:

https://media.ed.ac.uk/media/1_1am2dbni

Activities

There are loads of activities you can build online in your virtual learning environments from quizzes to Jupyter notebooks and polls, etc. I think if you’re pivoting to online learning for Coronavirus you should stay away from this, especially if you’re unfamiliar. Thinking about building interactive activity sessions online is something I would spend a lot of time on in the course design process. Can you really do this quickly halfway through?

Key Advice

Pivoting to online teaching is not the same as designing an online course. This is an exceptional circumstance and we should Keep It Simple (Seriously). You may even wish to consider what your top 3 learning outcomes are and focus on those[1].

Make pre-recorded content available to students wherever possible. If you have pre-recorded lectures from last year, you can upload these to your university’s media manager, virtual learning environment, or YouTube. You can record lectures using PowerPoint’s recording ability. These videos can be edited if necessary prior to upload.

Link to these on your VLE and create (or ask your resident learning technologist) discussion boards for questions.

Consider setting ‘office hours’ for questions – but be aware students may return to non GMT timezones. Try not to disadvantage students who are not in GMT locations.

Advice from JISC

You can view JISCs guidance here: https://www.jisc.ac.uk/guides/ensuring-continuity-of-learning-during-enforced-absence

Some mental health tips

Respect the situation. We are not all suddenly thinking about this because its fun or an opportunity to prove our innovative teaching styles. This is a marathon, not a sprint, and its about minimising the risk of spreading infection to vulnerable folks. Vulnerable folks are therefore most likely to need your support as an educator.

Try to build yourself a good work space. How you work at home may well have changed over the years. I find for some reason I now need a more dedicated work space, as opposed to my PhD years where I was fine working on the sofa.

Finally – let the cat scream for attention, acknowledge the toddler sitting on your lap, forgive the little stressy mistakes people will be making. We’re all in this together.

You Should Record Your Teaching

You should be recording your lectures. I’m salty about this.

Jilly

I’m about to go on two weeks annual leave so this is an excellent time for me to drop this particular post. Thanks to MichTheMartian on Twitter for reaching out and making me realise I haven’t ever really summarised this.

This post addresses the following concerns:

  • Students will stop showing up!
  • It won’t help students learn!
  • There’s no way to introduce lecture recording ethically in the current HE climate / Policy is hard / The union says no.
  • Its too technically challenging!
  • We just don’t have enough evidence
  • It makes me uncomfortable

Students will stop showing up!

If students stop showing up, there’s a bigger problem than you recording your lectures. 

Attendance at lectures is a complex, emotional and highly individual choice (1, 2, 3). Attending students may not necessarily be engaged in the teaching activity happening in front of them (4), and so anyone who is worried about students not showing up should be asking why their lectures aren’t ‘unmissable’. 

In her presentation here, Emily Nordmann has some practical recommendations for how to make the most of lecture recording, and she makes some excellent points about attendance. https://media.ed.ac.uk/media/1_4k30iisz

Although it could well be moot as there is plenty of evidence to show lecture recording does not affect attendance (5, 6), and evidence which suggests an effect is mostly students saying they thought they would be likely to stop attending (and not observing actual behaviour, 7, 8), and oft-cited research showing an effect has a lot of flaws, see Susan Rhind’s blog here and Emily Nordmann’s blog here. 

It won’t help students learn! 

There’s a commonly cited paper showing no evidence for attainment with increased lecture use (8). 

I think that paper’s probably right – for students who are otherwise grand, recordings don’t really help. I wouldn’t have used them when I was at uni. The case for using them is not about making everyone better, but removing barriers. I use the term ‘mainstreaming accessibility’ a lot. Having a recording policy removes some of the pressure from lecture recording (see Sarah Chinnery’s blog). If you have difficulties getting to class for any reason, knowing you can review the materials reliably is helpful, and stops you ‘outing’ yourself to your lecturer or your classmates. We have lots of example of this in our workshop (see below). Emily Nordmann (again!) has a lovely blog on why we need to start talking about the socially progressive case for lecture recording. It is a technology which helps remove one of the barriers to higher education. At Edinburgh, we have evidence suggesting that these positive impacts start in first year (9). 

In addition, I have some current research ongoing which strongly suggests that students who use lecture recordings to facilitate their studies do so in a very active and engaged manner. This work is funded by Echo360 and you can keep an eye on my Twitter feed to see more about it.  

In another project I’m involved with, we’re seeing some excellent secondary uses of lecture recording to help us think about what happens in classrooms and how that can help students learn. Emily has another great paper which we use at Edinburgh to support our students to study with lecture recordings (10) and we have produced open source guidance ourselves (11). 

Related – here’s some UK sector-wide discussion on Widening Participation with Lecture Recording. This project is ongoing and you should keep an eye on it

And follow our Twitter:

There’s no way to introduce lecture recording ethically in the current HE climate / Policy is hard / The union says no.

A good policy is the key here. You need to engage with the policy consultations that come round, and policy makers need to be cognisant of the power a good policy has in helping support the introduction of these technologies.  For me the take home messages are

  • The recording is a supplementary resource and should belong to the class that generated it. 
  • Lecturers should have to opt in to saving that resource in any long term way
  • Each class needs to set its own expectations around lecture recording (I talk about this in the podcast here and also you can see an example of me doing this at the start of our last Collaborative Cluster on Widening Participation with Lecture Recording meeting)
  • Lecturers should be able to opt out of lecture recording whenever they feel the need to (linked to the class expectations – this can also support student learning, e.g. during ethical discussions. I talk about the practicalities in this podcast)

Richard Goodman talks about moving to an opt-out policy at Loughborough

Melissa Highton talks about working with the union to develop good policies at Edinburgh. 

Its too technically challenging!

Aren’t you lucky that the University of Edinburgh has put together a completely free resource talking about Delivering and Evaluating Lecture Recording so you don’t have to start from scratch?

We just don’t have enough evidence

Overview of Edinburgh’s research

Me presenting on Edinburgh’s research

Edinburgh’s TeachingMatters Blog Lecture Recording Miniseries

Our TeachingMatters Podcast on lecture recording

I walked through how you can run your own evaluation and included a bunch of resources with the QAA here:

It makes me uncomfortable

I know it does. Believe me, it makes me kind of uncomfortable too. I’ve been on television, on radio, on stage on the Lyceum, written a book, and I still find it slightly uncomfortable watching myself stumble over words and have a brain blank in front of a class. 

But here’s the thing – students don’t care. They’re here to learn. And they want to learn. They’re not expecting you to be David Attenborough. They’re not expecting you to be perfect. And if you can use your mistakes to model good academic practice (jump to 5 minutes), maybe you can create better learners.

My research shows that lecture recording makes lecturers worry about getting things wrong, or being criticised, while students just view it as a tool (Show and Tool – or the free preprint if you can’t access that journal). This is a conversation you need to have with your students. Making mistakes is how we learn.

One of the big things I’d encourage you to make use of is the open source workshop we made available with the QAA (direct link to resource). Using this workshop we’ve developed new guidance on how to teach with lecture recording that will be getting printed and shared imminently. 

But at the end of the day – you’re the teacher. You’re there to help students. After the conversations I’ve been having lately with certain academics, this feels like a revolutionary thing to say. I am salty. I am salty that I need to defend students’ rights to have access to materials. I am salty that I need to justify writing “students should approach their lecturers for support” in student guidance. I am salty that academics are putting their personal comfort over the needs of their students.

You should be recording your lectures.

Can sin a-rithist?

JillyLeave a comment

Failte gu Fluffy Sciences! Is mise Jill NicAoidh. Tha aon cat agam. Seo Athena. 

In late 2019, Duolingo launched the Scottish Gaelic version of its app. My dad and sister have been learning Gaelic for some time, and I’ve been trying to pick up a few phrases here and there. I’ve been doing this mostly through Speaking Our Language, a brilliant BBC Scotland series that I think is supposed to take place in a post SNP victory Scotland where English has been outlawed and people wander around Glasgow stumbling through broken Gaelic with frightened faces. Its wonderful and I love it and you should watch it:

At school, I didn’t find languages easy, and therefore I considered them hard. Like many perfectionist people I would then announce I was terrible at languages. After a few weeks of playing around on Duolingo, I can confidently say I speak more Gaelic than German, which I learned at school for many years. I’m trying to avoid ‘classifying’ my language abilities these days as part thinking about how assessment and learning intertwine. 

In education conferences, particularly whenever gamification is mentioned, Duolingo is the Ur Example people use to illustrate how points, leaderboards, and rewards can be used to motivate learners. Both myself and my partner have taken the app up this month, he’s learning Spanish, I’m learning Gaelic, and I have some thoughts on how gamification and motivation tie in. 

I am very motivated to learn Gaelic. My little sister is currently shaming me, which is a big one, but there’s something beautiful about reviving a language that I see on signs every day, but is spoken by very few people. I recently learned that my grandparents used to speak Gaelic in the home, and my teuchter family must have done for many generations. It’s strange to think how quickly a language can disappear. 

There are lots of benefits to learning a language. There’s reasonable evidence that being bilingual slows the onset of Alzheimers, and learning new skills as an adult (and educator) can help you think more about learning. There is also, for me, a huge benefit in being able to read the street signs in my country. 

When you drive from England to Scotland you pass beautiful blue signs that read Failte gu Alba! I’ve had that said to me several times, but in my head I always read it as ‘Fail-ta goo Alba’. Now I read it, naturally, as ‘Fael-Cha gu Alaba”. Many people in Scotland use odd turns of phrase or strange grammar. The Scots dialect would say “It’s wanting cleaned”, and I see echoes of that in the way Gaelic constructs sentences, tha mi ag irraidh ti. I’ve no idea if these parallels are true, but I feel as though I’m recovering something precious. If it’s something I can do to roll back ‘Scottish Cringe’ I’m all for it. In primary school we were simultaneously taught to recite Scottish poetry but penalised for writing ‘yous’ and ‘wur’, and there’s a lot that’s needed to undo that damage. 

Learning on Duolingo is interesting though. I’m fascinated by silos in learning from a curriculum design point of view. There’s a phenomenon where if you learn something in one context you aren’t able to generalise it to another context. I feel like I’ve been fighting learning silos for my entire teaching career, and it frustrates me no end to find my own Gaelic abilities vanishing the moment I close the Duolingo app. I’ve peppered some Gaelic throughout this blog, all phrases I can reliably type into the app, and all of them I had to google in front of my word document. Duolingo does suggest you should write down as many phrases as you can remember after a lesson, but is that enough? When you scaffold ‘extra’ learning outside of class time, is that really divorced enough from the course context to break down these walls?

Both my partner and I have observed that our language skills aren’t persisting outside of the app’s ‘classroom’, even though we’re both motivated to learn. I have no answers for this problem, yet, but it’s been an interesting experience to have first hand. 

Tha mi a’ bruidhinn Gaidhlig, tha mi cho toilichte.  

Teaching Critical Analysis in Cancel Culture

JillyLeave a comment

Content Warning: Discussion of trans exclusion narratives

They say 2020 is the year of the blog (They being two or three people in my Twitter timeline), and I’m full of new year’s resolutions so let’s talk about Cancel Culture. 

Cancel Culture, if you are blissfully naive, is an internet practice where someone who says something controversial or ‘problematic’ is repeatedly attacked and boycotted in an attempt to minimise the dangerous opinion’s platform. You can think of it as a reaction to the Paradox of Tolerance, in that it is most often the political left does to protect those viewed as being less powerful. 

The fabulous YouTuber ContraPoints (Natalie Wynn) started January with a 100 minute long video discussing cancel culture after she herself was ‘cancelled’ for a recent video where she invited Buck Angel to provide a voice over for a quote. ContraPoints has put in a huge amount of effort into this video and it’s really one of her best, so I won’t explain the whole drama here, I’ll just point you toward her video. It’s here. You should really consider watching it.

This video really appealed to me for a lot of reasons. As you may well be aware, the University of Edinburgh has seen a lot of outrage regarding feminism and trans rights. The rest of this post, and indeed all of my content, comes from my perspective as a cis-gendered bisexual woman, and I would encourage you to do your own research on these topics, and to remember that I am no expert here. There are two important concepts here, trans-exclusionary radical feminism (TERF) which many people prefer to call ‘gender critical feminism’, and trans-inclusionary feminism. In a nutshell, the trans-exclusionary feminists are concerned that trans women will endanger female spaces and female safety. Trans inclusive feminism considers that trans women are women, and have the same need for female spaces as cis women. I use the words ‘trans-exclusionary’ and ‘trans-inclusive’ in this blog because at present I see that as the crux of the conflict. Many years ago, I understood gender-critical feminism to be about breaking down imposed boundaries of gender, but that’s not how the term is now used. It is now fairly universally used to mean trans-exclusionary philosophies. Maybe I misunderstood the term at first, or maybe it has changed, but for me it’s clearer to talk in exclusion/inclusion terms.  

The New York Times recently argued that TERFs are a particularly British problem, which I’m not sure I agree with, but there is certainly a type of older British feminist that you almost ‘expect’ to come out with trans-exclusionary views, such as J K Rowling. Recently Rowling tweeted in support of Maya Forstater, a woman who took her employers to tribunal for not renewing her fixed term contract after she repeatedly made trans-exclusionary tweets. Her complaint was not upheld. I was irritated by people defending Forstater, and made this tweet, jumping on the cancel culture bandwagon for JK Rowling. 

The Halo Reach realisation is real, incidentally. I remember being on the Halo fan forums at the time and enthusiastically telling a Bungie developer I’d be happy to have my multiplayer character grunt in a female voice, even if the model was the same. That memory, sparked after seeing some of the Reach remake trailers, struck me as a really clear example of why being correctly gendered in conversation is so important to people, in a way that cis folks like myself might find easier to relate to. On reflection, I wish I hadn’t said ‘fuck TERFs’ in the tweet, because that’s not going to invite any reasonable debate or encourage anyone to listen to my ideas. And this is the issue with cancel culture that ContraPoints’ video discusses. Cancel culture prohibits discussion.

While I was watching ContraPoints’ video, I was uncomfortably reminded of something my dad used to say to me when we were having political debate. My dad frequently calls me a ‘Trotskyite’, who’s too busy feuding with my fellows to pull together for the left as a whole. Much like myself, my dad will also say almost anything in a debate to win, but his observation about the left’s tendency to schism is a pertinent one. In the nineties, around our kitchen table, he was eerily prescient of the left’s challenges in the 2010s.

The Economist recently posted a brilliant article about why the Conservatives are the most successful political party in the world. If it’s behind a paywall for you, the answer is that the Tories are very flexible in morality and policy if it keeps them in power. They will be the party they need to be to get elected. The left, by contrast, will fight within itself about how exactly it should respect all members (see this Atlantic article, this Guardian article which seems to suggest that Labour should have done everything differently in the last GE). 

We live in a world where it is very easy to express an opinion, and where you can be cancelled for that opinion just as quickly. Take Joan Meiners who tweeted about the inaccuracy of peoples’ bee tattoos, not realising that those tattoos were a specific reference to the Manchester Evening News Arena bombing. 

I liked Joan’s original post when I saw it, and then I liked the tweets criticising it. Where do I fall in the outrage league?

How do we teach the academics of the future to critically analyse ideas when they see trusted, powerful, and adored figures like JK Rowling being told to die for what is an opinion? Especially when we say we’ll assess them, and the perceived penalty of getting it wrong is so high? In my first year science course, I tell my students I expect compassionate science from them, but I don’t tell them how I expect them to critique a piece of science compassionately. I tell them their best exposure to critical analysis will be in scientific papers, but we all know how bad peer review is for encouraging this kind of discourse.

Trans rights may seem like a strange road to thinking about how I teach critical analysis, but the concern and trepidation I felt about these issues must be a reflection of how it feels to critique something in a modern classroom. Our classrooms will have students who are aware of these debates, aware of these controversies, and whom we’re asking to critically analyse a piece of work. It is our responsibility to teach with an awareness of this culture. 

So this year I’m going to work on actively teaching compassionate critical analysis. I’m not totally sure how this will go yet, so I started drafting an outline. Have a look through the slides and see what you think, feel free to adapt or hit me up on Twitter to give me any feedback.

Cancel culture is cancelled. 

100 Papers

This is as close as I’ll come to an academic year in review

JillyLeave a comment

In 2019 I took part in the #100papers challenge. The idea is that you aim to read (fully) 100 scientific papers in a year. 

As I understand it, the challenge was born from the #365papers challenge. Some fools well-intentioned folks aim for averaging a paper a day for a year, and others thought “I’ll be lucky if I manage a third of it”. With both #365papers and #100papers, the idea is that you’ll commit to reading more if you’re publicly tracking it, and maybe also read more widely. I knew that #365papers would not be achievable for me, but #100papers might have been within my grasp (spoilers, it wasn’t). 

I really like setting myself challenges. I’ve done a variety of photography and reading challenges over the years. Tracking the papers that I read on Twitter is innately appealing to me. I also wanted to put a potted summary or key outcome from each paper onto my tweets to force me to read the papers instead of cheating the essence of the challenge by skimming. 

I have a pretty good work-life balance. I set aside a day a week to devote to research and I manage to keep that day protected about 60% of the time I’d say. How many papers did I read in their entirety in 2019? 

40. I read 40 papers cover to cover. 

I have some thoughts about this exercise. Firstly, I don’t think this is The Way to read papers. Something I noticed about reading whole papers was how pointless it often is. I teach students to be selective about how they approach papers, and when I was trying to find out how someone set up a study or I wanted an overview of a particular field, I wasn’t sitting down to read a whole paper, I was flicking to the relevant parts of various papers. So my first big takeaway is that reading whole papers isn’t something that I would prioritise over strategic paper skimming. 

With that being said, there is something quite meditative and indulgent about reading a whole paper. There were some very fun papers like Jenny Scoles’ one on messy boundary objects where the narrative itself is enjoyable. 

(There was also this deeply enjoyable rant where you could feel the authors’ visceral hatred of the right-brain-thinking myth.)

And I also really liked having the Twitter thread of all the papers I’d read, and the ability to jump back into that thread to share with people was massively useful. Bauer et al 2017, alongside reading Invisible Women, has changed my research practice quite considerably this year: 

The performative aspect of talking about the papers I’m reading online was also interesting. I think you can track what projects I was working on with this twitter thread. You can see when I started reading up on our Widening Participation cluster for example, and I like some of the conversations the spawned from the thread. 


In 2020, I’m probably not going to do the challenge again, but I’ll certainly be posting a papers thread, maybe #paperswotiread or something along those lines. The target of 100 fully read papers is not feasible for me, and if its not feasible for me, I’m not comfortable advertising it to those academics who may be following me. I’ve been thinking a lot this year about how I model what I view as ‘good’ academic practice, and I’m trying to make positive choices. So I’ll be doing something like this in 2020, just without the targets.  

How you doin’?

The peer observation cycle at the R(D)SVS is approaching its end, so its time to find a buddy and ask each other “how am I doing?” Feedback on practice that’s dear to my heart – my favourite thing!

Jilly3 Comments

The peer observation cycle at the R(D)SVS is approaching its end, which means we’re all hurriedly looking around for someone to come give us some feedback on how we teach. My teaching load has changed hugely since I started (hello new course organising responsibilities!) and I was feeling quite blasé about the peer obs process, mentally putting myself in reserve for those people who were undoubtedly going to run out of time and need a buddy last minute. 

Up until a colleague approached me and asked if we could buddy up because we teach very similar subjects. Why not do it right, after all? 

This is the first time in all my peer observation/feedback on teaching sessions that I’ve ever been observed by a more experienced colleague, and a considerably more experienced colleague at that (Dear Peer, if you’re reading this, your experience is simply a reflection of your very hard work, and in no way a commentary on years teaching 😉 ) And to my surprise, I found myself nervous about it. 

When I’m talking to people about our peer observation sessions I give lots of advice that I did not follow myself. For example, you have complete control over your peer observation sessions. If things are stressful or you’re not feeling it, you can always reschedule. But of course the time I’d scheduled with my peer just happened to fall over another colleague’s sick leave that I was having to unexpectedly cover for, a period of feeling under the weather myself, and a very stressful busy work period. But we went for it anyway. 

How did the session go? Well like many of my teaching sessions, I walked away thinking there was so much more I could have done, and my peer picked up on some of those in our debrief. But what was interesting was that the thing I’d asked my peer to focus on was what my peer considered to be the strongest part of the lecture. And this is a recurring theme in all my peer observations. The things I’m fretting about are usually not the things the peer picks up on. 

In this particular session I’d been asking about the engagement, and worrying about how the students were responding to the more ‘active learning’ parts of the session. My peer helped me see how positively they responded, and then was able to share some of their practice with me that I am definitely stealing drawing from next week when I continue the session. 

My peer also asked me a few questions, like why I didn’t scaffold in breaks, that made me think a little bit more about my approach to preparing teaching. Many of the questions my peer asked could have been answered “Oh I usually do that but…” and that alone is a fascinating observation. I spent a crazy amount of time designing these courses, and working hard on programmatic level innovation. Despite all that hard work, when I’m under pressure I default to teaching in the way I’m most comfortable with, the way I was taught. 

This realisation has also highlighted for me that building engaged and active learning opportunities actually costs me more preparation time than a more traditional lecture, despite the fact it seems like the student is doing most of the work. None of these observations are new to the field I’ll point out – people have been making this discovery for years, and  I suspect I’ve discovered this before too. I think the value of the peer observation session is helping to catch out those little bad habits you can slip back into. 

My final observation on the peer observation is that I’m really proud of myself for accepting the feedback. Working on feedback has been something of a project for me over the last few years. It reminds me of a time when a peer review came back on a paper I’d submitted, and I’d been grumping over the reviewer comments as you’re wont to do, until I got to the end and saw the reviewer’s name on this completely open journal. The reviewer was somebody I highly respected, and suddenly my entire perspective on the feedback changed. Having my peer be a more experienced colleague who I really respect was a great way for me personally to become more open to the feedback I was receiving. 

Ultimately lots to work on, and lots to be proud of, and all for a little bit of an uncomfortable an un-British conversation where we asked each other “how am I doing?”