Category: Knowledge Transfer

Posts in this category are about the various science outreach activities and learning I participate in.

You Should Record Your Teaching

You should be recording your lectures. I’m salty about this.

Jilly

I’m about to go on two weeks annual leave so this is an excellent time for me to drop this particular post. Thanks to MichTheMartian on Twitter for reaching out and making me realise I haven’t ever really summarised this.

This post addresses the following concerns:

  • Students will stop showing up!
  • It won’t help students learn!
  • There’s no way to introduce lecture recording ethically in the current HE climate / Policy is hard / The union says no.
  • Its too technically challenging!
  • We just don’t have enough evidence
  • It makes me uncomfortable

Students will stop showing up!

If students stop showing up, there’s a bigger problem than you recording your lectures. 

Attendance at lectures is a complex, emotional and highly individual choice (1, 2, 3). Attending students may not necessarily be engaged in the teaching activity happening in front of them (4), and so anyone who is worried about students not showing up should be asking why their lectures aren’t ‘unmissable’. 

In her presentation here, Emily Nordmann has some practical recommendations for how to make the most of lecture recording, and she makes some excellent points about attendance. https://media.ed.ac.uk/media/1_4k30iisz

Although it could well be moot as there is plenty of evidence to show lecture recording does not affect attendance (5, 6), and evidence which suggests an effect is mostly students saying they thought they would be likely to stop attending (and not observing actual behaviour, 7, 8), and oft-cited research showing an effect has a lot of flaws, see Susan Rhind’s blog here and Emily Nordmann’s blog here. 

It won’t help students learn! 

There’s a commonly cited paper showing no evidence for attainment with increased lecture use (8). 

I think that paper’s probably right – for students who are otherwise grand, recordings don’t really help. I wouldn’t have used them when I was at uni. The case for using them is not about making everyone better, but removing barriers. I use the term ‘mainstreaming accessibility’ a lot. Having a recording policy removes some of the pressure from lecture recording (see Sarah Chinnery’s blog). If you have difficulties getting to class for any reason, knowing you can review the materials reliably is helpful, and stops you ‘outing’ yourself to your lecturer or your classmates. We have lots of example of this in our workshop (see below). Emily Nordmann (again!) has a lovely blog on why we need to start talking about the socially progressive case for lecture recording. It is a technology which helps remove one of the barriers to higher education. At Edinburgh, we have evidence suggesting that these positive impacts start in first year (9). 

In addition, I have some current research ongoing which strongly suggests that students who use lecture recordings to facilitate their studies do so in a very active and engaged manner. This work is funded by Echo360 and you can keep an eye on my Twitter feed to see more about it.  

In another project I’m involved with, we’re seeing some excellent secondary uses of lecture recording to help us think about what happens in classrooms and how that can help students learn. Emily has another great paper which we use at Edinburgh to support our students to study with lecture recordings (10) and we have produced open source guidance ourselves (11). 

Related – here’s some UK sector-wide discussion on Widening Participation with Lecture Recording. This project is ongoing and you should keep an eye on it

And follow our Twitter:

There’s no way to introduce lecture recording ethically in the current HE climate / Policy is hard / The union says no.

A good policy is the key here. You need to engage with the policy consultations that come round, and policy makers need to be cognisant of the power a good policy has in helping support the introduction of these technologies.  For me the take home messages are

  • The recording is a supplementary resource and should belong to the class that generated it. 
  • Lecturers should have to opt in to saving that resource in any long term way
  • Each class needs to set its own expectations around lecture recording (I talk about this in the podcast here and also you can see an example of me doing this at the start of our last Collaborative Cluster on Widening Participation with Lecture Recording meeting)
  • Lecturers should be able to opt out of lecture recording whenever they feel the need to (linked to the class expectations – this can also support student learning, e.g. during ethical discussions. I talk about the practicalities in this podcast)

Richard Goodman talks about moving to an opt-out policy at Loughborough

Melissa Highton talks about working with the union to develop good policies at Edinburgh. 

Its too technically challenging!

Aren’t you lucky that the University of Edinburgh has put together a completely free resource talking about Delivering and Evaluating Lecture Recording so you don’t have to start from scratch?

We just don’t have enough evidence

Overview of Edinburgh’s research

Me presenting on Edinburgh’s research

Edinburgh’s TeachingMatters Blog Lecture Recording Miniseries

Our TeachingMatters Podcast on lecture recording

I walked through how you can run your own evaluation and included a bunch of resources with the QAA here:

It makes me uncomfortable

I know it does. Believe me, it makes me kind of uncomfortable too. I’ve been on television, on radio, on stage on the Lyceum, written a book, and I still find it slightly uncomfortable watching myself stumble over words and have a brain blank in front of a class. 

But here’s the thing – students don’t care. They’re here to learn. And they want to learn. They’re not expecting you to be David Attenborough. They’re not expecting you to be perfect. And if you can use your mistakes to model good academic practice (jump to 5 minutes), maybe you can create better learners.

My research shows that lecture recording makes lecturers worry about getting things wrong, or being criticised, while students just view it as a tool (Show and Tool – or the free preprint if you can’t access that journal). This is a conversation you need to have with your students. Making mistakes is how we learn.

One of the big things I’d encourage you to make use of is the open source workshop we made available with the QAA (direct link to resource). Using this workshop we’ve developed new guidance on how to teach with lecture recording that will be getting printed and shared imminently. 

But at the end of the day – you’re the teacher. You’re there to help students. After the conversations I’ve been having lately with certain academics, this feels like a revolutionary thing to say. I am salty. I am salty that I need to defend students’ rights to have access to materials. I am salty that I need to justify writing “students should approach their lecturers for support” in student guidance. I am salty that academics are putting their personal comfort over the needs of their students.

You should be recording your lectures.

100 Papers

This is as close as I’ll come to an academic year in review

JillyLeave a comment

In 2019 I took part in the #100papers challenge. The idea is that you aim to read (fully) 100 scientific papers in a year. 

As I understand it, the challenge was born from the #365papers challenge. Some fools well-intentioned folks aim for averaging a paper a day for a year, and others thought “I’ll be lucky if I manage a third of it”. With both #365papers and #100papers, the idea is that you’ll commit to reading more if you’re publicly tracking it, and maybe also read more widely. I knew that #365papers would not be achievable for me, but #100papers might have been within my grasp (spoilers, it wasn’t). 

I really like setting myself challenges. I’ve done a variety of photography and reading challenges over the years. Tracking the papers that I read on Twitter is innately appealing to me. I also wanted to put a potted summary or key outcome from each paper onto my tweets to force me to read the papers instead of cheating the essence of the challenge by skimming. 

I have a pretty good work-life balance. I set aside a day a week to devote to research and I manage to keep that day protected about 60% of the time I’d say. How many papers did I read in their entirety in 2019? 

40. I read 40 papers cover to cover. 

I have some thoughts about this exercise. Firstly, I don’t think this is The Way to read papers. Something I noticed about reading whole papers was how pointless it often is. I teach students to be selective about how they approach papers, and when I was trying to find out how someone set up a study or I wanted an overview of a particular field, I wasn’t sitting down to read a whole paper, I was flicking to the relevant parts of various papers. So my first big takeaway is that reading whole papers isn’t something that I would prioritise over strategic paper skimming. 

With that being said, there is something quite meditative and indulgent about reading a whole paper. There were some very fun papers like Jenny Scoles’ one on messy boundary objects where the narrative itself is enjoyable. 

(There was also this deeply enjoyable rant where you could feel the authors’ visceral hatred of the right-brain-thinking myth.)

And I also really liked having the Twitter thread of all the papers I’d read, and the ability to jump back into that thread to share with people was massively useful. Bauer et al 2017, alongside reading Invisible Women, has changed my research practice quite considerably this year: 

The performative aspect of talking about the papers I’m reading online was also interesting. I think you can track what projects I was working on with this twitter thread. You can see when I started reading up on our Widening Participation cluster for example, and I like some of the conversations the spawned from the thread. 


In 2020, I’m probably not going to do the challenge again, but I’ll certainly be posting a papers thread, maybe #paperswotiread or something along those lines. The target of 100 fully read papers is not feasible for me, and if its not feasible for me, I’m not comfortable advertising it to those academics who may be following me. I’ve been thinking a lot this year about how I model what I view as ‘good’ academic practice, and I’m trying to make positive choices. So I’ll be doing something like this in 2020, just without the targets.  

USS Strike

JillyLeave a comment

I want to tell you why I have chosen to join my fellow members of the Universities and Colleges Union (UCU) in industrial action from the 28th February.

I consider myself incredibly lucky in my career.

I am lucky, because I only signed on once after my PhD, for a short period of time. Many sign on for longer.

I am lucky because I knew that signing on would contribute to my National Insurance payments, which had been on hold, or only partially fulfilled, for the eight years of higher education I took part in.

I am lucky, because I finished my PhD at 26, and entered full time employment at 26. Many people do not finish their PhDs until their thirties.

I am lucky, because I was earmarked for a PhD on day one of my undergraduate degree, and I received exceptional support.

I am lucky, because I have been given fixed term contracts. Many academics are given guaranteed hours, or hours to be notified, and don’t even have the luxury of knowing how much they will bring home every month.

I am lucky, because my fixed term contracts ranged from three weeks, to three years, and so I have felt largely safe in my employment, as much as academics ever can . . .

I am lucky, because the bank decided to bend the rules on my mortgage, even though my contract did not qualify me for one.

I am lucky, because I’m coping with the mental health problems that accompany working in academia.

I am lucky because I am not juggling academia with a young family, because I genuinely love both teaching and research, because I am not stuck with one of the bullies as my boss, because my visa is not threatened by Brexit, because I happen to work in a field that is strong in the UK, because I’m publishing papers that happen to REFable, I’m lucky because I don’t want to quit . . . unlike them, them, them and them.

Yes, we have a good pension. An expensive pension. It is what the universities give us to make up for the fact that on average we earn less than we would elsewhere. We think that the creation, dissemination and curation of knowledge is vitally important for our students, and for our society, and so we put up with the challenges. One of our conditions of employment is that our employers take some of our money, and give it to us after our hard working life is done.

I am an experienced researcher, I’m an interdisciplinary researcher, and at the age of 32 I will be one of the youngest people to age out of the ‘six years post PhD’ definition of an early career academic. I am managing to keep my head above water, and my career going, and I just about feel safe now. The proposed cuts will take £12,000+ per year away from my pension.

 

I am what it looks like to be lucky in academia. Take our pensions, and academia will be lucky to have any of us left.

And May All Your Dreams Come True

Jilly3 Comments

For as long as I can remember, I have wanted to be a writer.

As a child, I filled endless notebooks with my stories. They were mostly stories about animals, or thinly veiled replicas of Lord of the Rings. I may even have tried my hand at the odd love story. At school, I kept a private tally of how often my essays were read aloud, or made a teacher cry. I love the written word.

When I was 29 years old, an editor approached me and asked me to write a book. That book, Animal Personalities, is currently available for pre-order.

Of course, when you achieve your childhood dreams, a weight lifts from your heart, a divine confidence settles in your soul, and you never again doubt yourself or your abilities. You become as happy as you always believed you would be . . .

I recently wrote a short case study about being a postdoc for Edinburgh’s “Thriving in Your Research Position” document from the Institute of Academic Development. In the case study, I talk about a spectral figure who has haunted me throughout my whole career: the Perfect Postdoc. She is always better than me. When I wrote my book, she somehow wrote a better one. She’s like a funhouse mirror version of me, and when I change, so does she. I’ll never be able to outdo her.

If you’re a long-term reader of this blog, you’ll know I’ve been thinking about failure lately. I explored my failures as an animal trainer, and meditated on how academia breeds an anti-failure culture. I’m also critical of the idea that all scientists have to be specialists – I’m not a specialist. I’m interdisciplinary and I love it. This leads me to another area of my academic life where the Perfect Postdoc is always one step ahead of me.

The Perfect Postdoc understands R much better than I do. I’ve spoken before on this blog about my frustrations while trying to learn R. While I have taught research methods and statistics for several years now, I’ve always hesitated to teach R. I’ve hesitated because, well . . . because I’m not brilliant at it. My code is ugly and often cobbled together, and I often find the community around R, places like stack exchange and stack overflow, are hideously unfriendly.

I’ve been lucky enough enrol on the Leadership Foundation for Higher Education’s woman-only Aurora programme this year. The first session was called Identity, Impact and Voice, where we explored how we can make a difference in our workplaces and communities. There were two-hundred plus women at the Aurora event in Edinburgh this month, and so many of us spoke about being afraid of ‘not being the best’.

The curious thing is, when I was listing my strengths, I never said I was “the best at [thing]”. My strengths are my communication skills, the fact I’m approachable, and my willingness to try new things. I firmly believe that in five years time anyone who doesn’t have R skills is going to find it very difficult to get a job in academia. Hiding my bad code means I’m not contributing to the R conversation happening right now. I have a voice. And I can have an impact too.

Hadley Wickham, who wrote some fabulous R packages, says:

So with that in mind, I’m going to start sharing my own R teaching materials more widely.  You can find my resources on Github (scroll down to find direct links to the exercises). The worst that can happen is that someone tells me my code is ugly. The Perfect Postdoc’s code is of course much prettier, but do you know what? Just like writing my book, writing that exercise was pretty fun.

Glory in your bad code. Glory in saying “I don’t know how to do that” in your local programming club meetings. Glory in your voice. There is nothing else like it.

 

The Finnish Lesson

JillyLeave a comment

For the record, I managed two whole plenaries in AMEE before I was overcome with opinions and had to blog about it.

First things first, AMEE 2017, an International Association for Medical Education, has been a bit of a revelation for me. Sitting in a crowd of 3800 medical educators, when you’ve only been on the job for fourteen months, is a bit overwhelming. But this has been one of the friendliest, most accessible conferences I’ve ever attended. It’s been a delight so far.

But I want to talk about the Finnish Education system here. Our second plenary of the conference was by Pasi Sahlberg, whose talk was titled “What can medical education learn from the Finnish experience of educational change?”

First off, it’s important to talk about the conference crush. It’s a thing that happens when you hear another researcher talk and their passion and excitement, and their insight into a topic, just sets your heart racing and before you know it you’re having idle fantasies of working in another research group. It happens to me about ten times a conference. I got a case of it listening to Sahlberg talk about the Finnish education experience. In about 15 years they managed to make massive improvements, and top the global league tables in many arenas of literacy. They improved so much they surprised themselves.

I think Sahlberg will be posting his slides on his website, but I quite enjoy taking my own things away a talk. The highlights to me were:

  • Teaching must be respected (in Finland you need an Masters degree to do any kind of teaching)
  • School systems should not be competitive with one another for ‘clients’
  • Value play and failure
  • The society you teach in needs to have high equity

 

Whether or not this is what Sahlberg intended to communicate, this is what I walked away with. There are so many questions that come tumbling out when I think about this. For us in Scotland, I really worry about the equity in our educational society. Any three students in my lecture could have paid three different fees to hear the same material. That worries me greatly. With the changing politics of the UK, we risk losing many of our hard-earned gains in society.

Sahlberg presented a slide which talked about ‘Global Educational Reform Movement’, and how it had spread (like a g.e.r.m.) from the UK in the eighties, and moved forward. I can’t be the only person in the room who was thinking about dear old Maggie Thatcher. Whether education must always be political is an interesting question (one opinion, one more). I have always been a political creature, and I believe there is politics in all we do. I found Sahlberg’s slides very convincing that we must create certain kind of systems in order to promote better educational outcomes.

Sahlberg also highlighted the value of play, briefly, and the value of what he called ‘small data’. These are subjects close to my heart. As someone with a big-data PhD, I now spend a lot of time on small data, and explore qualitative ways to evaluate what we do, because sometimes that’s the best method you can use to answer the question you’re interested in. I like these two elements because they are both things that are sometimes frowned upon in the environments I work. When I did my M.Sci, I had this feeling that I wasn’t allowed to get emotional about the animals, I wasn’t allowed to have fun in my job. Where did this come from? No one ever told me this, but it was part of my culture nonetheless. I still struggle a little with this.

This blog is called ‘Fluffy Sciences’ because I want to kick back against the ideas that ‘soft’ things, play, small data, feelings, are less valuable. What we do is massively complicated, asking questions like “how do we change a whole community in order to improve our education”, and not recognising how valuable that is results in any old person doing teaching, being given no support, and students who are treated as commodities, not people.

Here at AMEE, it’s incredibly empowering to be around so many people who recognise the importance of education research. Let’s hope that we can all take that confidence back with us to our schools as a beacon.

Reflections on Being a Terrible Trainer

Jilly1 Comment

I am terrible at training animals. No, really, I am. Any good behaviour my animals show is mostly entirely accidental. I understand the theory, I can correctly distinguish between continuous and variable reward schedules, but I lack patience, and I lack consistency, two of the key aspects of animal training.

There are loads of resources on how to train your pets, and loads of blogs talking about it, so this post doesn’t want to teach you how to be a better trainer.

This post wants to talk about reflection.

 

See, in education research we talk a lot about self-reflection and how important it is to the process self-development. In my experience, scientists and clinicians (and I include myself in this) are rarely as good at self-reflection as they think they are.

Last week I decided I was going to train Athena to give me her paw. I had three main goals:

  1. My big overall goal is to relax Athena. There has been a lot going on this year and her anxiety around people has gotten worse. By giving her something to think about and a ‘job’, I hope that she’ll start to feel more in control of her environment.
  2. My specific reason for teaching ‘paw’ first is that Athena is phobic about her paws being touched which means clipping her claws is a hassle. She’s also had occasional contact dermatitis on her paws when she gets into places she’s not supposed to, so getting Athena comfortable with presenting a paw would be a great help down the line.
  3. Training is a bit of fun for both of us!

 

Here is a short montage of nine days of paw training with Athena.

 

I’m going to use Gibbs’ Reflective Cycle to talk about this training experience.

 

Description

What we have here is continuous reward (Athena is being rewarded with a treat every time she performs the behaviour), with some less-than-ideal shaping of the behaviour. To encourage her I would take her paw in my hand instead of waiting for her to lift her paw spontaneously. Overall I was not using best practice training methods. Why? Well, impatience for one thing. I knew it would take far longer to shape the behaviour properly, and while I know continuous reward is a problem, I felt like she needed to be very motivated to sit and train (see the earliest session where she walked off after I annoyed her!).

 

Feelings

Overall I’m pretty delighted with Athena’s performance. I think she picked up the behaviour very quickly and all of our sessions were short and enjoyable. Watching the videos back I’m also surprised by how quickly she learned to sit patiently. I didn’t start out to teach the ‘sit’ but that behaviour became part of the game very quickly. Keeping the sessions short meant that even when things went a bit wrong, I wasn’t frustrated.

 

Evaluation

Doing one to two short (less than 10 minute) sessions a day was great for both of us. I typically did one in the morning and one at bedtime. They feel like just the right amount of work for both of us.

There are three things that could be improved: a variable reward ratio, shaping the behaviour more naturally, and for me to stop rewarding other behaviours. Of these, the third one is the big thing for me as a trainer, and always has been. I’m the type of fidgety, unpredictable person that is just generally bad at training, so keeping myself controlled in these sessions would be a big help to Athena.

 

Analysis

I explored a lot of dog training blogs when writing this post, not before starting (remember, patience is not one of my virtues). I found surprisingly few cat training discussions, with a few from Catster.  That very link was highlighting the importance of shaping behaviours without forcing the cat into doing anything. My two big mistakes during training (variable ratio reward and shaping behaviour naturally) are definitely things that the literature doesn’t like, but they are both extremely tricky skills to master. I knew about both, and yet diving in to get to my end goal I conveniently put both out of my mind.

 

This was what got me thinking about failure, one of my favourite topics at the moment. I hate failing, and yet there would have been nothing ‘wrong’ with waiting for Athena to do these behaviours herself. I wouldn’t have failed at anything unless I’d stopped training altogether.

 

Conclusion

What else could I have done? I could have thought about Athena a bit more during this whole thing. One of my initial motivations was for her to be less phobic about her paws being touched, and you can see in the video she’s still not 100% happy about it. There’s a huge value to the natural shaping of behaviour and it’s Athena’s affection for me that kept her willing to engage (and at least when she did walk away I let her!).

On the other hand, I’m really impressed at Athena’s demonstrable ability to be trained. With all the affection in the world, I don’t think she’s a particularly biddable cat. I hope that means she enjoyed the training sessions and found them rewarding. Establishing whether animals find training rewarding or not is a sometimes controversial topic, so I’d like to look into that in more detail.

Overall though, I’m really pleased she’s picked up the training, and I think I showed a lot of progress in the video too.

 

Action Plan  

We’re going to continue this training and move onto a differential reward reinforcement schedule, where Athena gets different type of reward depending on her performance. An ok performance will get a verbal reward, a good performance will get an ear stroke, and an excellent performance will get a treat.

Once we have a clear, clean lift of Athena’s paw into my hand on vocal command, I’ll start raising my hand and maintaining the differential reward schedule with the aim of Athena raising her paw above her head to hit my hand. I expect this will take a month or so to be achieved reliably if I stick with the current 1-2 training sessions a day.

I’d also like to introduce another trick, perhaps in a few weeks. While I’d like Athena to give both paws on command, I feel like it would be good to start with some more active behaviours (as she is a bundle of energy). I think consolidating her ‘up’ command would be a great one to start, which could maybe move on to a ‘jump over’ command.

 

 

 

Why is Reflecting Important?

I know I’m not a very good trainer, I don’t need the structure of a reflection cycle to tell me that, I have Athena who has successfully trained me to do several behaviours, while I still struggle to get her to do ‘up’ on command. What the reflection cycle lets me do is identify the weaknesses, and identify why they exist. One of the reasons I like the Gibbs cycle is that the analysis allows you to contextualise the ‘why’.

I find this very useful, especially with teaching. In my experience, many of my teaching ‘failures’ have come from my own problems, either my own desire not to fail, or not being clear enough about what I wanted. And yet this impacts on my students more than it impacts on me.

For example, here I was rushing to get to the paw touch phase of Athena’s training, even though one of my main motivations was to have Athena relax about paws. I was focusing on my feelings about what was going on, and not hers. Despite being pretty damn familiar with all this theory, in practice I was making the same mistakes that many others do simply because I wanted to feel better about myself. Vanity in teaching is a dangerous thing!

 

Teachers should not be afraid of mistakes or failures. They are a natural consequence of learning. This is not a good training video, I make many mistakes, and that’s precisely why I’m sharing it. Please feel free to make use of it (the YouTube link is here, there is also a copy on the University of Edinburgh’s Media Hopper service) without fear of hurting my feelings, or Athena’s! Being bad at something makes it a lot easier to learn from it!

MOOCs as a mechanism for behavioural change

Jilly1 Comment

Have you always wanted to hear my opinions on MOOCs but been unable to bring yourself to search through the MOOCs tag of this blog (or read the papers, or look at Twitter, or . . . never mind).

Well it’s good news for you! The Human Behavioural Change for Animal Welfare conference did a great job recording all the talks, including yours truly. The full set of talks can be found here, but I would highlight Melanie Connor’s talk on the Duty of Care projefct and Anna Saillet’s talk on maintaining behavioural change.

You can watch yours truly here: