Flipping the Research Methods Classroom, Parts 1-5

1: Why Blended Learning is the Way Forward to Teach Research Methods

Since I have started teaching graduate courses on research methods, I have struggled with the way in which such courses are usually taught. Why do we still teach research methods from textbooks? Most academics will agree that you learn best how to do research by simply doing it, and the traditional lecture format, where students are rather passive, seems inappropriate to achieve this.

For this reason, my Leiden colleague Alexandre Afonso and I have spent the last two years developing a new way of teaching research methods to our students, transforming our existing course into a flipped classroom using blended learning. The flipped classroom was developed with the financial and material support of Leiden University’s ICTO program and the Faculty of Governance and Global Affairs. Alexandre and I will describe what our flipped classroom consists of, how we set it up, and our experiences teaching it.

The idea behind our Research Methods 3.0 course was simple: in order to be able to spend more class time on practicing hands-on research skills, we needed to move some of our teaching online. In our department, graduate courses often only meet once a week and the duration of a typical course is 7 weeks, too short a time to cover the full range of social science methodologies. We quickly learned that our students did not need such a broad survey to carry out their own thesis projects. This led to our second decision: not only would part of the course be moved online, but it would also be modular, allowing students to pick and choose which research methods they wanted to specialize in.

The redesign of our course then centered on two parts. The online component contains a small number of learning modules, combining short video presentations of no more than 7 minutes with online readings, quizzes and assignments. The online modules focus on the most important research methods and techniques in our field (e.g. qualitative interviewing, survey analysis, regression analysis, case studies, etc.). The basics for each method is explained in the video presentations, after which students are expected to do a small number of readings and make a larger assignment. We also built a number of modules around academic skills. These modules targeted specifically those skills that we know our students sometimes struggle with: finding academic literature in repositories and specialized databases or – in an international program like ours – academic writing in English. The modules are accompanied by a podcast channel on which our colleagues discuss their use of particular methods in their research.

We also found it important to have a face-to-face component of the course, during which students could engage in group-based learning activities and receive direct supervision from a faculty member. Initially, this was going to be a series of in-class seminars during which students would be able to apply the insights gained through the online modules to their individual research projects. We found out — the hard way — that students would be at varying stages in their thesis project by the time they took our course. This complicated how we could teach the seminars. We therefore replaced the seminars with more general workshops tied to particular online modules, during which students would do assignments and exercises designed by us, rather than work on their own research projects.

The flipped classroom on research methods works like a toolbox that students can use to learn the methods they find useful for their thesis. It is flexible and not constrained by time like usual lectures: students can watch as many videos as they want, as many times as they want. Contact time in class is freed up as much as possible for interaction and discussing the concrete application of research methods.

Throughout the whole process of designing and then teaching our flipped classroom, we were lucky to receive the strong financial support of our Faculty. Being new to the game, however, we quickly discovered how difficult it is to transform a course like ours. In our next post, we will talk more about the practicalities of building a flipped classroom.

2: How to Build a Flipped Classroom

University teaching is not very different from the way Adam Smith or Max Weber taught a century or more ago. Aside from the inescapable PowerPoint, there is usually a lecturer standing in front of a group of students who take notes. The reason teaching stayed the same may be purely path dependent: departing from this format may go against administrative rules and habit. Developing new ways to teach requires an investment in resources in time and energy that always run scarce when the new semester looms. At Leiden University, we are lucky to have a great deal of institutional support and a clear commitment from the university for developing innovative forms of teaching.

And this meant . . . going to the film studio!

Leiden University wants to play a leading role in the development of open educational resources. There are several Leiden-originated massive open online courses (MOOCs) on Coursera, including a course on kidney transplants by the Leiden University Medical Center and our colleague Bernard Steunenberg’s MOOC on Politics and Policy in the European Union. Over the years, the university has developed quite an infrastructure to make these MOOCs, with several studios on campus to create instructional videos. Also Leiden’s Online Learning Lab employs professional videographers and instructional designers who specialize in online learning and in helping faculty members such as Alex and myself make the jump to online video content.

We had a list of demands that we wanted for our flipped classroom. These were, in no particular order:

1. No talking heads: we’re not movie stars and felt uncomfortable with being so prominently featured in our instructional videos. We also did not see the added value of forcing students to stare at our faces for 7 minutes at a time. That was exactly why we wanted to avoid the lecture hall in the first place.
2. No waste of time: the instructional designers had planned for us to do an online course on online teaching, a day-long session on curriculum design and then spend several weeks in the studio filming the videos. Since this was not a project on which we could work full-time, this was simply impossible for us to do.
3. No complicated technology. Time is limited, and we needed a format with a low learning curve.

Our instructional designers Joasia and Annemieke were entirely willing to adjust the standard procedure and we agreed on a few compromises:

1. Instead of us being visibly present in each video, only the brief introductory video (+/- 30 seconds) to each module would feature one of us welcoming the students and providing a general outline to the module. Remaining video would consists of animations with a voiceover.
2. Limiting the length of “talking head” sequences meant less time in the film studio. Instead, we made Prezi presentations that video-captured and added a voiceover narration to. While this meant one of us (Natascha) had to quickly learn how to use Prezi, we had the freedom to make the videos outside working hours in the privacy of our respective homes. And since nobody would be able to tell anyway, why not get comfortable in PJ’s and a glass of wine on the side?
3. Since our online modules did not contain anything crazier than a few videos and an quiz, we were able to build the entire flipped classroom within our existing course webpage on Blackboard. Again, having to build something brand new from scratch saved us some time.

In our next post, we will discuss what our face-to-face workshops.

3: Going Offline 

In our two previous posts, we wrote about the general idea behind our flipped classroom in Research Methods and producing the content for our online modules. Today we will discuss the offline series of workshops in which students gain hands-on experience with the research methods or skills they studied online.

When you do a big blended learning project such as this one, it’s very easy to get carried away by the new and shiny part: your attention will go predominantly to the online content. This makes sense: the online component is often not only the novelty aspect of the course, but also the more time-consuming part to produce, and the one that will last. It’s very easy then to fall into the trap, as we did, of not paying enough attention to the more familiar offline part of the course.

When we applied for funding to set up the flipped classroom, our idea about the offline component was as follows:

 Students will be stimulated to go back and forth between the theoretical material online and the concrete application of the methods in class. This course design will stimulate a much more experiential learning process than in a traditional research methods courses, as the course will assist the students in “learning by doing” research. The learning experience is also much more interactive than in a traditional course setting, as students are actively involved in each other’s research projects, jointly handling common challenges involved in doing research during the course seminars.

We were wrong. Our initial idea – to have students do the workshops, as they were writing their thesis – presumed that all students would at the same stage of the thesis project by the time they took our course. This was not the case: some had started but switched topics, others were already quite advanced and still others had not even started thinking about a thesis topic yet. In other words: the activities we envisioned them to do (e.g. carry out a qualitative interview with a respondent) simply bombed. We had to find a plan B.

It took a few iterations of teaching the course to find a good design for the workshops. We first dropped the ambition of connecting the workshops to the thesis projects. Instead, we designed specific workshop assignments that all students had to complete. Not all students appreciated this switch. Those who had already made some progress on their thesis felt that they were doing “double work.” Some students even asked why they were paying university fees to watch videos. After a few repetitions of the course, these criticisms receded. The course is no longer presented as explicitly linked to the thesis project, and students have come to appreciate the advantages of this format.

Other students objected to having to participate in academic skills workshops. They felt that skills like academic writing belonged in an undergraduate program, but should not be taught at the graduate level. As the people who read their writing, we strongly disagreed with this point of view. At the same time, we felt that we could give students the opportunity to prove us wrong. Now all students still need to complete the academic skills modules and answer questions on them at the final exam, but participation in the skills workshops is voluntary.

So what then do we do during the workshops? Just a few examples: mock interviews, peer editing of written work, locating and analyzing historical documents, carrying out a criminal investigation to apply process-tracing. Before every workshop, students need to submit a draft assignment linked to these exercises. During the workshop, students do the exercises and we discuss strategies to successfully complete them. Afterwards, students are allowed to revise their assignment once more, based on what they learned in the workshop.

We have found that students now score much better on the assignments and on the final exam. What’s more, we were surprised to notice that we actually achieved what we thought our initial course design would do: our colleagues who are supervising our students’ thesis projects tell us that students are now able to do things that used to require a lot more guidance from their supervisors.

We learned from developing a flipped classroom that you can be wrong, and that students are not always willing to try new and risky things when it comes to their education and perhaps their job prospects. But with trial and error, we can really improve the ability of students to acquire skills.

In our next post, we will tell you how we tried to improve “research-based” teaching in our department through this flipped classroom project.

4: Online, Open and Collaborative?

So far, we have written about the general idea behind our flipped classroom, the online environment we designed for this course, and the offline workshops that we organized around our online modules. Throughout this project, we wanted to design a course that would be as open and transferable as possible. On the one hand, this meant creating content under a Creative Commons Share Alike license, so it could easily be shared online. We also wanted to involve our colleagues who know much more about certain specialized research methods than we do.

When we designed the course’s online component, we included a section in which our colleagues told our students about their own research projects: which methods they used, what they thought were the advantages or disadvantages of these methods, and what they wished they had known before doing their research. We initially wanted to ask our colleagues to write on a blog, but we didn’t want to increase what students had to read or watch, so we eventually settled on podcasts. Alex has now recorded a number of podcasts with some of our colleagues on a range of topics, such as the comparative approach in researching tax policy, doing interviews with EU and Commission officials, or social network analysis applied to fair trade networks.

Involving colleagues created a new problem for us. Interviewing a colleague on, say, social network analysis is one thing. But if a student subsequently asked us to teach social network analysis, then we would not have the relevant expertise. So when applying for additional funding for this project, we asked for and received funding for colleagues to design modules for us in their areas of expertise. This meant that we could broaden the range of methods offered in our course quite extensively, while reducing our own role in it – something we felt would contribute to the project’s continuity of we stopped teaching the course at some future point in time.

Then things got out of hand. Our faculty board was happy to oblige with our funding request on one condition: could we use the project to do research within our faculty on open and collaborative teaching? Fast forward to September 2017, when a research team of four frantically bombarded colleagues in the Faculty of Governance and Global Affairs with e-mails and flyers to ask that they please, please participate in our survey on online and open educational resources. We are still analyzing the survey data but here are some insights that we can already share with you:

• A blend of online and offline is the preferred method among students for a course on research methods: interestingly, there is very little support for an exclusively online format.
• 92 percent of staff consider using open and online materials in their teaching, showing a high potential for these tools if the right infrastructure is made available.
• Staff often thought online materials saved time—they can be re-used, while lectures have to be delivered identically year after year.

Our last post in this series will discuss what students thought about the course.

5: Do Students Like Our Flipped Classroom?

We have now covered almost every element of our flipped classroom project: why we wanted a blended learning approach to our research methods course, how we designed the online modules for course, how we organized offline workshops for students to practice new methods and skills, and how we got our colleagues involved to develop new curriculum for us. We have saved the best for last: what did students think of our redesigned course?

When you undertake a big project like this, a project that requires so much time and energy, you are bound to get very invested in it. We were. We thought the flipped classroom was awesome. So when the course evaluations came back in, we were happy to read that students generally really liked the flipped classroom. Specifically, students liked the flexibility that the new course design gave them, they liked the look of our online environment and they thought the modules themselves taught them the relevant basics of the methods they needed to know. A few students even mentioned their appreciation for the academic skills modules! As it happened, they had been struggling with certain skills like academic writing, but as graduate students had felt embarrassed to ask for help.

Of course, not all evaluations were positive. In fact, some students had quite the opposite experience with the course: they had found the workload to heavy, thought the online format too impersonal and were dismayed with the experimental nature of the course. One student wrote, for instance: “Why are these teachers using us as lab rats?” We had been prepared for this type of negative feedback. Our instructional designer had already warned us that some students simply resist new forms of teaching, especially when they require students to be more actively involved in their own learning experience.

The comments that really surprised us were the ones along the lines of: “The teachers are only using this format to reduce their own workload. Why can’t they just give a traditional lecture instead of letting us do all the work?” Comments such as these were so much removed from our own experience working on the flipped classroom (so much work!) and our conscious choice not to do big lectures (not suitable for this course), that they really caught us by surprise. They taught us that, as we had been so focused on producing course content related to research methods, we had completely overlooked the fact that we needed to communicate much more clearly to students why we were doing this. We now start each new edition of our course with an introductory meeting that explains to students the rationale for choosing a blended learning/flipped classroom format, a message we keep repeating in the workshops and online.

All in all, it’s been quite the ride. We have spent a lot of time and energy into designing and teaching this flipped classroom in research methods. We have made mistakes along the way, but we have produced a course that we are proud of. Most of all, thanks to this project, we were able to spend a lot more time not just to course preparation, but also to its evaluation. This has taught us quite a few things about how our students are experiencing our teaching, that we had not been aware of before. So far that alone, we are now better off than when we started with this project.

With that, we’ll end this series of blogs. Thanks for reading along and feel free to send us your thoughts or suggestions. We look forward to hearing from you!