Introduction
This paper presents a design-oriented approach to address the challenge of bridging the gap between educational research and practice by empowering teachers to make use of research findings in their classrooms. Existing teaching practices often diverge from what is known in research as being effective (McGarr et al., 2017; Haigh, 2009; O’leary et al., 2015; Nuthall, 2008). Although a proclaimed aim of research in education is to improve the quality of teaching and learning (Century & Cassata, 2016). One of the challenges in using research to inform teaching practices is the time and effort it takes for teachers to consult research insights and apply them to their own practice (Nuthall, 2008). has proven to be difficult to make the results of educational research benefit teachers. Several studies consider the role and potential of using research to inform centrally developed, large-scale implementation processes (Tamborg, 2021). Such studies have developed important guidelines, including that, even though being informed by research, centrally developed initiatives should make room and support for individual adaptation, and that they should ensure compatibility to existing practices and initiatives (Larsen et al., 2019; Kjällander, 2021). Research has focused on how teachers can act as designers in participatory design and to empower teachers and include them in designs for learning (Knutsson & Ramberg, 2018). While there has been an extensive focus within the Scandinavian participatory design tradition of testing out methods to empower teachers in design problems, less attention has been given to the combination of such methods to address the theory-practice gap.
This is indeed a challenge worthy to discuss, as it is time-consuming and complex for teachers to individually consult research insights in the search for inspiration of how to best improve a given aspect of their teaching. Designs for learning could be potentially be a solution to this problem. As Knuttsson and Ramberg (2018) puts it, “The Scandinavian participatory design tradition is not only about design of new technology but also change and development of human’s’ thinking, organizations and communities ways to work and deal with problems” (p. 2). Using research as inspiration to improve teaching is a challenge since research in almost every aspect of education is comprehensive in size (and continuously growing), is often written in a technical language and only small parts of a given paper/research report are likely to be relevant for the individual teachers’ need. Despite these challenges, the potential of empowering teachers to make use of research findings from their initiative is, however, supported by the literature. Studies have found the success of implementation processes to be dependent on that teachers are involved in the development of the innovation to be implemented, and that the innovation fits the needs, routines, and practices of the end-users (Ärlebäck, 2017; Kuzle, 2017; Koichu & Pinto, 2019; Larsen et al., 2019). One could argue that the shortest way to meet these guidelines is to support teachers in better overviewing, interacting with, and making use of educational research. In this paper, we report from an experiment combining data science methods and a participatory workshop format seeking to achieve exactly this.
During the last decade, data-science methods have gained momentum in several scientific and practical disciplines in the humanities, and a series of different formats are now available that seek to facilitate bridge-building between diverse stakeholders (Venturini, Munk, & Meunier, 2018). However, these methods have to a large extent remained unexplored in the domain of educational research. In this paper, we explore the potentials of workshop formats as sites for researcher/practitioner partnerships to explore the purpose of exploring the practical value of research on the use of digital tools in mathematics education.
The workshop format investigated in this paper uses topic modeling analysis of research literature on digital tools in mathematics education, which were presented and discussed by mathematics teachers and researchers in the field of mathematics education. Using digital tools in mathematics teaching is full of dilemmas. If the tools are used, it can promote learning barriers, as students do not necessarily learn basic structures of mathematics (Jankvist & Misfeldt, 2015). On the other hand, if they are left out, students do not learn about the possibilities digital tools provide for exploring mathematics (Dreyfus, 1994). As such, divergent understandings of using digital tools arise among mathematics teachers and can complicate teacher collaboration and create unclear learning situations for students (Jankvist, Misfeldt & Marcussen, 2016). Topic modeling is an automatic method of content analysis used within the field of text mining. It is an unsupervised machine learning technique that uses statistical models to identify abstract topics or themes within large collections of textual data (Boyd-Graber, Hu, & Mimno, 2017). The workshops were thus based on pre-processed literature coming from a complex field of study, which was visualized in a format that allowed diverse workshop participants to explore and interact with it. The purpose of this article is to explore the challenges and potentials of this format to support teachers in being able to make use of research results. We begin the paper by briefly reviewing the literature on the theory-practice gap, design oriented approaches to establish partnerships between researchers and teachers and formats for involving teachers in data use.
Related work
The theory-practice gap in education
The term theory-practice gap is a widely researched and long-standing problem in the field of education and refers to a situation where existing practices conflict with what is known in research as being effective practices (O’Leary et al., 2015). Previous research has identified several explanations for the existence of this gap, including relational problems, practice failing to reflect theory, and theory being perceived as irrelevant to practice (Greenway et al., 2019). Relational problems refer to the institutional separation of practitioners, and researchers often imply that teaching is planned without direct input from researchers.
Practice failing to reflect theory refers to the challenge that practitioners either may be unwilling or unable to integrate research findings into their practice, or that they might interpret the research results differently than researchers and enact them differently than envisioned. Theory perceived as irrelevant to practice is described by Haigh (2009) as almost being natural since theories and results developed from research eventually become outdated. As noted by McGarr et al. (2017), for a teacher to change his practice according to research findings is not simply a matter of acceptance of relevant knowledge, but also an acceptance of research and researchers as an external authority to determine how teaching can and shall be carried out. Moreover, not all types of research on teaching efficiency are fit to inform teachers’ practices, either since they confuse teaching with learning, are not generalizable, or that they report results from a large and complex project, and fail to distinguish between relative components of the entire program (Nuthall, 2008). As we describe below, one suggested way to tackle this gap is by establishing partnerships between researchers and teachers.
Design-oriented approaches to establish partnerships between researchers and teachers
As noted by Kallio (2022), an important aspect of establishing effective partnerships among researchers and teachers is reaching an agreement on what problem such partnership should address. Often, researchers and teachers have different conceptions about what the main problem in a given context is, and reaching common ground on this matter is important for collaboratively developing adequate initiatives (Kallio, 2022). To reach such common grounds, Kallio (2022) formed a so-called Network Improvement Community (NIC) between researchers and teachers working with personalized learning (PL). NIC is an approach to bring researchers and teachers together, which pays specific attention to supporting collaboration on identifying problems in practice, their root causes, and to build robust ways of solving the problem through an iterative series of designs (Bryk et al., 2015). The concerned teachers in this study were experts and worked at the cutting edge of research on PL, and they often encountered a lack of research to support their decisions (Kallio, 2022). The specific NIC in this study was established for teachers and researchers to share and improve practices on PL and, later on, to scale these practices. This study found that The NIC partnerships were supportive in reaching common understandings of the problem(s) to address (Kallio, 2022). Yet, it was not straightforward for the partnership to identify adequate ways of solving the problem(s), which, needless to say, is equally as important as to agree on the problem.
In another recent study, Dirckinck-Holmfeld et al. (2019) described the potentials of participatory data design (PDD) as a format for teachers and researchers to engage in partnerships. PDD is an approach in which researchers and local teachers collaborate in collecting, interpreting and making use of available data that have the potential of informing and improving teachers’ practices (Jensen et al., 2017). When teachers are given access to data in educational contexts, this data is often provided without involving teachers in the choice of data and how it is processed and visualized. In some cases, are neither involved deciding how the information conveyed in the data should be converted into concrete decisions and actions. By involving teachers in decisions on these matters, Dirckinck-Holmfeld et al. (2019) found that PDD could aid what Wenger et al. (2002) refer to as an evolution of the community of practice, rather than imposing a foreign and predefined structure onto it.
Kjällander (2021) explored how workshops with researchers and teachers could support the development of a digital educator’s tool. To analyze the workshops, Kjällander (2021) used Learning Design Sequence (LDS), which is a model that is normally used to design and analyze teaching in classrooms. However, they adapted the framework in order to establish an environment where the participants could cooperate to develop the educators’ tool. The study concludes that the educators acquire a more specific language that is closer and more common to the curriculum, such that the tool can be trusted by educators. Thus, the workshop format provide a foundation for establishing partnerships between researchers and practitioners, and in some ways bring research and practice closer.
Closely related, Ørngreen and Levinsen (2017) conducted a review on workshops as a research methodology specifically for e-learning purposes. They suggest that workshops can be divided into three categories; workshops as means, workshops as practice, and workshops as research methodology. They argue that workshops include shared agendas and often include experienced people as well as promote genuine participation in groups that are of sizes such that all can be heard:
The workshop co-constructs a place for collaborative negotiation of meaning-not only between participants, but also between facilitators (the researchers) and participants, who both during and after the workshop adopt and adapt to what is being discussed, performed and learned. Through this, workshops bring us close to practice without being in practice (p. 78).
Research seem to indicate that researcher and teacher partnerships hold the potential to support teachers in identifying problems and in developing solutions to address them, and that workshops are especially fruitful for minimizing gaps between participants and their knowledge. Even more, research suggest such partnerships may be useful for developing teaching practices based on both research, as in the case Kallio (2022), and on data, as in the case of Dirckinck-Holmfeld et al. (2019). As we describe below, there are, however, different conceptions of to what extent and how teachers should be involved in choosing, interpreting and making pedagogical decisions based on data.
Formats for involving teachers in use of data
The potential for using research and data for improving teaching and maximizing learning has been discussed heavily during the last decade, and several large research projects have sought to change teachers’ practices based on research evidence. Perhaps one of the most cited of these projects is Hattie’s (2008) comprehensive meta-review that led to principles for teaching described in the volume Visible Learning. We can describe Hattie’s (2008) approach as being rather top-down in that insights from research are thought of as the main authority for identifying what to change and implement, whereas the voice of the individual teacher is given less emphasis. A slightly different project developed in a Nordic context is that by Qvortrup (2017) and colleagues, which sought to inform teachers’ professional judgment by providing teachers with data on their students’ performance and well-being. The main idea in this project was that teachers’ should be invited into the process of interpreting collected data and exercise professional judgment based on this foundation.
This idea of involving teachers in developing interventions or modifying their own teaching is supported by insights from the recent interest of exploring the issue of implementing research findings into practice (Jankvist et al, 2017). A central insight developed in this strand of research is the need for involving teachers in the development of the innovation to be implemented and in adapting it to the individual teachers’ needs and existing practices (e.g., Ärlebäck, 2017; Kuzle, 2017; Larsen et al., 2019). In the context of this paper, one implication to draw from these results is that it is problematic to assume that best-practice as identified in the research, can be implemented 1:1 into teachers’ practices. Rather, research findings must be adapted to fit the needs and practices of the individual teacher who is envisioned to use it. According to Koichu and Pinto (2019) implementation processes as such often inhabit an assumed distinction between researchers as the producers of knowledge (innovation designers) and teachers as the consumers of this knowledge (innovation enactors). As these authors note, tensions related to the alignment of these two actors are frequently referred to as an issue in implementation processes (Koichu Pinto, 2019). Much like the theme of this special issue, they suggest thinking of the researcher/teacher relations in implementation processes as partnerships rather than as two separate agentic entities (Koichu & Pinto 2019). In the eyes of Koichu and Pinto (2019), such a partnership includes that teachers should be directly involved in the research leading to innovation designs. As noted by Dreyøe (2019), there are indeed several steps preceding the implementation of a given research-informed change of practice. In the context of this paper, particularly three of these are important, namely that teachers should be involved in 1) selecting what research literature to inform the change, 2) interpreting this literature so that it can inform a given change, 3) implementing that change in the classroom.
Building on the guiding principles of involving teachers in the development of the innovation to be implemented, teachers should ideally be involved in each of these three steps. The remaining question is thus what formats support such teacher involvement when we are aiming to support teachers in navigating and interpreting large amounts of complex research literature.
One of the promising formats to achieve this is so-called data-sprints. According to Venturini, Munk & Meunier (2018), the workshop format data-sprint has the potential to actively involve users in generating insights usable for their own and others’ practice. Data-sprints are characterized by deep user involvement, by not just implementing results, but also by selecting, visualizing, and interpreting the results. Participants with different backgrounds are brought together in data-sprints to collaboratively work on a set of data and research questions. This format has been used extensively within science and technology studies (Munk, Venturini, & Meunier, 2019). According to Venturini, Munk & Meunier (2018), a data-sprint is characterized by being preceded by long and intensive preparatory work at the beginning of the sprint, so that most of the research infrastructure already is provisioned and prepared. Another key point is that data-sprints are not open in the sense that only invited people can join, as they need to have the right competencies in order to contribute. A sprint consists of six phases; 1) Posing research questions, by the issue experts invited. 2) Operationalizing these questions into feasible data science projects. 3) Provisioning and preparing the datasets. 4) Writing and adapting code. 5) Designing the data visualizations and interface. 6) Eliciting engagement and co-production of knowledge (Venturini, Munk & Meunier, 2018). However, it is not so that all of these phases must be apparent for it to be a data-sprint – the phases rather reflect what can be expected.
To our knowledge, data sprints have yet to be implemented in educational settings. In this paper, we seek to contribute by examining to what extent and how data sprint workshops. More specifically, we explore data sprints in relation to teachers’ difficulties of integrating research findings in their practice, which is relate to practice failing to reflect theory (Greenway et al., 2019). As data sprint is a relatively new method, we take an open and explorative approach by addressing the following research question_
What are the enablers and barriers related to facilitating data workshops based on visualizations of research literature aiming to support teacher dialogue about the use of digital technology in mathematics education?
Context
As described in the introduction, this paper draws on experiences from a data sprint workshop based on topic model analyses.
Method, data collection, and approach to analysis
This paper is based on experiences from a data sprint conducted in a partnership between three teachers and two researchers. The research design is based on an experimental workshop format that sought to support teachers in engaging in dialogues about the use of digital technology in mathematics education. In this respectsthe data-sprint provides a setting to study the theory-practice gap, in that we aimed to support teachers in more direct usage of research literature by providing a scaffolding, visual overview and support from researchers who were familiar to the discourse in the concerned academic field of study. The participants in the data-sprint are thus both researchers and practitioners. It is thus possible to study the differences among these groups when they interact with research. It is the shared patterns among the participants that are of particular interest to this study. Creswell (2002, p. 470) describes shared patterns as “common social interaction that stabilizes as tacit rules and expectations of the group.” They consist of behavior, belief, and language. Time is of the essence for a group to adopt shared patterns, in our case we are interested in understanding the challenges and potentials, which is why we are interested in studying the clashes between the shared patterns of practitioners and researchers. We are thus looking for discrepancies among ideal (patterns that should occur) and actual patterns (that actually occurred), by raising these as questions to the participants.
The data and method in this paper consist of two layers per se. The first and primary layer is the evaluation data, entailing the video, and audio recording of the evaluation of the workshop. 2) The main outset for the work in the workshop is the topic model visualization and abstracts of dominant research papers. From here on they will be referred to as 1) Evaluation data and 2) Topic modeling data. One important note is that the participants did not choose the data per se in this data-sprint as it was predetermined, however, all the participants were informed of this choice before joining the workshop, this can be seen as an indication of interest in the data.
Data-sprint
There are several key elements from the data-sprint that influence the challenges and potentials that arise from using this format in order to support teacher dialogue. The first and potentially the most obvious is the choice of data source(s). In this case, the choice fell upon research literature on technology in mathematics education. An important characteristic of the data is that it must be relevant for the participants, and this data is relevant for the researchers as it is their primary product of work. For the teachers, literature is something different, as their practice is one of the primary objects of study in the literature. The common aim for the participants was to understand and interpret the topics addressed in research, to get insights into research, its results, and how it potentially can affect practice. The promising potential of this workshop is that it allows teachers to explore a large body of knowledge through an intuitive and dynamic interface.
Data input for the topic model
The second layer consists of all research papers in the technology-focused working groups from the Congress of the European Society for Research in Mathematics Education (CERME). We applied topic modeling1 on all these 336 papers, which resulted in 25 distinct clusters. Topic modeling is a machine learning model that identifies hidden semantic structures in a text corpus. It combines two assumptions: 1) that a research paper is composed by a mixture of topics, and 2) that a topic is a weighted distribution of all the words in the collection of research papers. Each topic is formed by grouping words that often co-occur in a paper, thus indicating the semantic meaning in a text. Topic modelling is a heavily documented and tested way of identifying hidden semantic structures in large corpora and bodies of text (Blei, Ng, & Jordan, 2003). A topic model can be visualized dynamically via pyLDAvis (a python package), see Figure 1.
Figure 1
pyLDAvis visualization of the topic model, with topic 10 selected.
By clicking or hovering a topic on the left panel, the most relevant terms for this topic are shown and ordered by relevance on the right. For each of the topics, we also identify the top five most prominent papers. Based on the visualization in Figure 1 and the five, most prominent papers’ titles and abstracts for each topic, the topics’ meaning and discourse can then be interpreted in the context of the common theme of the entire corpus. A typical analysis of a topic would be to identify a label (short or long), which adequately specifies the discourse made up by these two sources.
Evaluation data
This data consists of the evaluation of the data-sprint. The participants were two lower secondary school mathematics teachers, one mathematics teacher educator, two Ph.D. students in mathematics education (one of them is the first author of this paper), and one Post-Doctoral researcher in mathematics education (the second author of this paper). The data-sprinting approach supports participants in mapping complex problems and still maintaining the individual actor’s perspective on the problem. At the end of the data-sprint an hour-long evaluation/focus group was conducted to collect data on the process. The evaluation was conducted as a semi-structured focus group. The evaluation had a number of aims, one was uncovering how and in which way topic modeling and data-sprints can create a shared understanding of challenges in mathematics education research and practice. Another aim was to understand the potentials and challenges this method has for teacher-researcher collaboration, and how it can impact their practice. These overarching aims were operationalized to nine concrete questions. The first author of the paper led the evaluation by sticking to the nine questions.
Data collection and approach to analysis
Interviews can provide researchers with respondents’ accounts of what they do, how they do it, and when they do it (Kvale & Brinkmann, 2015). More importantly for this context, interviews give the researcher insights into respondents’ accounts of why they acted as they did in a given situation—that is, their underlying reasons.
In order to analyze the data from the evaluation, this data was audio-recorded. This recording was then transcribed and logged in a spreadsheet, containing information on each utterance, the speaker, and the time. As described, our research question takes a particular focus on identifying challenges and potentials related to facilitating data workshops based on visualizations of research literature aiming to support teacher dialogue about the use of digital technology in mathematics education. Similar to previous studies (e.g. Duffy et al., 2014; Philip & Cameron 2008), we take an open and exploratory approach to identify barriers and enablers rather than applying a theoretically informed definition/classification typology. An open or grounded approach to analysis is often characterized by seeking to explore and understand data in its own right rather than to apply or validate concepts developed from other contexts (Tan, 2010). Since workshop formats based on data science methods is a relatively new phenomenon in empirical educational research, we find this open approach fit for supporting us to gain initial insights into teachers’ experiences of such methods. To support our analysis, we however draw on an understanding of enablers as characteristics of the workshop that allowed the teachers to engage in research-based dialogues about using digital tools in their mathematics education. Likewise, we consider barriers as characteristics of the workshop that prevented the teachers from engaging in research-based dialogues about using digital tools in their mathematics education. In our approach to analysis, we initially processed the evaluation transcripts to identify instances of enablers and barriers. New, we investigated recurrent themes within each of the two categories.
This process led to two main barriers and two main enablers from the workshop. The barriers concerned perceived difficulties related to the complex language used in the paper and the fact the topic modeling analysis processed the text corpus independently from the teachers’ interest. The enablers regarded that the workshop sparked interest in research concepts, and perceived opportunities that the workshop format could minimize the research/practice gap. Below, we describe these challenges and potentials in greater detail and provide empirical examples from the interview to illustrate their nuances.
Results
In the following section, we present our analyses of data on the teachers’ evaluation of the workshop. The analyses are organized in two separate sections focusing on the challenges and potentials respectively. After the analyses, we will discuss the relationship between the challenges and the potentials, point to improvements of the workshop format based on insights from analyses, and situate the contributions of this work about existing research in the field.
Barriers
Barrier #1: Research vocabulary is perceived as foreign to teachers, and there is a need for researchers to support interpretation
A recurrent theme in the teachers’ evaluation of the workshop was an experience that the data visualizations of the topic model included some concepts esoteric to research, which were perceived as foreign to teachers. This is illustrated in the two quotes below uttered by two different teachers.
There were several categories that I found difficult [to name, red.]. (…) But again, I think it has to do with the difference between the language of researchers and of practitioners. Research tends to have very clear concepts about things, which I don’t know much about (l. 68).
During the workshop, I sometimes thought: this is a word I have heard before, but it’s miles away in my memory (l. 71).
A key feature of topic models is that they do not carry meaning prior to the interpretation of the topics they include. Conducting such interpretations is done by developing meaningful labels that compile the set of terms that form a given topic. Consequently, such interpretations require prior knowledge and an overview of the field that is being modeled. The fact that the teachers were not in possession of either led to difficulties in interpreting and interacting with the mode. This is illustrated in the quotes below:
Researchers who are not teachers themselves sometimes use an academic language where it’s difficult to keep up if you are not immersed in it. This creates a gap, where the texts are not accessible because it’s too abstract and does not get used, which really is a shame (l. 75).
As highlighted above, these teachers conceive the model as abstract, inaccessible, and therefore difficult to make sense of. In our design of the workshop, we had to some extent anticipated this challenge, which was why not only teachers but also researchers partook in the workshop. Our intention with setting up such a team was that researchers could support translating and explaining terms to teachers to ease the interpretation. In the evaluation, this need was echoed numerous times by the teachers and is illustrated in the quote below.
If we had only been practitioners present today, we would have been screwed, I think. There are simply things and words here that we know nothing about.
This is also a viewpoint that was expressed by the researchers, who even appeared to be genuinely surprised that teachers recognized some of the terms.
I was completely surprised that some of you were able to recognize elements of the model before. I have seen these topic models before and wondered how you would react to it because it’s text that is written on our [researchers, red.] premises. After all, it’s research literature that is first and foremost written to other researchers.
This need for researchers to support the interpretation of the topics brought with it some consequences, one of which being that it was difficult to involve practitioners in the dialogue on equal footing. Not knowing the discourse in the data set positioned the teachers as reliant on the researchers’ interpretations of the concepts that were foreign to them. This is supported by the following quote from a researcher, who here explains his perception that it was difficult to give voice to the practitioners and their perspectives in the dialogue.
I think that one of the things that made it challenging to give the practitioners a voice in the discussion was that, everything that had to be classified, in itself did not have any relation to practice. At most, it was a studied practice, articulated in a very specific discourse.
Another consequence related to the foreign language and discourse in the data is that while the teachers had been introduced to new concepts in the workshop, these still came across as abstract, detached from practice as they knew it and therefore difficult to act upon. As illustrated in the quote below, the limited usefulness of insights from the workshop both concerned the potential for converting the insights into new classroom practices, but also for sharing these insights with peers.
I think it has been interesting, but that it will be difficult for me to actually do something new for my students or my colleagues. I think that it is too bad, but I feel like I need a few days because now it is pretty erratic. It is not very tangible for me for the time being (l. 3).
This quote indicates that the knowledge gained by the teachers at the workshop was at a surface level, where they neither felt capable of acting upon nor sharing these insights with others. While the participation of researchers perhaps addressed the most imminent problem related to knowing and to some extent understanding the concepts of the model, it did not seem to be sufficient for enabling the teachers to bring insights back to benefit their school practices. This issue is related to a second challenge concerning the more general role of practice in the workshop, which we will address in the following.
Barrier #2: Having visualizations be directed by problems in practice instead of the results from a purely grounded analysis
Per definition, topic modeling is an approach that processes a text corpus on its premises in the sense that it identifies hidden semantic structures otherwise known as topics based on a statistical model. The analyses above showed that the teachers participating in the workshop experienced a gap between research and practice partly due to the difference between research discourse and mathematics teaching as is experienced by practitioners. Due to this gap, the teachers problematized the topic modeling approach foregrounds research topics and does not take issues or themes from the classroom into account.
Teacher: Now, I haven’t really brought my practice into the discussion. If I had done that, the story would be completely different, and I actually miss that.
Since the topics and terms came across as abstract and complex, the teachers found connecting the model to their own practice experience and needs difficult. Neither the topic model nor our facilitation of the workshop seemed to support the teachers in anchoring their need as a starting point to engage with the model. While we envisioned that a topic model of research literature could inspire and help teachers to develop answers to practical problems, the workshop design had not supported the teachers in phrasing the problems that potentials answers should address. In the evaluation, both researchers and teachers suggested an approach where more emphasis was given to practice. This excerpt from a teacher is an example of that.
Teacher: I think it would make more sense to practitioners if it were closely linked to practice, for example by grouping them according to real practice problems. That would be awesome because it would make it easier to know where to look.
In the evaluation, this utterance was followed up by a researcher asking a clarifying question. The dialogue below shows this question and how it was responded to by a teacher.
Researcher: If one tried to mirror the topics to practical problems in the classrooms, could you then imagine it would be easier to suggest recommendations based on your discussion?
Teacher: Yes, because that would make it recognizable for practitioners. I definitely think that would make a big difference.
The teacher’s response indicates that a substantial part of the challenge to relate the topic models to practice was that of recognizability; the terms in the model seemed too far away from the everyday practices of teachers for them to use it. In the quotes above, it is not entirely clear whether this particular problem concerned that the statistical model failed to foreground “real practice problems” or whether the interpretation of the topics should have taken these practice problems more into account. In the discussion, we will reflect upon approaches to tackle both suggestions. Before, we will however illustrate two key potentials of the workshop that can be inferred from the evaluation.
Enablers
Enabler #1: Introducing and sparking interest in research-based concepts
The section above illustrated how the dense research vocabulary that formed the topics in the model made it difficult for teachers to interact with, which limited its perceived practical value. In the evaluation, the teachers on several occasions however uttered that a derived effect of this nature of the topics was that they were introduced to new concepts, which they found valuable to learn more about. One of the teachers participating in the workshop explained that he took notes during the session to write down and remember terms and concepts he did not know before. As illustrated in the excerpt below, this teacher explains that these new concepts sparked his interest in digging deeper into these concepts and their meaning.
I created a document where I wrote all the keywords I had not heard about before. I wonder why I haven’t heard about these before. (…) I think this is very interesting. When I read the terms, I just skipped many of them, because I didn’t know what they were, and I just looked at the ones I could relate to. It’s very cool to get a different perspective. And there are also things that I would read up on after this workshop, e.g. what TPACK and semiotic is, to get a deeper understanding (l. 49).
As this excerpt illustrates, the topic model presented what he describes as a “different perspective” on mathematics teaching with technology and allowed him to only focus on the concepts that resonated with his needs and horizon of experience. A central prerequisite for this potential, therefore, seems to be the presence of the researchers who could briefly introduce and unfold the meaning of concepts such as TPACK. Besides merely learning new concepts, this teacher also explained how new concepts facilitated an awareness of phenomena that he had not considered before. This is illustrated in the following excerpt:
One of the things that I wrote in my document, which I find extremely interesting, is gestures, the relation between the student and the screen. I have never thought about that. It is fascinating to learn more about it, even though I have never been presented anything about this in the teacher training program (…).
Although we argued in the previous section that the teachers perceived it to be difficult to act upon the insights they had acquired during the workshop, this teacher’s utterance however indicates that these insights potentially can lead teachers to build a more elaborated view of the processes at stake when teaching or learning mathematics with digital tools. As we will unfold below, there were also examples of how the format of the workshop indeed held the potential to minimize the perceived gap between research and practice.
Enabler #2: Minimizing the perceived gap between research and practice (bring teachers closer to research – not the other way around)
Despite the perceived gap between research and practice, we have described previously in the analyses, the teachers also saw several ways of refining and improving the workshop design that could minimize this gap.
In particular, the teachers expressed that working in-depth with the model and naming the topics could potentially help minimize the distance between research and practice. More specifically, this potential concerned the opportunity to link the topics to problems familiar to their teaching mathematics experiences with digital tools. By doing so, the participants envisioned it to be easier for practitioners to both navigate the research literature and make research more accessible and valuable.
But I think it would be cool because I do have that feeling of doubt about finding research. People do occasionally ask if I use research, and I’m like, hmm. So I definitely think that this is a great way to navigate research (l. 87).
I think that naming the topics and the model that we constructed today could make the distance between research and practice shorter (l. 15).
In line with the teachers’ critique of topic modeling, as it was conducted in this workshop format, as foregrounding research at the expense of practice, they see clear potential in redesigning the workshop by taking the outset in issues prominent to teachers, rather than what is prominent in the literature.
But I believe that it would make more sense for practitioners if it is linked closely to practice, what R proposed, to thematize them according to practice (l. 87).
Discussion and conclusion
The data-sprint, as it was conducted in this context, sought to establish practitioner/researcher partnerships with an aim of bringing practitioners closer to research. Through our analyses, we learned that there are both potentials and challenges regarding supporting teacher dialogue in the context of data sprint workshops. On the one hand, the workshop format has the potential to minimize the gap between research and practice (Venturini, Munk & Meunier, 2018). However, this gap only seems to be minimized from practice to research, as there was not much time spent on understanding and discussing the practical implications of the model to develop such shared patterns. This could be remedied by hosting a data-sprint with a longer time-frame as suggested by Venturini, Munk, and Meunier (2018) and Creswell (2002). This format does, however, have the potential to overcome the relational problems as described by Greenway et al. (2019). Another potential concerns practitioners’ opportunity to learn and discuss new concepts and phenomena that they might have never heard of or even thought of as important. This obviously opens new possibilities with regard to reflections on one’s own practice and potentially overcoming theory being perceived irrelevant for practice (Haigh, 2009). This data-sprint attempts to make it more apparent for teachers, that research can inform teaching practice (Mcgarr et al., 2017), by bringing research closer to practitioners and engaging them with the research.
The experiences reported in this paper show that it’s not only a matter of providing teachers with adequate data and research sources as in the case of Qvortrup’s (2017) and Hattie’s (2008) work. Regardless of how intuitive this information might be presented, support for interpreting this information is still needed. Our study shows that establishing practitioner/researcher partnerships can be an effective way of accommodating this need.
On the other hand, there were several challenges with the data-sprint as it was conducted. The fact that visualizations were guided by research instead of practice was problematic as the practitioners could not identify connections between the generated overview and issues in their teaching practices. The fact that the visualizations were constructed beforehand and, in a way, where it is not possible to design the visualization, but rather to manipulate them, also collides with the fifth phase of data-sprints, where participants are supposed to design the data visualizations and interface (Venturini, Munk & Meunier, 2018). Another challenge was the complex research vocabulary that had to be translated for the practitioners.
Although conducted on a small scale, the research presented in this paper adds to existing research on research-informed practice in several ways. Firstly, we have introduced a workshop format that uses data science visualizations to support research-informed teaching practice. While data-sprints as such are used in several contexts, it is rather new as a workshop format within the field of education. Secondly, this workshop format situates teachers as the primary enactors of research rather than the target group of an intervention based on research. This allows for a higher degree of context sensitivity, where teachers can act upon insights from research while considering their knowledge of the specific group of students they are teaching. Thirdly, our analyses have provided preliminary findings on the challenges and potentials related to this format, when it is shortened in length and pointed to improvements in the design of such workshops to address issues and accommodate potentials as they are expressed by the participants at the workshop.
We see two distinct connections between the challenges and potentials denoted in the analysis. Firstly, the potential and challenge regarding research vocabulary and learning of new concepts are closely related and together form a paradox; if we were to eliminate the need for translation, no new concepts would appear, thus making it impossible to learn new concepts. However, the opportunity to gain insight into new phenomena would still be possible, but the dialogue would not include research-based concepts that allow for greater detail and clarity than a dialogue solely based on the individual teachers’ practice experience articulated colloquially. This is a paradox as the introduction of research concepts both hinder teachers in engaging with the model and at the same time triggers new learning. Developing a shared language (Creswell, 2002), combining the concepts of research and the local anecdotal teaching experiences, can contribute to establishing practitioner/researcher partnerships (Koichu & Pinto, 2019).
Secondly, the potential and challenge regarding visualizations being guided by research instead of practice and minimizing the gap from practice to research are also closely related, but not in a paradoxical way as in the case above. The design of the workshop centered on research as the starting point for discussion, which was pre-supposed to be meaningful from the perspective of the practitioners. Our analysis shows that taking a starting point in research made it difficult for the teachers to relate it to their own practice. The role for teachers was thus perceived as to incorporate their practical experiences within a research discourse, which they perceived as alien. We see two ways of addressing this challenge. One way would be to keep the topic model in its current form and revise the approach to interpret it. Instead of merely interpreting the topics, we could have facilitated a dialogue with emphasis given to the classroom experiences related tospecific terms of topics. In this way, research would still be the outset for the discussion, but the role and relevance of practice experience would have been articulated stronger. An alternative is to revise the model altogether. Instead of taking an approach to topic modeling as an unsupervised endeavor, where the model clusters x distinct topics only on the premises of the text corpus itself, it is possible to construct a semi-supervised model integrating users’ domain knowledge via ‘anchor words’. By doing so, users can guide the topic model in the direction of these words, allowing for strategies that promote specific aspects important for a specific set of users. Anchoring encourages (but does not force) the model to search for topics that are related to these words, helping us identify topics of interest, enforce separability of topics as well as finding aspects around topics. A number of anchoring strategies can be enforced; 1) Anchoring a single set of words to a single topic (Can help promote topics not naturally emerging when running an unsupervised model), 2) Anchoring single sets of words to multiple topics (Can help to find different aspects of a topic that may be discussed in several different contexts), 3) Anchoring different sets of words to multiple topics (Can help enforce topic separability if there appear to be chimera topics that are not well separated). This approach is, therefore, more technically demanding than merely revising the approach to interpreting the model, but nonetheless seems to target the issue at stake more precisely.
In sum, this paper has contributed by introducing data-sprints as a mode for establishing partnerships between teachers and researchers with an aim of processing, visualizing research findings, and supporting teachers in interpreting these. Besides introducing this format and identifying its potential and challenges, our analyses point to several opportunities.
Future directions: towards aspects for teacher-engaging data-sprints
The study described and investigated in this paper is naturally limited by taking outset in a single case, in which all participants chose to partake in the data sprint due to their high interest in the topic. In that respect, it is important to onte that the findings of the study must be considered preliminary in terms of the potentials of data sprints as a participant approach to tackle the research-practice gap. On top of this, the data sprint workshops from which we report in this paper only to a limited extent succeeded in addressing the envisioned issues. However, we believe the experiences described above point to several important aspects to account for in attempts to bring research closer to practice. The first aspect concerns engaging teachers in choosing the data to ensure their motivation and the dataset’s perceived relevance. The second aspect is that the data should be processed on the basis of principles defined by the teachers. This aspect promotes the relevance for teachers’ own practice, as this is their anchor point. The last aspect is supporting teachers in interpreting the visualizations produced in the data sprint. All of these aspects heavily call upon activating and involving the teachers in the decision-making process. Furthermore, our analysis and discussion show that it is advisable to closely follow the principles and phases of the data-sprint design in order to create shared patterns—behavior, beliefs, and language—among the participants to establish practitioner/research partnerships.
Notes
[1] For a more detailed account of the exact topic modelling process see Herfort, et al. (in revision).
Competing Interests
The authors have no competing interests to declare.