Introduction
Although secondary analysis of qualitative data is not a new research approach, it is not yet commonly used in engineering education research. Heaton (2008, p. 34) describes secondary qualitative data analysis (SDA) as involving “the re-use of pre-existing qualitative data derived from previous research studies.” Within this broad definition, one form of SDA that is especially uncommon in engineering education research (EER) is the practice in which researchers who were not part of the original project team work with qualitative data they did not help collect and potentially even apply theories and analytic lenses that were not part of the original research plan. Because these new researchers have different relationships with the data, ensuring quality in this form of SDA requires a relational approach to conducting research that links those we term data originators and those we term secondary analysts. Toward this end, we have recently completed a project in which we sought to engage the EER community on the potential affordances of SDA in qualitative research, to understand the reasons why this approach remains relatively rare in our field, and to use our experiences with SDA to propose practices and principles that can inform this approach moving forward.1 One outcome of that project is the relational approach detailed in this editorial.
We advocate for SDA because the advantages of shared approaches to leveraging existing datasets can include (1) reduced time to publication (particularly for graduate students), (2) reduced load on participants (particularly those from populations marginalized in engineering, who may receive numerous requests to participate in research studies), (3) maximized use of data collected with public funds, and (4) greater equity in the field through data transparency. In spite of these advantages, challenging questions remain—most notably, how to legally and ethically conduct SDA without sacrificing quality or harming participants, and how to conduct robust high-quality analyses when data were collected for another purpose. Although challenges remain, we argue that it is possible to make advances on these questions. Thus, here we offer practical guidance for engineering education researchers conducting SDA, as well as address some frequently asked questions about SDA that can arise.
Guidance for Engineering Education Researchers Conducting SDA
In response to calls for data sharing and open access, in 2016 the scientific community released the FAIR (Findability, Accessibility, Interoperability, and Reuse of digital assets) principles for managing scientific data (Wilkinson et al., 2016), which focus primarily on ensuring that (typically quantitative) data are formatted and published for others to use. However, these principles offer little attention to considering individual participants or the ethical implications of secondary analysis. Thus, these principles have generated significant further discussion amongst researchers over the rights of participants and communities with respect to their data (even after consent is provided). Such concerns have been particularly important to historically marginalized communities, where data misuse has a long history. One response emerging from the Research Data Alliance is the “CARE Principles for Indigenous Data Governance” (Carroll et al., 2020). The CARE principles emphasize Collective Benefit, Authority to control, Responsibility, and Ethics as pathways for using Indigenous data and knowledge for collective benefit, centering research participants with whom data are generated in discussions of data sharing. The framework authors note, moreover, that the CARE principles extend beyond Indigenous communities to many populations or communities “wanting to maintain high levels of trust and accountability in the use of data about their communities” (Carroll et al., 2020, p. 8). We would extend their argument further to suggest that researchers should attend to these issues for all study participants.
In exploring qualitative secondary data analysis, we build upon the FAIR and CARE principles (see Figure 1) by making explicit the roles of researchers. To that end, we propose the SHARE principles for qualitative data sharing and secondary analysis:
Stewarding collaborative relationships
Honoring context of data
Aligning questions and data
Responsibly reusing data
Expanding capacity and ownership
Figure 1
Principles for Data Sharing.
Our framework, shown in Figure 1, also encompasses the notion of data as being “made” and “handled” in interpretive qualitative research which has implications for how we think about quality and SDA (Walther et al., 2016).
In Table 1, we provide explanations of each of the SHARE principles as well as reflection questions for those seeking to share data and those seeking to analyze shared data to consider collaboratively. Importantly, when navigating SDA, it may be essential to persist, question, and communicate with collaborators to proceed positively through any perceived or real misalignment between original and secondary research purposes and goals.
Table 1
SHARE Principles for guiding data sharing and secondary data analysis (SDA).
| PRINCIPLE | EXPLANATION | REFLECTION QUESTIONS |
|---|---|---|
| Stewarding collaborative relationships | The deeply contextualized nature of qualitative data often requires intentional stewardship to facilitate collaborations between researchers who generated the data and those seeking to use it to ensure that the data are used in ways consistent with the participants’ consent and understanding and the larger context in which the data were collected. This consistency typically requires significant dialogue between the original and new researchers about the contexts in which the data were collected, expectations and practices related to confidentiality or anonymity, and the epistemologies and positionalities of all collaborators involved in the work. Where feasible, data originators should serve as collaborators on projects and co-authors on publications. |
|
| Honoring context of data | Data originators and secondary analysts need to consider where, how, when, and by whom data were collected while maintaining necessary protection of participants’ confidentiality and terms of consent. This may include maintaining confidentiality or anonymity or, if the original study was designed to publish identifying information, ensuring that the information is disclosed and used in ways that honor participants’ interests and intents. |
|
| Aligning questions, frameworks, methods, and the data | As with all research studies, research questions should be appropriate to the existing data set, including the framework that guided collection, the data collection methods, and existing analysis. SDA can be approached collaboratively where a researcher with specific questions works with a data originator to determine whether a dataset is amenable to the new questions and approaches, or it can be approached inductively where secondary analyzers review samples of the data in search of potential research questions that could be answered, considering gaps in existing literature. |
|
| Responsibly reusing data | Ethics and trust are critical to any data sharing and analysis project. It is imperative to conduct research that has the potential to benefit the original participants or the population they represent. In sharing data, it is also important to develop a trusting relationship between the data originator and secondary analyzer that acknowledges the vulnerability involved with sharing a data set. |
|
| Expanding capacity and ownership | Sharing data can fulfill the need to acknowledge diverse approaches to capability development and build capacity of the research community by bringing new researchers into the process without requiring them to collect their own data. SDA can also broaden ownership of data so that others can shepherd it as well. The mutuality of sharing the data through SDA can help data originators and secondary data analysts experience the data in meaningful new ways. |
|
While these principles can help guide researcher interactions, questions and considerations remain regarding the legitimacy, practicality, and ethics of SDA. We consider some key issues in the following section, sorted into categories and formatted as frequently asked questions (FAQs).
Frequently Asked Questions about SDA
Bringing Legitimacy to SDA
Do you need to collect your own data for a paper to be publishable?
Because SDA is not currently a common approach in our field, reviewers, authors, and readers might wonder whether an article is publishable if it is based on a study where the authors did not collect the data themselves. Research quality and alignment between the various parts of a manuscript are key to publication criteria, regardless of who collected the data. Reviewers can sometimes be unduly harsh when providing feedback on research based on data that was collected for different purposes, so it is incumbent on authors to justify the purpose, objectives, and methods for an SDA, maintaining the tenets of stewardship and collaboration (S), how the context of the data was honored (H), and how the data aligns (A) with the new SDA. Though the authors were involved in the original project, a recent SDA by Deters et al. (2024) illustrates one approach to presenting such justifications. It will be important to find ways to promote resources for early career researchers to share their work while maintaining ownership and getting credit for being stewards of their shared data. Avenues to promote these practices for professional development of early career researchers can be a focus in moving SDA forward in the engineering education community.
Ethics as Related Specifically to SDA and Primary Study Participants
Can you do SDA with data collected from minoritized populations?
As discussions around data sovereignty are increasingly common around the globe, conducting SDA with data from minoritized populations raises a number of additional questions, in large part because of a long history of abuse in which such data have been unethically re-used without participants’ consent and/or for purposes that offer no benefit and may harm the original participants. While at the time this editorial was written the US does not yet have broad data protection or sovereignty regulations, the European Union passed the General Data Protection Regulation (GDPR) in 2018 (Wolford, 2018), which includes the principle of purpose limitation, or the idea that data need to be used only for purposes explicitly approved by the individual from whom it was gathered. The US Institutional Review Board (IRB) practices often rely upon analyzing wording in consent forms to determine the legality of additional analyses, but these analyses do not always consider the principle of purpose limitation. In a pilot SDA study we did with interviews collected from Indigenous engineering professionals, the Institutional Review Board overseeing the original data collection required the research team to send each original participant an informational email including a description of the expanded research team, details of the intended secondary data analysis, a copy of their prior interview transcript, and a request for a reply granting or declining permission. Most participants replied affirmatively, with several not replying (in which case we did not use their interviews). One participant requested additional details be redacted from their interview prior to SDA, and another participant declined because they wanted to be de-anonymized so that they could get credit for their ideas. Collectively, we believe that this approach centered on ethical considerations and made the least imposition on participants while giving them a measure of control over how their data were used. This experience further shaped our articulation of the principles regarding stewarding relationships (S) and honoring the context of the data (H).
Practicality of Engaging in SDA
Do repositories already exist? How do they work?
A number of publicly accessible qualitative data repositories exist, such as the American Education Research Association (AERA) and Inter-university Consortium for Political and Social Research (ICPSR) Partnership for Expanding Education Research in STEM (PEERS) Data Hub (AERA & ICPSR, 2024), and the Qualitative Data Repository (QDR) (Elman, 2024). There are also resources within Data Access and Research Transparency (DA-RT), a website and repository with guidelines for data access and sharing of data in a way that protects human subjects (American Political Science Association, 2016). Some funding agencies now require researchers to upload de-identified data to these repositories. Some repositories can combine multiple datasets. There are significant challenges associated with accomplishing sufficient de-identification of data that public sharing will not compromise participant identification, and at the same time this level of de-identification often obscures so many important contextual details that it might radically reduce the quality and usefulness of the data (Juros, 2022). Importantly, guidelines and examples exist for how to handle these issues. For example, the field of political science has grappled with the issues and challenges with qualitative data sharing, documented as The Qualitative Transparency Deliberations (Jacobs et al., 2021). There are also issues related to human subject research more broadly, for example, protecting the identity of participants, that result in tension between getting research protocols approved by Institutional Review Boards (IRB) and efforts at data sharing. However, such tensions extend beyond the mechanics of data sharing and require creative, intellectually robust solutions. Given these challenges, our view is that an approach that involves the original researchers (who collected the data) collaborating with the new team (doing SDA on the dataset) might be a better fit to the needs of our field. As highlighted in all of the principles of SHARE, there are ways to overcome these challenges with careful attention and there is the need and responsibility to do so.
Practicality of Engaging in SDA
Can you bring new research questions (RQs) to a dataset collected for a different purpose?
Thinking about SDA really forces the field to think about the fundamental questions about what empirical research is, and how theory, methodology and data can and should be aligned to arrive at research findings (the A in SHARE). Putting aside issues of data ownership and its relation to the participants in a study, there is no essential reason why a study can’t start with a dataset and then work backwards to think what questions might be posed to this dataset, and what theory will best align with the study and inform the analytical approach, particularly when researchers meaningfully consider the kinds of questions posed by the SHARE framework. While the content of the data and the context in which it was collected are obvious constraints on what questions can or can’t be addressed, the richness of many qualitative data sets often allows for a wide range of emergent questions to be explored through a variety of lenses. These considerations then also apply to the possibility of combining datasets to examine patterns across contexts and further illuminate the transferability of findings. Clearly, such combining has to be done intentionally, and with careful examination of the compatibility of the datasets. Achieving compatibility, for example, might involve using subsets of datasets that align with subsets of other datasets to support meaningful comparison while acknowledging the limitations results from the selection process.
Project Planning for SDA
Can you design a project with SDA as a goal?
You can design a project with SDA in mind, but it is not a necessary requirement. Doing so resolves process questions in advance and allows a team to build the structures to support data sharing (and the SHARE principles could serve as a guide) but does not eliminate the need for processes. For example, one could build into the study design in a process whereby people submit proposals for using a data set. This would put in motion a review process to determine if the researchers were actually asking questions that the data could answer and all of the information needed to make that decision could be part of the proposal. If intentionally designed for sharing, a research team could also build in the resources needed to clean the data and prepare it for sharing. A significant advantage to designing for data sharing in advance is the ability to address data sharing in practices for ethical research and human subjects research approvals before participants agree to engage with the project. Practices for closing out the project and data could also be pre-determined. Importantly, these practices do not mean that datasets can only be used for sharing if they were designed for such at the start. Instead, it means that going forward, researchers can, and perhaps should, consider such possibilities with new datasets but also allow for the practice of leveraging existing datasets in new ways.
What is Next?
We envision this editorial as a means to re-energize a conversation about advancing the field of engineering education research, building on earlier efforts such as the 2016 special issue about data share in Advances in Engineering Education (Johri et al., 2016). We hope it will encourage all members of the community, including both researchers and reviewers, to consider the opportunities that SDA affords and what these opportunities mean for the nature of research. We argue that SDA can, and should, play a critical role in expanding the EER community and that this expansion can apply internationally. Significant resources (time and/or money) are expended in developing datasets from the perspectives of those curating the data and those offering their experiences to researchers; maximizing the impact of that data through responsible reuse honors these efforts. SDA has the potential to expand the EER community by making data available to those who do not have the resources (time or financial support) to collect their own data. Collaborative SDA in particular can also provide professional development opportunities by providing a space for new researchers to partner with experienced researchers in mutually beneficial ways by actually handling data rather than just talking or reading about how to do so. Our pilot projects using SDA in EER provide concrete examples of these benefits (Paretti et al., 2023).
At the same time, we acknowledge that many questions remain including:
How do we truly value collaboration and the time to support SDA that this method entails?
How do we obtain funding for this work and the ongoing stewardship?
How do we develop the trust needed to be able to engage in this work?
We argue that there is no one “right way” to do this work. Rather, doing SDA well requires a more thoughtful and nuanced approach, and we look forward to seeing how the community adopts and adapts the SHARE principles to reshape the future of our field.
Importantly, we view the SHARE principles as the next step in an on-going conversation—that is, it is part of a dynamic exploration rather than a static framework. We hope that this editorial will inspire active change and further conversation and reflection. As researchers start engaging in collaborative SDA, collectively we will develop ways to overcome challenges in and bias against using such tools.
To that end, we see a number of different avenues for moving this conversation forward. For example, researchers could incorporate the potential for SDA into new studies and proactively advertise availability of their data for secondary use. When presenting original projects at conferences, researchers could state that the data is available for future reuse and invite potential researchers to come and chat after the presentation. In written publications, authors might add a section to indicate whether the data are available for additional analysis and include an invitation to contact the authors if interested. Within our departments, we could solicit colleagues to create a departmental repository of data to use with research methods classes, or even doctoral dissertations. Also, journal editors can call specifically for papers that rely on SDA and make it a requirement for authors to indicate if the data are available for SDA. By normalizing the use of SDA, we can all push the conversation forward into action.
Notes
[1] An overview of the project and its outcomes can be found on this website: https://enge.vt.edu/researchfacilties/secondary-data-analysis-research.html.
Acknowledgements
We would like to thank everyone who has contributed to this project, including Joachim Walther and Nicola Sochacka, who helped develop the grant, all workshop participants, and Hope House, Tiffany Cunningham, and Lucinda Shewchuk, who provided logistical support. We also thank the participants who were involved in the two projects that were used for SDA.
Funding Information
This material is based on work supported by the National Science Foundation under Award No. 2039864. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Competing Interests
The authors have no competing interests to declare.