Transparency in Qualitative Research?
Until recently, efforts towards increasing research transparency within the CHI community have primarily focused on quantitative, hypothesis-driven research. Transparency, which refers to “honesty about the research process”, is also key for assessing the rigor of qualitative studies.
In an effort to bring transparency to the review of qualitative research, a group of qualitative researchers crafted a set of guidelines for a successful qualitative research submission for CHI2020. As a result, there was an outpouring of responses on social media (Twitter and Facebook) from researchers in the community. While the effort was appreciated and there was concurrence on the need for such guidelines, many valid concerns were also raised, and the responses indicated that researchers passionately care about this topic. The guidelines were taken down as they did not reflect the views of the different researchers in our community. The purpose of this panel is to spark discussions surrounding transparency and contribute to the refinement of the guidelines through an engaging debate.
Qualitative researchers with diverse backgrounds and stances will discuss some of the key concerns with achieving transparency in qualitative research, share examples from their experiences, and present views on if (and how) these concerns can be addressed.
Panelists’ Positioning Preview
Core Issues to Discuss
- Defining Transparency. What does transparency entail in the context of qualitative research? Even though the meaning and assessment of rigor depend on the qualitative study in question, can we define the criteria for assessing rigor and transparency for qualitative research more broadly? Given the delineation of transparency into process transparency and data sharing, what are their respective benefits and which aspect is more instrumental in the review process?
- Protecting Participant Identities. The impracticability of fully anonymizing collected data is a major concern with data sharing and this can be especially injurious when studies involve sensitive data and participants from marginalized or vulnerable populations. How can we assess such risks involved in
a study so as to set more appropriate standards for transparent communication based on the risks involved? Furthermore, even if prior permissions are sought for data sharing, participants may not be very inclined to share information if they think the data will be openly shared . Would communicating the benefits of data sharing to the participants and clarifying that they are the potential beneficiaries of the studies (where they are) help in mitigating this challenge?
- Limited Overlap in Researchers’ Experiences. Qualitative research in HCI encompasses a myriad of approaches with various theoretical, empirical, and epistemological underpinnings. Researchers often have experiences with a subset of these approaches and may be assigned papers outlining approaches they are not familiar with. In such cases, how can transparency help in communicating the distinctiveness of the research more clearly to the reviewers?
- Supplementary Materials. Even when authors endeavor to elaborate on their research processes, publication page limits can hinder the inclusion of such details. While additional details can be included in the supplementary materials and other venues, the paper might still be the most perused document and viewed as the main artifact of the submission. How can we help authors decide what details to include in the paper and in the supplementary material? How can we encourage weighty consideration of supplementary materials in the review process?