Expanding participation with an audio member checking approach

Much of the theorizing and data analysis in SD and in other fields is often left to researchers (Collins 2022). While research teams may include PWLE in the creation of the interview questions or in the leading of an interview, few methods exist that include PWLE in the thematic analysis and synthesis of the collected data. Methods like member checking through line by line transcript analysis while thorough aren’t always accessible for research participants. Member Checking is a qualitative research approach that was first defined in 1985 by Lincoln and Guba as the testing of “data, analytic categories, interpretations, and conclusions” with members of the stakeholder group(s) from whom the original information was collected. It is considered one of the most significant methods within qualitative research for establishing or strengthening the credibility of a study” (Doyle 2007). A reason for conducting Member Checking is that it “acknowledges that participants hold valuable knowledge and can be integral to the analysis and interpretation of research findings” (López-Zerón 2021).

At Moxie, we created an audio member-checking approach to include research participants in an aspect of data analysis. This audio member checking approach expands participation by engaging people’s knowledge and allowing people who participated in interviews (or other qualitative methods) to validate the messages coming out of the research by responding to anonymized and aggregated audio clips from the research. The following example show the various ways people are engaged in their knowledge and in the process in which people can respond to the data being shared.

Caregiver member checking sessions

In a project for a healthcare client, Moxie focused on understanding caregivers' experiences supporting a relative or friend so that the client could align their value proposition (the value they provide with their service) to caregiver needs and challenges. As many readers may know first hand or second hand, “caregiving is a silent service that often goes unpaid and underappreciated.” To respond to this challenge, the client “offered programs and services that provide caregivers with resources and support to ensure their ongoing ability to provide care” (Collins & Dales).

After data collection through in-home interviews and day-in-the-life shadowing, we used our audio member checking method to validate and engage caregivers in the themes that had emerged from the coding. An audio member checking approach works as follows: after an interview, designers code each of the interviews into categories and themes using a grounded theory approach using an audio transcription and coding software called Dovetail (Dovetail). To share the emerging categories and related data, short segments of audio clips from interviews are compiled into thematic clips. Research participants listen to and discuss the emerging categories from these audio clips during group member checking workshops.

In two separate sessions, the caregivers who had been interviewed participated in listening and responding to the aggregated audio clips. One benefit of this approach, as one team member (Molly) noted, was that it created a space for Caregivers to share their experiences on their terms without the expectation of having to re-hash their stories for the sake of the research process. This was important as one risk to participating in research processes is reliving negative experiences. While research participants are paid, there is at times an emotional toll to participating in the research.

Our team saw how using audio member checking could create reciprocity in the research process by creating methodologies that focused on reciprocity. When a SD process integrates values such as reciprocity and care into participation it can provide further value to participants as it asserts respect and agency for the participants and as a result increased participant ability to take their own action.

One goal in our research approach is to create some type of benefit for the people who participate in the process, regardless of the outcomes of the overall client collaboration. In this case a follow up support group was an action that came out of the research process. While this was not the intention of our session, nor part of the research plan, it is an example of participants having ownership over their stories and experiencing motivation to act. At the end of the session, participants shared reflections that made the session feel more like a support group than a research setting. For example, one motivation we observed was the need to compare experiences. In this case, participants would ask questions of each other to see if others had similar experiences. The following quotes speak to moments in the workshop where participants centered their interests in learning from each other rather than being solely guided by our research team’s facilitation agenda. Our agenda was to host a session to check the validity of the data collected and themes that were emerging. Yet, participants had their own goals as well such as connecting with other caregivers and comparing experiences.

A moment where a caregiver asked the group if they also have pushback from the person that they are caring for:
Caregiver 2: “This is a question. I don't know if I should be asking but do any of you get like the, the person that you take care of? Like if you don't wanna do this [they say] just don't do it.”

Caregiver 1: “I've had that conversation, but not with the threat of finding somebody. It's just more just don't do it and, and he gets angry and, and he'll, he'll shut down, we will discuss, he doesn't wanna talk.”

Another moment when a caregiver asked if strangers ever try to help them when in public:

Caregiver 3: But I do, do anyone ever when they're taking their persons outside or out in public, do other people like jump in and start trying to help and assist as, as if you're not doing anything, right? Cause I get that a lot.”

A reflection at the end of the session:
Caregiver 2: “I didn't realize how much caregivers have in common, you know, I thought I was alone”

Similarly to support groups, caregivers expressed gratitude and empathy toward one another for sharing and connecting about their experiences.

“I just wanna say thank you to everyone, you know, for talking and you know, my hearts are with you and I know exactly how you guys are feeling and I empathize dearly with all of you and, and thank you for sharing with me.” - Caregiver in Member Checking Session 1

This enhanced the process as it integrated reciprocity into the process between the research team and caregiver participants. Caregivers who found the session useful asked to create a follow up support group.

“I am hugely good with resources to the degree my husband's try trying to get me to start a youtube channel. I, I don't know if I can do this or not. I'd like to put my phone number out there and if any of you guys need help with things, finding things, getting resources.” - Caregiver Session 1

“I'll be reaching out via text. Would y'all, would y'all want to do like a monthly or weekly zoom call where we just sit and just hang out with like a beverage of choice and just kind of blow off some steam?” - Caregiver Session 1