YSPH Biostatistics Virtual Seminar: "Telehealth Use to Support Management of Anxiety and Depression Among African American Women"

Cindy (00:00):
Thank you, everyone, for coming to this presentation, Dr. McCall is currently a postdoc at Yale. She got her MBA in 2010. No, wait, her MPH in 2010, her MBA in 2016, and her PhD in health informatics in 2019? '20.

Terika McCall (00:21):
'20.

Cindy (00:23):
'20, sorry. At the UNC. And she's been here at Yale for the past year. We recruited her to be a post-doctoral fellow [inaudible 00:00:36], and we're doing a targeted recruitment. She's hopefully coming to bring needed health informatics focus that no one in our department has had before on usability and user-centered design. And she's already done a lot of work in this area. And she'll talk about her research. And welcome, Terika.

Terika McCall (00:59):
Thank you for that introduction, Cindy. And so, I'm going to go ahead and share my screen now. Okay. Let me minimize this. Here we go. All right. So, good afternoon, everyone. Today, I will present about my research, which focuses on... And I just want to clarify, can you see my screen before I get started? If someone can confirm.

Cindy (01:27):
Yes, yes.

Terika McCall (01:28):
Beautiful. Okay. Today I'll present my research, which focuses on the use of telehealth modalities to support management of anxiety and depression among African-American women. Okay. So, the topics covered in this talk explore the barriers to use of mental health services among African-American women, modalities they consider acceptable to receive support for managing anxiety and depression, user-centered recommendations for content and features that should be included in a smartphone app culturally tailored to support management of anxiety and depression in African-American women, and results of a usability testing of an initial prototype of such app. I will conclude by briefly discussing my current projects and future work.

Terika McCall (02:20):
So, let's start with a little background. In 2019, it's estimated that one in four Black women in the US experienced mental illness in the past year. This is a significant increase from previous years. In 2018, it was one in five. Anxiety and mood disorders, such as major depressive disorder, are among the most common mental health conditions within this population. And approximately 16% of non-Hispanic Black women reported having generalized anxiety in their lifetime, and around 27% reported experiencing depression in their lifetime. Historically, mental illness has been under-reported in the Black community, therefore the true burden may be significantly higher than reported prevalence estimates. Although there is a great need for mental health care, Black women significantly under-utilize treatment, with only about 13% receiving mental health care in 2019. Furthermore, approximately 30% of Black women who reported experiencing mental illness in that past year did not receive any treatment during that time. Now, keep in mind that these numbers are pre-COVID, and so it's expected that they have only gotten worse.

Terika McCall (03:37):
So, numerous studies have found that tele-health interventions increased access to services and resources and are effective in helping participants reduce anxiety and depressive symptoms. However, the majority of published studies were conducted with predominantly White participants, which may affect the generalizability of the results to other racial and ethnic groups. Previous studies have shown that African-American women are comfortable with participating in mHealth research and interventions, and 80% own smartphones. So, there is definitely great potential to remedy the disparities in mental health service utilization by leveraging use of smartphones for information dissemination and delivery of mental health services and culturally informed resources. Given that the current mental health services are under-utilized by this population, this community should be surveyed to determine the acceptability of using mobile technology for this purpose.

Terika McCall (04:37):
So, this brings me to discussing the attitudes towards seeking mental health services and use of Mobile Technology Survey and my first research questions, which are what are the barriers to use of mental health services among African-American women? Is the use of mobile technology acceptable to African-American women in terms of delivering mental health services and resources to support management of anxiety and depression? Also, are some modalities more acceptable than others? And does that vary by age or education level?

Terika McCall (05:11):
I will now discuss the survey methods. So, a self-administered web-based survey was launched in October 2019 and closed January 2020. Women 18 years or older who identified as Black, African-American, or multi-racial, Black/African-American and another race, and reside within the United States were eligible to participate in the survey. Participants were recruited [inaudible 00:05:37] sampling, and recruitment methods included receiving an invitation to take the survey via an anonymous link distributed through listserv, whose membership is primarily African-American women. So, for example, the National Council of Negro Women, university student group listservs, and church membership listserv. Participants were also recruited via post on social media, for example, Twitter, and following a snowball sampling technique, respondents were allowed to share the link to the survey with their networks.

Terika McCall (06:12):
So, the attitude towards seeking mental health services and use of Mobile Technology Survey consisted of 80 questions. The survey domains included questions from validated instruments to measure attitudes towards seeking professional psychological help, using an adapted version of an inventory of attitudes towards seeking mental health services. It also included questions that screen for presence and severity of anxiety and depression using the generalized anxiety disorder, GAD-7, scale, and the patient health questionnaire PHQ-9, respectively. Additionally, questions were included to explore the acceptability of using a mobile phone, specifically voice call, video call, mobile app, or text messaging to communicate with a professional to receive support for managing anxiety and depression. Descriptive statistics were calculated as means, standard deviations, and ranges for our continuous variables and as frequencies and percentages for categorical variables for sample characteristics and responses to the questions about the use of tele-health modalities to receive mental health services.

Terika McCall (07:24):
Fisher's exact test was used to determine whether an association exists between the response to the questions about comfortability with using each telehealth modality to communicate with a professional to receive help for managing anxiety, and age group and education level, respectively. This was similarly done for depression. Additionally, independent sample T tests were performed separately to assess group differences in mean scores for the PHQ-9, GAD-7, as well as psychological openness, help-seeking propensity, a difference to anxiety stigma, indifference to depression stigma, and inventory total scores between the groups of participants who agreed with the various telehealth modalities to communicate with a professional to receive help for managing anxiety and depression, versus those who disagreed. In addition, thematic analysis was conducted on responses to the questions about concerns when using the modalities to receive mental health treatment or counseling.

Terika McCall (08:29):
So, now, I'll dive into the results by first discussing the characteristics of the respondents. 395 respondents completed the survey. Respondents' mean age was 44.8 years old, and all identified as either Black, African-American, or multi-racial. Most respondents identified as non-Hispanic. And furthermore, 79% had obtained a Bachelor's degree or higher, and annual household income was reported as less than 50,000 for about 40% of respondents. Approximately 94% also indicated that they had health insurance. Regarding their mobile phone use, approximately 98% of respondents reported use of text messaging, and around 73% indicated texting four or more times per day. The majority of respondents indicated that they use mobile apps, and 73% indicated using a mobile app four or more times per day. Most indicated that their phone had video call capability, and approximately 80% reported use of video call and 38% indicated using video call at least one time per week.

Terika McCall (09:43):
So, approximately 29% of respondents to this survey reported being diagnosed with anxiety in the past. Questions from the generalized anxiety disorder, the GAD-7, scale were asked to screen for current presence and severity of anxiety. The study findings show that younger African-American women were more likely to have higher levels of anxiety than older African-American women. The 25 to 34 year old age group, followed by the 18 to 24 year old age group, had the highest percentage of individuals with moderate or severe anxiety. The results revealed a trend of anxiety severity gradually decreasing among the older age groups.

Terika McCall (10:27):
Similarly, about 29.6% of respondents reported being diagnosed with depression in the past. And questions from the PHQ-9 were asked to screen for current presence and severity of depression. Study results revealed that younger African-American women were more likely to have higher levels of depression than their older counterparts. Once again, the 25 to 34 year old age group, followed by the 18 to 24 year old age group, have the highest percentages of individuals with moderate, moderately severe, or severe depression. The results showed a trend again of depression severity gradually decreasing among the older age groups.

Terika McCall (11:11):
So, approximately 40% of the respondents answered yes to the question during the past 12 months, was there any time when you needed mental health treatment or counseling for yourself but didn't get it? The survey results show that the most common reasons African-American women do not seek mental health treatment or counseling was needed basically across not knowing where to access service, lack of time, and stigma from others, knowing that they are utilizing mental health services, were all the most common reasons they gave. The use of tele-health modalities to deliver mental health services may help to diminish some of these barriers by providing information about how to access affordable services and offer a discreet way to receive care in a preferred setting such as the privacy of their home.

Terika McCall (12:00):
So, for the comparison of modality acceptance, response options were on a five point Likert-type scale, ranging from disagree to agree. The graph to the left shows that approximately 73% of survey respondents indicated agreement, which means they selected agree or somewhat agree as their response regarding the use of a video call... I'm sorry, of a voice call, to communicate with a professional perceive help to manage anxiety. And furthermore, about 66% of respondents agreed with the use of a video call, approximately 48% agreed with the use of a mobile app, and 46% of respondents agreed with the use of text messaging. Moreover, the graph to the right shows that approximately 72% of respondents agreed that having the option to use a voice call to communicate with a professional, if they are dealing with anxiety, would be helpful. Furthermore, about 68% of respondents agreed with the use of a video call, approximately 49% agreed with the use of a mobile app, and 49.6% of respondents agreed that use of text messaging would actually be helpful.

Terika McCall (13:15):
So, in the table, response options were dichotomized as agree, indicating responsive agree or somewhat agree, or disagree, indicating a response of disagree or somewhat disagree. When we take a look at the accessibility of the different modalities by age group, the results show that younger African-American women, those that were less than 50 years old, were significantly more likely to endorse the use of text messaging, mobile app, and video call to communicate with a professional to receive help for managing anxiety, compared to older African-American women.

Terika McCall (13:54):
No statistically significant associations were found between education level, having a Bachelor's degree or higher, or less than a Bachelor's degree, and agreement with the use of text messaging, voice call, mobile app, or video call to communicate with a professional to receive help for managing anxiety. I just want to highlight that the undecided responses were initially excluded from analysis. However, sensitivity analysis was performed using Fisher's exact test. Undecided response were combined with disagree and somewhat disagree response to form a category for those who did not indicate agreement. And this was done in order to determine whether the statistical significance change when undecided responses are included. All sensitivity analysis results were supportive of the main findings.

Terika McCall (14:45):
So, now, when we look at modality acceptance for support with managing depression, the graph to the left shows that approximately 70% of respondents indicated agreement regarding the use of voice call to communicate with a professional to receive help for managing depression. Around 64% of respondents agreed with the use of a video call. Approximately 45% agreed with the use of a mobile app. And similarly, text messaging... Sorry. Text messaging was also 45%, 45% agree with text messaging as an acceptable mode of communication. So, the graph to the right shows that approximately 71% of respondents agree that having the option to use a voice call to communicate with a professional, if they are dealing with depression, would be helpful. Furthermore, around 68% of the respondents agreed with the use of a video call, approximately 49% agreed with the use of text messaging, and around 47% of respondents agreed that the use of a mobile app would be helpful.

Terika McCall (15:51):
So, looking at the accessibility of the different modalities by age group, the results of the study showed that younger Black women, less than 50 years old, were significantly more likely to endorse the use of text messaging, voice call, a mobile app, or video call to communicate with the professional to receive help for managing depression. No statistically significant associations were found between education level and agreement with the use of voice call, mobile app, or video call to communicate with a professional to receive help for managing depression.

Terika McCall (16:25):
However, there was one significant finding that emerged in regards to education. Respondents who reported less than a Bachelor's degree were more likely to endorse the use of text messaging to communicate with a professional to receive help for managing depression than respondents with a Bachelor's degree or higher. So, this finding may be largely due to age, especially since 18 to 24 year olds are the adult age group that uses text messaging the most. All sensitivity analysis results were supportive, except for the case where statistically significant associations were found between education and agreement with the use of mobile app to communicate with a professional to receive help for managing depression. So, those with less than a Bachelor's degree were more likely to agree with the use of a mobile app.

Terika McCall (17:16):
So, now, I'm going to move on to discuss the focus groups which were conducted in part to answer the research question, what content and features should be included in a smartphone app tailored to support management of anxiety and depression in African-American women? So, the focus group sessions were held at a Durham County library and at the University of North Carolina at Chapel Hill in January 2020. Women 18 years or older who identified as Black, African-American, or multi-racial were to participate. Participants were recruited via a web-based survey that was launched prior to the focus groups, the survey I just spoke of, as well as postings on social media sites including Facebook and Twitter. Flyers were also posted in the community in public locations.

Terika McCall (18:09):
So, the topics of the focus group's discussions included past and current causes of anxiety and depression and coping skills used, attitudes and perceptions towards mental illness and receiving mental health treatment, contents and features needed in a smartphone app designed to help African-American women manage anxiety and depression, and barriers and facilitators to use of a smartphone app for mental health care. So, the sessions were audio recorded and transcribed verbatim. And [Vivo 00:18:41] 12 software was used to conduct thematic analysis, and focus group transcripts were analyzed to rebuild themes related to the preferences and concerns when using a mental health app. Reoccurring themes across the focus groups were also noted.

Terika McCall (18:58):
Now, I'll dive into the results by first discussing the characteristics of the focus group participants. So, 20 women attended one of the four focus group sessions, and each group consisted of five participants. All participants identified as Black, African American, or multi-racial. And the study participants ranged in age from 21 to 79 years old, with a mean age of 36.6 years old. Also, 75% of the participants were less than 50 years old. So, the focus group participants shared that in the past, they primarily use mental health and wellness apps that had features for meditation, mood tracking, calorie intake and activity monitoring, and deep breathing exercises. In addition, the use of music apps and listening to podcasts were popular. Inspirational messages on social media apps were also mentioned as being beneficial to mental wellness.

Terika McCall (19:57):
So, the recommendations from the focus group participants focus on the type of content and features that should be included in the app, as well as suggestions to increase app usage and establish trust with users. So, here's a snapshot of the user-centered recommendations, and I'll now take a few minutes to discuss them in more detail and share a few quotes from participants. So, many suggestions were given on the content that should be included in an app designed to help African-American women manage anxiety and depression. Recommendations were either informational or inspirational. For example, participants stated they would like information about how to find a Black woman therapist in their area, guidance on how to deal with common stressors such as microaggressions and imposter syndrome, and information about events in the area to connect with other Black women. Inspirational messages and encouraging stories about how others overcame adversity were also desired. The participants also recommended having positive and supportive messages in the app and suggested readings that promote mental wellness.

Terika McCall (21:06):
So, regarding information to find a therapist, a participant voiced, she likes to be able to, "Find Black women therapists in the area," but also listing what insurances they take, their hours, all that good stuff. Because, again, those barriers. People are just like, "Where are these people?" So, the recommended app features now, basically they allow users to monitor their progress, practice coping techniques, and connect with others. So, for example, participants suggested that the app have features to tracking anxiety, depression, and mood. In addition, they recommended guided meditation, deep breathing, and other coping techniques narrated by a Black woman.

Terika McCall (21:48):
Participants stated that most meditation apps use a British or Australian voice to narrate. However, they desire to hear what they described as a Black [bunty 00:21:57] voice. In other words, the caring voice of a middle-aged African-American woman. Lastly, they recommended having group chat rooms to connect with other users and the ability to connect with therapists through the app via messaging or video call. So, for example, regarding a feature to track anxiety, a participant voice she would like, "If you can have a history in there and then you can look back and see when was the last time that you had this anxiety and what helped when you have this anxiety attack or whatever. You don't just end it with what you see. You can write down what helped and then you can go back to that."

Terika McCall (22:36):
Participants voice that they would be more likely to use the app regularly if they found value in it and the app was easy to use. So, specifically participants emphasize the importance of feeling a sense of community when using the app, being able to connect with a therapist, and managing anxiety and depression through learning coping skills. The user interface design should also be clean and the app should be intuitive and easy to use. They also recommended having an accountability feature such as a daily mood check-in. Gamification of the app was also suggested to give the users a sense of accomplishment and make the app more sticking. Participants voiced that if an app is too cumbersome, there are too many notifications, or if the exercise take too long to complete, it would discourage use of the app. Overall, the focus group participants were excited about the idea of an app tailored to help them manage anxiety and depression.

Terika McCall (23:31):
So, for example, regarding providing some type of incentive to use the app, one participants stated, "Although it would be a superficial incentive, I feel just like the apps that keep track or give you stars or anything that I guess that you've completed something in itself can be validating to people, especially good habit building the stuff like that." So, most of the participants' concerns now were about security and privacy. There was concern that the app could be hacked and their data disclosed. Furthermore, participants had apprehension about who would own the app and the data sharing policies. There have been many data breaches in the news which concerned participants, and they were also aware that many apps sell user data. The primary concern was that the data would be used to harm them personally or Black women in general.

Terika McCall (24:23):
For example, regarding data sharing and use, a participant voiced her concerns stating, "I was watching the news on the treadmill last night, two nights ago, and they were talking about how these big companies on these dating apps are selling data, and so I would be worried about who has control of the data that's happening in the app. And what are they going to do with it now that they have all this information about Black women?" So, there was a general concern that the community in itself could be harmed with someone having this information and using it for the wrong purposes.

Terika McCall (24:57):
So, following the focus groups, I held usability testing sessions to evaluate the initial prototype of the app, and many of the recommendations from the focus groups were included in the initial prototype used for usability testing. So, I worked with a team of undergraduate computer science students at UNC on development of the prototype, and here's displayed the screenshot of the app home screen. The app contained features such as a guided thought journal, information about anxiety and depression, including facts about the prevalence of anxiety and depression among African-American women, self-assessments for anxiety and depression using the PHQ-9 and GAD-7. It also had a mood rating feature and graphs to track trends in depression and anxiety severity. It also had resources, too, that linked them to the Therapy for Black Girls therapist directory and podcasts, and a self care planner. Additional recommendations from the focus groups will be included in the next version of the app.

Terika McCall (26:00):
So, individual usability testing sessions were held for each participant at UNC School of Nursing bio behavioral lab. Women 18 years or older who identified as Black, African-American, or a multi-racial were eligible to participate. Participants were recruited via posts on social media, as well as a recruitment listing on Research for Me at UNC. Flyers were posted in the community at local coffee shops and at the county library.

Terika McCall (26:33):
So, the Cognitive Walkthrough Method was used to evaluate the user interface on how well it supported users in learning to complete tasks. Specifically, this method was used to evaluate the ease with which users can perform a task with little to no formal instruction or informal coaching. Participants were read the persona and scenario and then asked to begin the four tasks. They were also told to speak aloud their thoughts and actions so that it could be recorded by the Tobii software. So, in the background, the Tobii software recorded video of the participants completed the task, including taps on the phone screen, eye movements, and the amount of time spent on each task. The questionnaire for user interface satisfaction was self-administered after the usability testing was complete to collect data on system performance and the most positive and most negative aspects of the app.

Terika McCall (27:30):
So, in order to limit participant burden while testing most of the features of the app, there were two different personas and scenarios with four tasks each. Participants were either given scenario one or scenario two, and activities included finding out the recorded levels of anxiety or depression for the past six weeks, finding information on how to manage anxiety or how to overcome depression, adding a new entry into the guided thought journal, creating a plan for self-care, and locating a therapist to schedule an appointment. So, participants did not enter any personal information into the app. The instructions for each task included dummy information for the participants to enter to complete the task.

Terika McCall (28:18):
So, 15 participants tested the usability of the app using the Cognitive Walkthrough Think Aloud Method. Participants range in age from 20 to 66 years old, with a mean age of 29.8. And all identified as either Black, African-American, or multi-racial. Furthermore, approximately 87% had obtained a Bachelor's degree or higher, and the majority of participants indicated that they use mobile apps four or more times per day. So, there was 15 participants in total, and eight participants were given the tasks for scenario one. Most participants were able to fully complete each task with limited or no assistance. On average, participants took longer to complete the task compared to the benchmark times.

Terika McCall (29:08):
Adding a new entry to the journal proved to be the most cumbersome task, as it required participants to be thoughtful when they entered the tests and select their feelings from a list. Two participants had difficulty with locating the information on how to manage anxiety, searching in the resources section of the app instead of info. To locate a therapist, participants were required to tap a button that linked out to a website with a directory that basically consisted of Black women therapists. And although all participants were able to complete the task of locating a therapist, many reported that the interface of the website itself was not mobile-friendly and required a lot of scrolling to find out if the therapist was accepting new clients.

Terika McCall (29:54):
So, seven participants were given a task for scenario two. All participants were able to find out their levels of depression for the past six weeks and reported the most recent level of depression recorded in the graph. One participant had difficulty with locating the information on how to overcome depression. Once again, searching in the resources section of the app instead of in the information section. And participants also mentioned that they really did like the ability to track their self-care. So, time to complete a self care plan varied due to the typing speed of the participants. Some people were able to get it pretty quickly, whereas others, it took them a bit of time.

Terika McCall (30:37):
So, for the first scenario task, the results of the usability testing showed that participants were mostly satisfied with the user interface of the app. The average mean score for each domain range from 7.2 to 8.3, on a scale of zero, low satisfaction, to nine, high satisfaction. The ease of learning to use the app and the app capabilities received the highest scores of 8.0 and 8.3, respectively. For the second scenario task, the average mean score for each domain ranged from 7.5 to 8.8. Most of the scores were eight or higher, as you can see. The scores may have been higher for scenario two because it didn't include a journal entry which had more steps to complete than the others test.

Terika McCall (31:26):
So, the Tobii Pro 2 glasses were calibrated for each participant prior to beginning usability testing. And here's a video that basically it shows the Tobii Pro glasses recording the eye movements of an individual and what they focus on as they navigate the apps to complete tasks. This information is useful in understanding user's behavior. It can be used to inform design of the app to make it more intuitive by knowing where they are more likely to look.

Terika McCall (31:55):
As you can see from the heat map of the managing anxiety tips and journal entry screens, participants spent most of their time focused on the left and middle of the screens, with the spot of greater intensities, the red and orange areas, indicating areas of greater visual focus. Furthermore, some of the insights gleaned from participants during the cognitive walkthrough sessions were that important information should be placed on the left side or the middle of the screen. Now, this is consistent with standard recommendations for designing apps, and previous research shows that people spend 80% of their time looking at the left side of the screen.

Terika McCall (32:33):
So, overall, participants felt the app was easy to use, organized well, and had fast processing speed. General recommendations for improvement included adding more graphics and color, including instructions for data entry text boxes, a tutorial for first time, and crisis support monitoring. Regarding content, participants thought the information provided was of high quality. They also liked that the links that we provided were helpful, and they linked to outside resources such as a directory where they could find Black women therapists in their area. They also liked the recommendation that the graphs and the texts provided information to track their anxiety and depression severity. So, they liked that they were able to track their anxiety and depression severity over time because that was one of the comments they made in a focus group, is that they would want to know how they're doing and be able to monitor their progress.

Terika McCall (33:30):
So, participants recommended, though, that a curated list of Black women therapists be provided in app to avoid linking out to websites that aren't mobile friendly and require too much scrolling. Participants gave positive feedback on the usefulness of the anxiety and depression checkups, the self-care planner, and the guided thought journal. And the recommendations for improvement included adding a footnote to anxiety and depression history graphs to indicate that it can be used to track trends, and also adding instructions to journal entry features to inform users that they can actually use the talk to text feature for journal entries to reduce burden. Regarding navigation and error prevention, participants thought the icons were helpful for guidance on navigating the app, and they liked the menu dropdown. Clearly lists all the features. They also noted the great prompts in the guided thought journals. And participants recommended that the information feature be relabeled to make it clear that it contains mental health information, and that also the click-through sequencing be improved.

Terika McCall (34:38):
So, what I just went through, this mixed methods study, was extremely informative, and so I just want to talk a little bit about the limitations of it though. So, in regards to the survey, the main limitations were the mode use that administer the survey and the recruitment methods. The survey was only available online, and although this method may increase privacy and reduce respondent burden in completing the survey, women who did not have access to email or a social media account may not be able to complete the survey by accessing the link. In regards to recruitment, participants were recruited through convenient sampling and encouraged to share the survey, email, or social media post with their networks. And although no personally identifiable information was collected in the survey, and respondents actually access it through an anonymous link, social desirability bias could have resulted if the respondent personally knew anyone on the study team.

Terika McCall (35:39):
In addition, the sample consisted of mostly highly educated women with health insurance, so this limits my ability to broadly generalize the findings to all African-American women. However, previous literature has shown that African-American women with health insurance are still less likely to use mental health services compared to their White counterparts. A limitation of the focus groups now was that most of the participants were under 50 years old, so the sample skewed towards capturing the thoughts and opinions of younger African-American women. This may limit the generalizability of the findings to older African-American women. And also given the stigma of mental illness in the Black community, participants may have felt peer pressure to give socially desirable answers to the moderator's questions.

Terika McCall (36:28):
One of the main limitations, now, of the usability testing study is that due to the geographical restriction in recruiting participants, the results may not reflect the opinions and perceptions of a nationally representative sample. Also, most of the participants were under 50 years old so the sample skewed towards capturing the thoughts and opinions of younger Black women. In addition, personal preferences may also be rebuilding the feedback, which may not actually be helpful in improving the usability of the app. Another limitation was that the study did not focus on the efficacy of the app to reduce anxiety and depressive symptoms, as this would require a randomized control trial and significant resources to be done properly. However, despite these limitations, the proposed study design actually yielded useful data that will help researchers to understand the mental health needs of African-American women, the accessibility of using various tele-health modalities to provide support, and provide a framework for designing a smartphone app to help this population to manage anxiety and depression.

Terika McCall (37:37):
So, why is it important that I conducted this mixed method study? So, first, within a community that under-utilizes mental health services, it is important to understand the barriers that prevent them from receiving care and how telehealth modalities may be used to mitigate or eliminate them, incorporating their preferences for tele-health modalities to communicate with a professional to receive support they increased engagement, and improve treatment outcomes, whether it is the primary mode of care or used in adjunct to in-person services. Also, a one size fits all approach to designing mHealth interventions may lead to more options but continued disparity in receiving mental health care. To increase the likelihood of adoption and continued engagement, mHealth interventions should be tailored to address the community's needs and preferences. In the focus groups, we discussed what continent would be useful. This is an important step in the process of designing a digital health tool. Because you may find that what you feel would be most useful and what the end user feels is most useful are misaligned. This is how you ended up with a very nice product that misses the mark in providing value to the user.

Terika McCall (38:56):
Lastly, usability testing is critical. Poor usability contributes to low engagement and impacts the effectiveness of the intervention. The goal is to make the digital health tool as intuitive as possible. In conclusion, incorporating the recommendations from intended users and knowledge of their technology use behaviors may help to mitigate future intervention-generated inequalities. In this study, I focus on tele-health [inaudible 00:39:25] support management of anxiety and depression among African-American women. However, the methods that I applied in this study are necessary when creating a digital health tool for other communities as well.

Terika McCall (39:38):
So, I'll now briefly speak about my ongoing projects and future work. So, currently, I'm working with two interns this summer on redesigning the app based on the recommendations from the focus group and findings from usability testing. I'm also working with two African-American licensed clinical mental health counselors to develop the culturally-adapted cognitive behavioral therapy components for the app. Focus groups will be conducted to receive feedback on this upgraded prototype, and after incorporating feedback from the focus groups, I will conduct usability testing session on the new prototype. Results of the usability testing will then be used to further refine the prototype to improve its functionality and ensure that it is intuitive prior to the pilot study. I'm also wrapping up a project now for the Eastern Caribbean Health Outcomes Research Network cohort study, in which I train the fellows on how to conduct one-on-one usability testing of the new data sharing platform with community members from eight Caribbean islands.

Terika McCall (40:47):
So, the goal is to advance the dissemination of health data throughout the Caribbean through this publicly available tool. The findings from the usability testing are now being used to make the platform more intuitive and to increase user satisfaction. Also, we recently received notice of award for a diversity supplement for the personal health libraries for Formerly Incarcerated Individuals Project, whose PI is Dr. Karen Wang. The overall objective of the study is to develop and test strategies in partnership with formerly incarcerated individuals to increase acceptance and use of this PerHL mobile app. And since Black women in the US are incarcerated at nearly twice the rate of their White counterparts, my role on this study is to assess the mental health care and social support needs of formerly incarcerated Black women, work with the developers to refine the PerHL to include culturally informed content to support their mental health, and assess user satisfaction with the components of the PerHL that have been tailored to support the mental health of formerly incarcerated Black women.

Terika McCall (41:56):
So, work that I will begin now within the next couple of months includes conducting semi-structured interviews to determine the effects of the COVID-19 pandemic on the mental health of Black American women. So, this was recently funded by the Yale Women's Faculty Forum, and so I'm appreciative of that support. So, that study should pick off within the next couple of months. I will also begin collaborating on a project exploring tele-mental health service use among the LGBTQ+ veterans during the COVID-19 pandemic using EHR data. I also plan to submit my K01 mentored research scientist career development award proposal to conduct a multi-site pilot study to evaluate the feasibility, acceptability, and preliminary efficacy of the mobile app to support management of anxiety and depression among African-American women. The findings from the pilot will be used to further develop the app in preparation for a randomized control trial.

Terika McCall (43:01):
So, last but not least, I'm excited to teach a new course that I developed on user-centered design of digital health tools next spring at the School of Public Health. And I also want to plug the new consumer health informatics lab, CHIL at Yale, which will open this fall. And so, this is going to be a research space dedicated to the development and evaluation of consumer-facing products. Our mission is to improve the health of individuals by supporting innovative research through consultation on equitable design and usability testing of consumer health products for diverse populations. We are committed to assisting investigators in the development of applications and wearables that empower users to manage their own health and facilitate patient provider communication and shared decision-making.

Terika McCall (43:53):
So, here are selected references. And I would like to acknowledge my mentors for their guidance and feedback on my research activities, and also acknowledge the funding that I received from the National Library of Medicine's Institutional Training Grant for research training in biomedical informatics and data science. Thank you so much for your time, and I'd be happy to answer any questions at this time.

Cindy (44:19):
So, Terika, there are some questions in the chat. Thank you very much. I really appreciated that talk. Very well done. And there are some questions in the chat, some of which may have been answered by following up in your... If you want to breathe for a second and maybe just glance at them and see if they were already answered.

Terika McCall (44:42):
Okay.

Cindy (44:42):
Let's see.

Terika McCall (44:57):
Okay, so there's the question of, it says, 45%... This is Nathan. And please let me know if I didn't interpret your question-

Nathan (45:07):
Yeah, yeah. Hi. I can ask it. 45 wasn't the exact number, but it was in the 40s, and you were talking about, the word wasn't physician, it was professionals. And I think I'm asking it there correctly, but I was wondering what type of question... Because I could imagine I'd be okay texting a physician if it was just, "Hey, can you refill my script?" But it would be a different situation about if I had a real problem." That that was where I was going with that.

Terika McCall (45:38):
Right. So, the question that was asked is, would they feel comfortable communicating with a mental health professional if they needed support for managing anxiety or depression? And that, it was a pretty broad question, right? And so, from that, you can glean that it's just they want to communicate with them... And it can be interpreted that they want to communicate with them if they're having an issue with anxiety or they're having an issue with depression. But when we actually dig a bit deeper and looked at some of the open responses and their concerns with using the different modalities, we found that when it came to text messaging, they were fine with it, like, "Okay, if I need to text about a simple question or a medication refill," or something like that, but they felt like text messaging... And I didn't go through, there's a lot of data from this study, the concerns they had with different modalities, but they had concerns about using it for... For instance, if I'm having some type of panic attack. Right?

Nathan (46:41):
Okay. Okay. Yeah. I didn't say text, but I meant... So, you did tease that out. Okay, good.

Terika McCall (46:46):
Yeah.

Nathan (46:47):
Great. Thank you. Excellent presentation.

Terika McCall (46:48):
No problem. No problem. Thank you. Let me see if I can follow [inaudible 00:46:54].

Cindy (46:54):
[Hamad 00:46:54], was your question answered about the type of professional?

Terika McCall (47:04):
I think I just answered that one.

Cindy (47:06):
Oh, you did answer that one. Okay. How about, Denise's? I think Denise's was answered. Denise, did you-

Terika McCall (47:12):
[inaudible 00:47:12] population. So, I believe for the survey, the sample was pretty representative. If I go back to that slide, we had a pretty good breakdown. I'm going to have to click through a lot of things. But we had a pretty good breakdown when it came to the survey because we had a pretty large sample. But when it came to the focus groups and the usability testing, it skewed more towards having younger folks that participated. And going forward... So, here we go. Oh, actually, one more, one more. Yeah. [inaudible 00:47:54] can see when you look at age group in years, so we had a pretty nice breakdown by this age group, 18 to 24, or 25-34, and so on. But with the focus group's usability testing, there were more people that were, under 50.

Terika McCall (48:14):
And so, I think going forward, it may not be as much of an issue because I've been encouraged by my mentors to narrow down, [inaudible 00:48:25] can attest, to narrow down who am I going to focus on with this app? Right? And so, what we gleaned from the survey is that 25 to 34 year age group had the highest when it came to their GAD-7 and their PHQ-9 scores, they were highest with no severe anxiety or severe depression, so that's the group that I intend to focus on going forward when I'm thinking about tailoring this app a little bit more. So, Denise, please let me know if I didn't answer your question fully, or if you have a follow-up question.

Denise (48:56):
Yeah. So, I do. Sorry. My somewhat followup was... And again, I don't know what the social demographic characteristics are for African-American women across everything, but it seems like at least in this group that they were fairly highly educated, and whether or not that's really representative and if you're missing a target population that may have more... And again, this is one of the issues with surveys all the time, but if you're missing a target demographic there that potentially could help in doing some of your modifications, et cetera.

Terika McCall (49:36):
Correct. Yeah. So, it was certainly representative as far as age. Going forward, I'm going to do more purposive sampling to get more people who have less than a Bachelor's degree and those who may not have as high of an income. But what I'm finding is that even among those who are more educated, they are still not utilizing mental health services as needed. Basically, even within this survey, we found that 40% were saying that there was a time in the past year that they needed mental health services and they didn't go get it. And they listed barriers such as insurance and whatnot. So, I think that it's a problem even across socioeconomic status, but going forward, I'm going to do more purpose of sampling so that we can capture more information on those who may not be as highly educated.

Cindy (50:38):
I think Robert and/or Hamad had their hands up. Who wants to go first?

Speaker 5 (50:44):
I just had a very brief follow up on that. I mean, presumably, only 3% that just have a high school degree is an artifact of the snowball sampling.

Terika McCall (50:59):
Right.

Speaker 5 (51:03):
You've sort of touched on this a little bit, as to the disadvantages of that and the need to do more purposeful sampling, but I just wanted you to say a little bit more about why you went down that road.

Terika McCall (51:17):
Yes.

Speaker 5 (51:18):
Yeah.

Terika McCall (51:20):
Yeah. So, I went down the road of one [inaudible 00:51:23] basically launching the survey and partnering with a large national organization like National Council of Negro Women, who has a broad reach, to just find information about Black women and their mental health care needs and their desire to actually use tele-mental health modalities. Because I wasn't able to find this information out there as I was getting ready to plan my dissertation. And so, this is just a starting point for me and just finding out where are we as far as accessibility within this community. And the less than the high school diploma, those that only have a high school diploma, that's where the more purpose of sampling would come in in the future and trying to capture data from those individuals, maybe doing more community outreach centers, things of that nature, so that I can get more of those people.

Terika McCall (52:24):
And so, when thinking about how I would conduct this survey in the future, I think that it was great to have it being web-based, but I think, also, if I could maybe have paper surveys and camp out in a community health center, then I would capture more of those people that may not be as highly educated or may not have as high of an income. Right? So, thinking about where are those people and how will I get information from them so that their needs are also captured in how I'm designing the app so that I can make sure that it's going to be valuable for them.

Speaker 6 (53:04):
Yeah. So, I was just commenting... First of all, congratulations on this very cool and important work. And very timely. I mean, this area of inquiry is just emerging right now. I think it's also really innovative and wise to have consumer-facing center. And along those lines, a lot of institutions, including Yale, have these mental health apps for direct communication with providers. There's an opportunity there because there's going to be a lot of big data. I was wondering if you were thinking about trying to get data either from those companies or from Yale itself on usability access effectiveness of those apps. I gave the example of Talkspace [inaudible 00:53:50] which again is actually provided for Yale employees.

Terika McCall (53:57):
Yeah. So, that actually is a good idea. I hadn't thought about that, but you bringing that up, that's something for me to explore in the future. I know that the VA also has their own set of mental health apps, and I'm starting to collaborate with folks over there and do some work over there. So, thank you for bringing up that point, that some of these big commercial apps, too, maybe able to get data from them and to explore that. Thank you.

Speaker 6 (54:25):
Yeah. And that many people at Yale have been successful in getting that data. As an example, [inaudible 00:54:31] has gotten [inaudible 00:54:32] data [inaudible 00:54:33], so I think there's that potential there to do very cool stuff.

Denise (54:38):
The other suggestion I would make just in terms of your K awards and maybe potentially now, if you're going to do this work here as opposed to North Carolina, is the CTSA has cultural ambassadors, which reach out into the community through churches and other types of things. And so, when you're thinking about your K award, that might be a nice collaboration, as well, to try and reach some of those people in the community.

Terika McCall (55:09):
Oh, absolutely. Thank you for bringing that up because I'm definitely looking to build my network here. I know the players in North Carolina, so now that I'm here at Yale, that will be a very useful resource, and collaborating with them would be great. Thank you so much.

Cindy (55:26):
And she is very well-integrated with Eric through Karen's work and Marcella, so this is totally... This will be a great opportunity, I think.

Terika McCall (55:42):
I'm scrolling through to see if I have any more questions.

Cindy (55:46):
You've got a couple comments from Garrett, and I don't know, maybe you've talked to Garrett offline or if Garrett wanted to voice them. I think they were maybe more suggestions.

Terika McCall (55:57):
Yes. He sent me some articles.

Cindy (56:02):
Okay.

Terika McCall (56:02):
Thank you, Garrett.

Garrett (56:03):
You're welcome. Those are more kind of big pick, maybe like next thing, like you might want to put some training on these in your K award for future R1s, that sort of thing.

Terika McCall (56:15):
Okay, cool. I appreciate that. I'm going to look at these links that you sent me. Any more questions, comments? Well, thank you, everyone. I will not hold you for the last two to three minutes. It's the holiday weekend. I hope everyone has a great time. And get some rest. Thank you so much for having me. And feel free to reach out to me, email, if you have any follow-up questions. Thank you so much.

Cindy (56:57):
Thank you very much, Terika. Thank you for Biostats hosting. Have a great weekend.

Terika McCall (57:04):
Bye.