Social Work Students’ Self-Efficacy Toward Direct Practice Skills in Field Education Using Virtual Simulations and Scripted Role Plays

: Simulations with professional actors and scripted role plays with peers are effective methods to increase direct practice skills. However, little is known about how simulations or scripted role plays conducted virtually can influence social work students' practice self-efficacy. MSW students enrolled in field seminar courses across two universities were invited to participate in an exploratory, repeated measures assessment utilizing the Counselor Activity Self-Efficacy Scales (CASES). One university (n=100) implemented the use of standardized clients, played by professional actors within field seminar; the other university (n=61) implemented scripted, peer-led role plays. Significant differences were found in pre/post scores among MSW students that participated in simulated client experiences within their field seminar. Simulations and scripted peer role plays may need to be more integrated into social work curricula when opportunities for in-person direct practice skill development are limited due to hybrid or fully remote field placements. Applied learning in social work education must be re-envisioned so programs can prepare MSW students to be effective practitioners in today’s rapidly changing environment.

Educational disruptions due to the global COVID-19 pandemic placed limitations on social work students' ability to learn and apply direct practice skills in traditional curriculum methods (Kourgiantakis et al., 2021;Menon et al., 2021). Of concern were the lost opportunities for skill development within field education learning experiences. Field education, considered the signature pedagogy in social work education, allows students to integrate knowledge and practice skills in real practice situations (Council of Social Work Education [CSWE], 2015). Field settings provide opportunities for students to demonstrate social work competencies with clients, families, groups, organizations, and communities. It is in these supervised, in-vivo experiences, that students can integrate theory and practice in hands on interventions with actual clients. When the COVID-19 pandemic quickly transitioned social work programs and field work to an online format during the 2019-2020 academic year, in-person learning experiences were largely altered, or eliminated altogether, thus significantly impacting direct practice skill development for social work students. Responses to the pandemic included taking measures to prevent the spread of the coronavirus, physical distancing measures and finding practicum opportunities for students that involve direct practice with clients. Some schools moved to Virtual Practice Fridays and utilized simulation-based learning (Kourgiantakis et al., 2021;Lee et al., 2020); Online simulation-based learning activities have positive preliminary findings and implications for social work education and practice, and can help fill the gap when there is a shortage of in-person practice opportunities in the field (Kourgiantakis et al., 2021). Additional research is needed to determine effectiveness of simulation-based learning specifically in the field of social work education.

Context and Rationale
The Southern California Field Directors' Consortium is composed of 12 universities that collaborate and coordinate throughout the academic year to ensure equity among placement opportunities for all social work students in the region within their respective field education program. Given the lost opportunities for in-person experiences in field education, two social work programs in the consortium sought to enhance direct practice skill development for the 2020-2021 academic year by virtually implementing simulations or scripted peer role plays within field seminar classes. Simulation and role plays are two empirically supported pedagogies that are found to be effective at increasing assessment, communication, and intervention skills (Bhattacharjee, 2014;Bogo et al., 2014;Kettula & Berghall, 2013;Kourgiantakis et al., 2020). However, little is known about how effective these two pedagogies are when implemented virtually in the context of social work education. The programs had differing student populations and program structures such that a robust comparison study was not warranted. Instead, we conducted an exploratory assessment with repeated measures to understand how these two pedagogies, when implemented virtually, impacted MSW student's direct practice self-efficacy.

Background Online Learning
Distance education, virtual or online learning in social work has an interesting past from the use of satellite television to iPads, social media, and avatars (Hitchcock & Young, 2016;Vernon et al., 2009;Wretman & Macy, 2016;Young, 2014). The use of technologies has been met with resistance for a variety of reasons including learning outcomes and the efficacy of an online format (Wretman & Macy, 2016;York, 2008). However, the social work tech literature has greatly expanded in the past several decades with evaluations of online learning, indicating this method produces similar results to traditional classroom learning models (Cummings et al., 2012;Wretman & Macy, 2016). Additional research has shown that virtual environments help students increase cultural awareness (Stauss et al., 2018), clinical skills (Rizzo et al., 2019), and increase student preparedness for practicum (Phillips et al., 2018). More research is needed to fully understand the ways technology can support or impede social work education, but the COVID-19 global pandemic has shown how important it is for social work education to be flexible, adaptable, and open to utilizing virtual learning methodologies. These methods can incorporate simulated or scripted peer role plays to help increase student self-efficacy.

Simulations With Actors
"Simulation-based learning is an experiential method that utilizes actors portraying simulated clients in scenarios designed to emulate social work practice" (Kourgiantakis et al., 2020, p. 434). The use of simulated learning and training is well developed in medical education and medical personnel evaluation (Issenberg & McGaghie, 2013;Motola et al., 2013). The seminal work of Bogo and colleagues (2014) has influenced the social work profession's adoption of such learning techniques. Simulations are a promising pedagogical method for teaching social work practice in the classroom , and can be implemented in a variety of formats such as online, with actors or clients, or small group formats (Goldingay et al., 2018;Osborne et al., 2016;Wilson et al., 2013). Simulation-based learning can provide social work education programs with the ability to evaluate student competencies effectively (Lee et al., 2020;Sewell et al., 2020), and virtual simulations have been shown to help increase clinical skills, develop cultural competence, and further prepare students for the field (Lee, 2014;Phillips et al., 2018;Rizzo et al., 2019).
There is very limited research on the use of professional actors in social work education. However, some exploratory studies found that social work students described their experience with the actor-simulated clients in very positive terms and described the experience as close to real-life (Petracchi, 1999;Petracchi & Collins, 2006). Using professional actors brings the experience closer to reality due to actors having greater professional background in role play, whereas peers may feel uncomfortable, awkward, and unfamiliar with acting in role plays in a client role. Students may also find it difficult to play the role of the client in the beginning of their social work education due to lack of confidence and familiarity.

Role Plays
Role play pedagogy has been shown to be effective in meeting learning outcomes in a variety of contexts and content areas (Allemang et al., 2021;Jones & Conner, 2021. The efficacy of role plays in learning environments is often cited outside the discipline of social work. Role playing is widely recognized as an important experiential teaching method in higher education across a range of disciplines such as education, nursing, clinical psychology, counselling, pharmacy, veterinary medicine, and medicine (Fulton et al., 2019). In role plays, students are given opportunities to simulate specific scenarios to help prepare students and increase their level of readiness for the field. In a safe environment with their peers and instructors, they observe others and practice developing their competence.
Scripted role plays provide a structured framework that is often tailored to learning outcomes by providing written lines for all participants to use as they act out scenarios (Hargreaves & Hadlow, 1997;Villadsen et al., 2012). Compared to more traditional teaching methods such as direct instruction, memorization, or recitation of knowledge/skills, the literature supports the idea that scripted role plays may better meet the learning needs of today's students (Bobbit et al., 2001;Elmore, 2021;Jones & Conner, 2021;Osborne et al., 2016;Rosa, 2012). For social work students, scripted role plays allow students to problem solve and practice decision-making in relation to intervention and assessment skills in a low-stakes environment. These role play learning experiences are then further developed when students can critically reflect on their or their peers' performance/engagement within the activity and construct linkages to how these experiences can transfer to the field (Jones & Conner, 2021). This combination of role play and reflection can be an effective assessment measure that enables increased self-efficacy and awareness. Jones and Connor (2021) assert that reflection of role play experience supports students' exploration at both the cognitive and affective levels. Furthermore, Allemang and colleagues (2021) affirmed previous findings that experiential learning strategies such as role plays enhance perspective-taking, provide an opportunity to practice skills in a safe space, and is perceived positively by students and instructors.

Self-Efficacy
Self-efficacy refers to an individual's belief in their capacity to demonstrate specific skills, knowledge, or abilities (Holden et al., 2017). Given the remote or hybrid option of their field placement, it was assumed that our social work students had decreased opportunities to practice counseling skills (i.e., direct practice) with in-person clients. As such, it was warranted for our field programs to understand how students perceived their ability to deal with client issues and perform various counselor behaviors taught within their direct practice coursework, which were also conducted remotely. Direct practice skills in social work involves helping clients through challenging situations by assessing clients' needs and directing them to resources. It is imperative that more research is done to evaluate the adoption of virtual practice methodologies within social work education generally and field education specifically.

Methods
A purposeful sample of MSW students from San Diego State University (SDSU) and California State University San Marcos (CSUSM) that were concurrently enrolled in practice and field seminar courses in 2020-2021 academic year were invited to participate in the study. Data collection was administered twice during the Fall 2020 semester, once at the beginning of the semester (August 2020) and again at the end of the semester (December 2020) through Qualtrics, an online survey platform. All research was in accordance with the ethical standards of the San Diego State University Institutional Review Board. This study was verified as exempt by the Institutional Review Board (Number: HS-2020-0185).

Field Seminar Format
The purpose of the field seminar is to build a collective understanding of the profession of social work through the mutual and purposeful sharing of students' experiences at their various internship sites. For the 2020-2021 academic year, all courses within the California State System were required to be virtual, including field seminars. Given that many field agencies continued remote or hybrid internships for the 2020-2021 academic year, our field education programs expanded content to compensate for the potential loss of direct practice skill building opportunities by implementing simulations or scripted peer role plays in field seminar classes.
Simulations. Simulated practice experiences utilizing standardized clients portraying scenarios commonly encountered in social work were implemented in field education seminars for foundation and advanced year MSW students at SDSU. Students were concurrently enrolled in direct practice courses focused on teaching evidence-based practices such as motivational interviewing (MI), cognitive behavioral therapy (CBT), and solution focused therapy. All simulation practice experiences were conducted on a virtual platform (Zoom). Each student had the opportunity to conduct a one time, 15-minute assessment with the standardized client (professional actors or school of theatre, film, and television students). Students selected and employed evidence-based practice skills in their assessment that they believed were appropriate given the information they received about the "standardized client" on the assigned case vignette. Other students in the class not interacting with the standardized client were asked to carefully observe the live exchange and provide constructive feedback to their peer after the simulation activity. This provided an additional professional development opportunity for students to learn how to give appropriate feedback to their peers as they are often called to do in the profession.

Scripted Peer-Role Plays.
Scripted peer-role plays were implemented in field seminar classes for advanced and foundation year MSW students at CSUSM. Scripted peer role plays were utilized to provide a more standardized experience among role play activities. All scripted role plays occurred over Zoom. Initially, a role play demonstration was conducted with the entire class; then, students utilized small breakout sessions to practice skills in smaller groups. Students playing the role of the client were given specific scripts and vignettes and were coached to remain consistent and avoid deviating from the information provided. The scripted role plays addressed various competency areas, including engagement, assessment, and intervention. Finally, the students playing the social worker and client roles were given time to discuss and reflect on the role play experience. Additionally, field faculty and peers provided immediate feedback to highlight student strengths and areas of development.

Measures
We used the three scales that comprise the Counselor Activity Self Efficacy Scales (CASES) to assess student's self-reported efficacy on therapeutic skills, session management, and handling of challenging counseling situations. CASES were administered prior to the first scheduled field seminar in August 2020 and the last scheduled field seminar in December 2020. CASES was designed for student populations and focuses on beginning practitioner skills (e.g., reflection, listening) that directly translate to social work learning competencies (Lent et al., 2003). As such, we felt this was an appropriate measure to understand how virtual experiences with role play and simulations can contribute to social work student's practice self-efficacy. However, these scales are only designed to understand student perceptions of their own counseling capabilities and should not be considered an objective measure of how well students function in counseling sessions.
Counselor Activity Self-Efficacy Scales. The CASES questionnaire consists of 41 items (Lent et al., 2003). The three subscales are: helping skills self-efficacy (15 items), session management self-efficacy (10 items), and counseling challenges self -efficacy (16 items). For each item, respondents are asked to use a nine-point Likert scale (0= no confidence, 9= complete confidence) to indicate their perceived confidence in their ability to perform various counseling tasks. Each subscale can produce a mean by summing all the corresponding items together and dividing by the number of items. Sample items include, on a scale of 0-9 (0= no confidence, 9= complete confidence), "How confident are you that you could work effectively over the next week with a client who is clinically depressed?" and "How confident are you that you could use restatements effectively with most clients over the next week?" Internal reliability estimates for each scale ranged from α=.91 (helping skills) to α=.92 (session management skills) to α= .95 (counseling challenges skills). The full CASES scale produced an alpha coefficient of α=.97, which is consistent with previous research (Bagheri et al., 2011;Lent et al., 2003).

Analysis
Data collected from the online surveys (August 2020 and December 2020) were uploaded into SPSS (v. 25) for analysis. Because one of the goals of this study was to describe the perceived effectiveness of direct practice skills over a semester, we used univariate analysis to explore the overall characteristics of each scale (helping skills, session management, and counseling challenges). For simulation participants, pre/post responses were linked together by student ID. We used a repeated measures within-group analysis to explore differences in self-efficacy regarding direct practice skills from August 2020 to December 2020 and between advanced year and foundation year students. For scripted role play participants, no same students participated in the August 2020 and the December 2020 data collection (though all were current students in the Department's MSW program). As such, paired sample t-test to establish mean differences from pre/post could not be computed because there were no valid pairs in the data. Therefore, we combined all August and December data for the scripted role play participants and conducted independent samples t-test to compare the CASES scale means of foundation and advanced year students.

Scripted Peer Role Plays
The CSUSM Department of Social Work implemented scripted, peer-led role plays within field seminars. Foundation and Advanced year MSW students, for a total of N=61, completed the questionnaires for a response rate of 40%. Of the 61 total respondents, 42 respondents completed the questionnaire in August 2020 and 19 completed the questionnaire in December 2020. Foundation level students accounted for 49% of respondents (n=30) and advanced year level students accounted for 51% of respondents (n=31). Demographic data were not collected in the current scripted role play student sample; however, student demographic data are available for the department as a whole. Nearly half (49%) of the Department's students identified as Hispanic/Latino, with smaller percentages identifying as Asian American (3%), African American (3%), or multiple races (4%). Additionally, many students identified are bilingual and first-generation college students (California State University Institutional Research & Analyses, 2021).

CASES Subscale Means by Student Level
Means of the CASES subscales by student level (advanced or foundation year) for August 2020 and December 2020 are found in Table 1. For all three subscales, in December 2020, advanced year students had higher means of perceived self-efficacy than foundation year students. There were no statistically significant differences between foundation and advanced year students' self-efficacy scores for all three sub-scales at either the August or December CASES data collection times. Helping skills self-efficacy. This CASES sub-scale consists of 15 items. Participants were asked to indicate how confident they felt in using general skills effectively with clients at their field placement over the next week. The lowest item mean was related to participant's confidence to assist the client to role play or rehearse behaviors in session, (M=4.68). Differences between foundation and advanced year students for this item was not significant (p=.068), though approaching significance at the .05 level. The highest item mean (M=7.20, p=.296) was related to the counseling skill of listening (e.g., capture and understand messages the client is communicating). No significant differences were found between foundation and advanced year students.

Session management self-efficacy.
This CASES subscale consists of ten items specific to session management tasks. The lowest item mean in the session management subscale was related to participant's confidence in building a clear conceptualization of the client's counseling issues (M=5.48, p=.774). Difference between advanced year and foundation year students was not significant. The highest mean item was for "helping your client to explore their thoughts, feelings, and actions" (M=6. 65, p=.160). No significant differences between the highest item means between foundation and advanced were found.
Counseling challenges self-efficacy. This CASES subscale consists of 16 items specific to displayed client behavior or client symptomology. Participants were asked to indicate their confidence level in working with clients who displayed various challenging behaviors. The lowest perceived confidence means were for working with clients that show signs of severely disturbed thinking (advanced, M=4.45, p=.204). The highest mean item (i.e., highest perceived confidence) was related to working with clients that differ from you in major ways (e.g., race, gender, age; (M=7.25, p=.884). No significant differences between foundation and advanced year students were found on these items.

Simulation Participants
Foundation and advanced year MSW students enrolled in field seminar courses were invited to participate in the study. The School of Social Work implemented standardized clients, played by professional or student actors, within each field seminar for Fall 2020. A total of 100 MSW students completed the questionnaires for a response rate of 50%; responses were linked by student ID. Of the 100 participants, 54% were advanced level, 41% were foundation level, and 4% did not indicate their program year. Demographic data for the current simulation student participant sample are unavailable, but school-wide student demographic data are available. Most of the School's graduate students identify as White (37%) or of Hispanic/Latino (31%) ethnicity and 22% are first generation college students (San Diego State University Institutional Research & Analyses, 2021).

Pre and Post CASES Subscale Means
Simulation participants perceived self-efficacy across all CASES sub-scales increased from August 2020 to December 2020. Means increased for all items within each sub-scale (helping skills, session management, and counseling challenges). For the helping skills sub-scale, the most considerable mean difference from pre (M=3.40) to post (M=6.95) was for the item "Immediacy (disclose immediate feelings you have about the client);" p<.001. The session management sub-scale item "Remain aware of your intentions during sessions" had a significant mean difference among simulation students from pre (M=4.42) to post (M=8.00), p<.001. For the counseling challenges sub-scale item "How confident are you that you could work effectively over the next week with a client who is extremely anxious" had the most significant mean difference from pre (M= 4.05) to post (M=7.42), p<.001.

CASES Subscale Means by Student Level
An independent-samples t-test was run to determine if there were difference in CASES subscale scores at pre and post between advanced and foundation year simulation students. Advanced year students had less perceived self-efficacy with counseling challenges at baseline (pre), a statistically significant difference of .91 (95% CI, -.05 to 1.89), t (44)=1.92, p=.031. Means of the CASES subscales by student level (advanced or foundation year) for simulation participants are found in Table 3. For both groups of students, means increased from pre to post. At pre-assessment, foundation year students had more perceived self-efficacy than advanced year students in session and counseling skills. However, at post assessment, advanced year students had slightly more mean increases in session skills and counseling challenge skills than foundation year students.

Discussion
This study aimed to explore MSW students' perceived self-efficacy by the virtual learning method of two pedagogies. Results indicate an increase in student confidence in using direct practice skills following participation in both simulation and scripted role play from August 2020 to December 2020. Scripted role play participants appeared to have higher confidence levels at the beginning of the semester when compared to simulation participants; additionally, simulation participants had larger mean increases compared with scripted role play participants from August 2020 to December 2020. The variability in demographic makeup between the two universities may have impacted the higher baseline confidence levels for CSUSM students. Additionally, students' simulation was recorded and shared with their respective internship supervisors, providing another opportunity for feedback and reflection, while scripted role play students did not have the opportunity for additional reflection with their field supervisors. This may have contributed to the higher increase in mean scores for simulation participants overall. Most notably, CASES means did increase for all students regardless of pedagogy. It should also be noted that the results from the CASES questionnaire among advanced-year scripted role-play students were collected from a different group of participants than the foundation year students within the scripted role play group in August and December. Moreover, there were no significant differences between advanced year and foundation year students on any of the CASES subscales for either scripted role play or simulation participants.

Implications for Social Work
Due to the COVID-19 pandemic, most student internships at both CSUSM and SDSU utilized most or some aspects of telehealth service delivery, which was not the typical experience for student interns at either university prior to the pandemic. The virtual simulations and scripted peer role plays provided an opportunity for supplemental practice of telehealth services, which will most likely be a service delivery method to stay in the post COVID-19 era. We offer the following observations and next steps given our study's findings: 1. We did not collect any demographic information from the participating students. Therefore, race/ethnicity, age, and part-time/full-time status may contribute to mean differences among participants. Future research exploring the effectiveness of these pedagogies in virtual environments should account for individual student demographics.
2. Though some role plays and skill-building demonstrations are included in field seminars, gaps exist within students' ability to fully engage in role play while effectively demonstrating skills such as rapport building, active listening, conveying empathy, and reflecting content. Standardized clients through virtual simulation may have a more realistic learning opportunity that further facilitates active student engagement above and beyond what can be achieved during a peer-to-peer role play.
3. In our study, the student's field instructor and standardized client provided structured feedback to each student engaged in the simulation practice experience because the simulation was recorded and shared with their respective internship supervisor (i.e., field instructor). For scripted role play students, feedback was only received from their peers and field faculty, not their supervisors in the field. These additional opportunities for feedback and reflection may help explain the significant differences of pre/post self-efficacy among simulation participants overall and not by student level. However, since pre/post analysis could not be conducted with scripted role play participants, it will be important for future studies to examine effectiveness of virtual scripted role plays pre/post.
The COVID-19 pandemic quickly shifted service delivery to virtual environments, which limited or reduced in-person service delivery as individuals moved towards more virtual service delivery. We recommend that social work programs, including field education programs, enhance students' direct practice skill learning opportunities. Furthermore, real-life situations rarely afford a student the opportunity to elicit in-depth performance feedback from a client as it may not be ethical or appropriate, bearing in mind inherent power differentials and that the focus should be on the client's needs. Therefore, it can be a powerful learning experience to be able to hear how the standardized client or peer perceived the student's performance. For example, all students were able to receive immediate feedback on how the "client" (actor or peer) perceived the student's direct practice skills in relation to areas such as: efforts to make an authentic connection, efforts to demonstrate understanding and empathy, opportunities to collaborate, respecting client's right to self-determination, and efforts to approach the session with cultural humility. Despite the hopeful return to more in-person experiences, intentional efforts to increase simulated or scripted role play opportunities in field seminars and practice courses would allow for thoughtful skill development focused on telehealth training, which will move social work education into a future that is becoming more and more technologically reliant.
Moreover, another noteworthy approach that was utilized in this study is the interprofessional collaboration between the School of Social Work and the School of Theatre, Film and Television. Creating interprofessional learning opportunities for students using role play and drama can be an effective pedagogic method in interprofessional education (Villadsen et al., 2012). Not only does this provide social work students a chance to practice with individuals who have greater role play expertise, it additionally gives acting students an opportunity to play the client role and develop their own work experience.

Limitations
The results of this study should be carefully considered. First, the reliability and validity of the CASES instrument are still considered "under construction," psychometrically speaking, despite initial promising findings (Lent, 2020). It is important to note that self-reported measures, especially in field education can be inaccurate. Overestimates of skills, knowledge, and abilities in relation to the CASES could have happened among the study's participants. This phenomenon and taking into account student's learning in other aspects of their program could have influenced student's increases in self-efficacy. Therefore, we cannot be certain to what degree that the virtual pedagogies of focus in this study contributed to participants increased self-efficacy. However, our study found sufficient reliability that will contribute to these efforts.
Second, due to the COVID-19 public health orders, our recruitment and data collection efforts were altered, limiting both universities to virtual methods. For students experiencing "Zoom fatigue" and other academic stressors, our findings should not be considered representative of the department or schools' entire social work graduate student population. These methods also likely recruited students that were more comfortable with online activities and who likely had higher computer literacy skills. Additionally, it was not possible to conduct a repeated-measures analysis of the scripted role play group due to having different groups who completed the pre-and post-survey. This impacted the ability to compare both simulation and scripted role play groups statistically. Further studies are needed to determine if there are significant differences among MSW students participating in scripted peer role plays.

Conclusion
The COVID-19 pandemic quickly shifted service delivery to virtual environments, which limited or reduced in-person service delivery as individuals moved towards more virtual service delivery. The COVID-19 pandemic also highlighted the vast injustice of the digital divide among higher education, particularly in student and field site's ability to respond to the COVID-19 pandemic. It is more important than ever to prepare social work students to work with clients, families, groups, and communities-both through virtual and in-person means. Preparing social work students for the virtual environment through direct practice skill activities allows students to gain a better understanding of the complexities of telehealth service delivery. We encourage social work educators to continue to adapt teaching practices to respond to the reliance of technology practices such as telehealth, while also pursing efforts to close digital divide disparities. The use of scripted role plays and simulations may be one way to better prepare students direct practice skills development and increase self-efficacy in post COVID-19 social work education.