Translating Concept into Act for Multi-Cultural Practice : Comparison of Students ’ and Field Instructors ’ Perceptions of Diversity Training Effectiveness

Education for culturally competent practice increasingly is a responsibility for social work educators. Using data collected for an evaluation of the field education component of a large, Midwestern social work program, the purpose of this study is to shed light on students’ application in the field practicum setting of classroom training in culturally competent practice. Responses were obtained from field instructors (n=76) and students (n=70). Students reported learning in areas dealing with diversity at statistically significant levels; however, instructor ratings of student competence were significantly lower than student ratings. Results are discussed in light of necessary attitudes, knowledge, and skills. Implications for program monitoring and improvement, education, and further research are discussed.

authors suggest objectives related to cultural competence and several learning activities related to each curriculum area, including field.
Other more specific, classroom-based educational strategies abound, many of which are experiential in nature, in order to foster both cognitive and affective learning.
Although strategies vary, many are designed to increase students' awareness and knowledge of self and others in terms of racial, ethnic, or cultural identity.For instance, Aponte (1995) requires students to write about their experiences with an unfamiliar cultural group.Chau (1990) details a technique termed "ethnic self-profiling," which involves a process of identifying feelings associated with various words that refer to racial or other forms of human diversity.Following a semester of activities, Torres and Jones (1997) ask students to write an "integrative cultural paper" in which they must address their own and others' cultural identity, dynamics of aversive incidents, and thoughts about their potential for culturally competent practice.The discussion of vignettes containing examples of bias provides students with the opportunity to increase their ability to think critically about the function of bias (Latting, 1990).The utilization of "cultural guides" (Ronnau, 1994), "intergroup dialogues" (Nagda et al., 1999), web-based discussion groups (Van Soest, Canon & Grant, 2000), and "critical incident interviews" (Montalvo, 1999) allow students of diverse group memberships the opportunity to interact in order to gain awareness and knowledge of one another.Finally, Boyle, Nackerud and Kilpatrick (2000) describe a non-classroom-based international immersion experience.
Articles that describe educational strategies, such as those mentioned above, have made several important contributions to diversity education, including providing social work educators with a source of teaching strategies that can be adapted for use in numerous diversity related courses.Furthermore, this body of literature illustrates approaches for combining didactic and experiential teaching methodologies, strategies considered important in reaching the range of adult learners enrolled in social work programs (Knowles, 1990).In addition, the strategies described target change among various combinations of the dimensions of awareness, knowledge, and skills.Several strategies (Latting, 1990;Montalvo, 1999;Nagda et al., 1999;Van Soest et al., 2000) also include a focus on social justice issues and diversity practice.
Although many authors have strongly supported the need to evaluate the effectiveness of educational strategies for cultural competence, relatively few have reported outcomes.Of these, most examine specific educational interventions.Table 1 briefly describes interventions, objectives, measurement, and results for eight studies that report outcomes of educational interventions designed to increase competence for practice with diverse populations.Although the interventions vary somewhat by conceptual foundation and strategy, most of the objectives are similar in their aim to increase students' levels of awareness, knowledge, or skills.A few of the authors, however, were also interested in objectives related to the methodology, for example, the creation of a safe, facilitative environment (Ronnau, 1994;Van Soest et al., 2000) or students' reaction to the intervention itself (Montalvo, 1999).The eight studies share other characteristics as well.All of the studies measured outcomes by obtaining students' perceptions of learning immediately after the interventions were completed.A few authors measured outcomes solely by students' verbal or written comments (Chau, 1990;Torres & Jones, 1997).Others increased the methodological rigor somewhat by including Likert-scale or openended survey questions (Ronnau, 1994;Montalvo, 1999;and Van Soest et al., 2000) or formal qualitative analysis (Latting, 1990;Nagda et al., 1999;Boyle et al., 2001).Pre-post testing with a standardized measure of cultural competence was utilized in only one study (Boyle et al., 2001).Finally, all eight studies reported positive outcomes for every objective with the exception of one, which was related to skill building (Torres & Jones, 1997).
In summary, existing evaluations of the effectiveness of educating social work students for cultural competence have focused on immediate outcomes of specific classroom interventions and relied on students' perceptions of learning for outcome measurement.Without exception, the various educational strategies have produced positive outcomes.We know very little, however, about how students are applying what they know about cultural competence to their practice.
We have located only one study that reported on the application of cultural competence education to practice.Rittner, Nakanishi, Nackerud & Hammons (1999) surveyed agency-based social workers with at least two years post-MSW experience in order to examine the effect of cultural competence content in MSW curricula on social work practice with small groups.A large majority of those surveyed indicated that diversity content was part of their MSW education, most frequently recalling attention given to areas of age, culture, gender, national origin, and race.Despite the reported breadth of exposure, however, respondents indicated difficulties with applying diversity content to their small group practice.Many respondents acknowledged that they frequently failed to address group member differences through their interventions.In addition, the repertoire of interventions was extremely limited among those who reported addressing diversity.The authors concluded that their results suggest MSW diversity content is applied in very limited ways among social workers practicing with small groups.
If cultural competence involves a developmental process, the knowledge and awareness developed in classroom learning should result in a display of skills in the field practicum and in later practice.The literature has documented classroom strategies to develop knowledge and awareness and continued recall of cognitive content several years into practice; however, the literature also reports limited application of this content in post-MSW practice.Notably missing from the literature is information about one of the steps in the developmental process: application, integration, and amplification of cultural competence learning in the field practicum.A national survey of social work faculty documented this gap (Le-Doux & Montalvo, 1999).Respondents indicated that they observed minimal to no linkage of classroom instruction for cultural competence with students' fieldwork.The authors point out that this is particularly troublesome since the field practicum represents the primary opportunity for students to integrate theory and practice.It is in the field that students are expected to apply the awareness, knowledge, and skills learned in the classroom to actual practice with diverse populations.
The other major omission in the literature is an evaluation of cultural competence learning from perspectives beyond the student per se.Field instructors, in particular, are in a unique position to observe the application of classroom learning and, through supervision, to promote further growth.Their perspectives could be expected to provide an invaluable addition to faculty members' observations and students' self-reporting.
Using data collected for an evaluation of the field education component of a large (approximately 600 students) Midwestern social work program, the purpose of this study is to shed light on students' application in the field practicum setting of classroom training in culturally competent practice.Responses were obtained from students and field instructors.Thus, this study adds to the literature in three ways: (a) by looking at cultural competence in the field practicum, (b) by eliciting field instructors' perspectives on student cultural competence, and (c) by allowing for a comparison of students' and field instructors' perspectives.The data were collected during the 1999-2000 academic year.Although each level of field practicum (baccalaureate, foundation, concentration) includes objectives related to diversity, this study focuses specifically on the MSW foundation practicum.

CONTEXT
The MSW program which serves as a base for this study uses a model for diversity education in which this content is offered in specific courses.Students choose between focusing on race/ethnicity or women's issues, although both courses include content related to within-group diversity and a range of other diversity issues.Both courses use a variety of didactic and experiential techniques.In addition, as appropriate to the course, diversity content appears in most of the courses in the curriculum.Faculty are required to document in materials for their annual performance evaluations that content on women, ethnic minorities, and gays/lesbians is appropriately included in their courses.Such content is included widely across the curriculum, as documented in a recent review of course syllabi conducted as a part of the self-study for reaffirmation of the Council on Social Work Education (CSWE) accreditation (Kondrat et al., 2002).
Students begin the MSW foundation practicum during the second quarter of the first year after completing one course in each of the foundation areas: human behavior, generalist practice, policy, and research.At the time the program evaluation was conducted, students are typically enrolled in the diversity course during the third quarter, when they would have been completing the final quarter of foundation field.The study was conducted at a time when the respondent cohort of students had completed the first year of the program and were beginning the second concentration year classes and practicum.In contrast to previous studies, then, this study attempts to evaluate the effectiveness of diversity training which occurs in a number of different ways throughout a major portion of the curriculum rather than the effects of specific activities confined to a single course.

METHODS
For the field program evaluation, mail surveys were designed to collect data from students and field instructors.For this study, respondents were students in the concentration year, along with concentration field instructors.The student survey asked students to rate their achievement of foundation learning objectives, stated as foundation level competencies, using a retrospective pre-test post-test design.The field instructor survey asked instructors to rate the level of competency of their incoming concentration students on the same foundation level competencies in a post-test only design.Ratings were obtained late in the first quarter of concentration field instruction.Figure 1 details the data collection timeline.
The format of the student survey included pairs of "when I began my MSW II field placement" (post-test) and "before my MSW I placement" (retrospective pretest) items on which students rated their skills on a five-point Likert scale from 1 = totally disagree to 5 = fully agree.The retrospective pre-test is a way to obtain preand post-intervention ratings in a single data collection, which was necessary for the evaluation timeline.Furthermore, retrospective pre-tests represent a way to enhance validity by guarding against response-shift bias that may occur in a pretest, post-test design where the perception of the dependent variable is initially abstract.For instance, in a traditional pre-test, the student might be asked to rate her or himself on "being aware of and able to observe appropriate boundaries with clients in regard to self-disclosure and dual relationships."Having never experienced boundary dilemmas, the students' rating of such behavior may be based on the abstract perception of the expected or desired behavior.Over the course of the field practicum, if the student actually encountered boundary issues or struggles with self-disclosure or dual relationships, he or she then had more concrete experience on which to rate actual awareness/behavior.The change in consciousness may result in a lower rating at post-test, not because the student got "worse" in this skill, but because the student did not have an adequate basis on which to make the pre-test rating.Conversely, a student rating of improvement also could have been unrelated to actual improvement in the skill but, instead, it may have related to having more information about the issues.The format of the field instructor questionnaire asked the respondent to evaluate her/his student's professional preparedness and skill level as the student began the concentration placement.The items mirrored the content of the student questionnaire.Respondents were directed to answer the questions in relation to their current student or, if providing concentration field instruction to more than one student, to responding in regard to the student whose name occurs first alphabetically.Because instructors were asked to rate incoming students who had just completed their foundation practicum, instructor-respondents were not placed in the (potentially biasing) position of having to rate the effectiveness of their own work with the student.There was no effort made to match student and instructor responses.
The surveys were mailed to students' home addresses and field instructors' agency addresses.Completed responses were returned by mail or hand-carried to a collection location in the MSW program office.Identification numbers on the surveys allowed an administrative staff member to follow-up with reminders and additional mailings to non-respondents.Data were entered into SPSS by a doctoral student and analyzed by one of the authors who served on the field practicum evaluation committee.
Responses were obtained from 76 field instructors (68% response rate) and 70 students (40% response rate; an additional 16 student responses were unusable because a second mailing contained the wrong instrument).Reflective of the MSW student population, the majority of students were enrolled in the clinical concentration (81%) and as full-time students (76%).The average number of years of prior work experience in the human services field for respondents was four, although 35% reported no prior human services work experience.Experience ranged from none to 29 years.Field instructors reported having provided concentration field instruction for six years on average, although 33% reported two years or less.Overall, experience ranged from being a first time concentration field instructor to providing this level of field instruction for 27 years.
Field instructors rated the degree to which their agency was able to provide students with experience dealing with clients or systems that represent a range of diversity with special reference to ethnicity, gender, culture, and sexual orientation.On average, the rating of this item was 4.6 on a scale from 1 = totally disagree to 5 = fully agree.Some 70% of respondents fully agreed that the agency was able to provide these experiences and 26% agreed to some extent, indicating that there would have been opportunities to observe the students' skills in culturally competent practice.
The analysis reported here focuses on two items directly related to diversity and preparation for culturally competent practice.Figure 2 details the items.

RESULTS
To determine whether students' skills increased during the foundation practicum, we compared the retrospective pre-test and post-test ratings using paired sample t-tests.T-tests for both items were significant (both items p<.0001) (see Table 2).
To determine whether students and instructors perceived similar levels of competence, we compared student post-test and instructor average ratings, using independent samples t-tests.Using the Bonferoni procedure to take into account multiple comparisons, the p-value for significance is .05divided by 2, the number of comparisons, or .025.The t-tests for both items indicate that instructors rate student abilities significantly lower than students (p=.021 and p=.005, respectively) (See Table 3).

Field Instructor Survey
At the time the student began his or her current placement at my agency...The student demonstrated awareness of and sensitivity to ethnic, minority status, and cultural issues in assessment and goal-setting for clients and client systems.
The student demonstrated awareness of the impact of gender, ethnic, minority, and cultural issues on her/his own interactions with clients and others.

MSW Concentration Student Survey
When I began my MSW II field placement...I was aware of and sensitive to ethnic, minority, and cultural issues in assessment and goal-setting for clients and client systems.I was aware of the impact of gender, ethnic, minority, and cultural issues on my own interactions with clients and others with whom I interact.

DISCUSSION AND IMPLICATIONS FOR SOCIAL WORK EDUCATION
This study is a first attempt at determining MSW student cultural competence in the field practicum.The study also goes beyond earlier evaluations in obtaining data from multiple respondents, although it should be noted that responses from students and field instructors may not match (i.e., concern the same student).
Consistent with previous studies of classroom education for cultural competence, students in this study report acquiring a significant level of knowledge and skills during the time of the first MSW field practicum.In this sense, the foundation field practicum objectives are being met.However, concentration field instructors, on average, reported their incoming students as having less developed cultural competence than the students reported.This apparent difference warrants further investigation.
Possible explanations for the difference noted here include instructors and students using different criteria for judging cultural competence, based on individual differences, experience, and/or point of reference.Although demographic data on race/ethnicity of students and instructors were not collected in the study from which the present data were drawn, the proportion of minority group members is higher among field instructors than among students.Furthermore, field instructors who have greater experience may have a different understanding of cultural competence.Ideally, this would be a better understanding, although the possibility exists that instructors with many years of practice might actually have a less sophisticated understanding, depending on how thoroughly they have accessed and utilized emerging new perspectives on cultural competence.Finally, students may actually have skills that instructors were not able to observe.Field agencies are under increasing pressure to maintain productivity, which may translate into less contact between instructors and students (Reisch & Jarman-Rohde, in press).Another possible explanation for the difference may lie in the way the questions were asked.Students were asked about their awareness and in knowing how to take into account cultural issues, while instructors were asked about students' ability to demonstrate their awareness and knowledge of how to take into account cultural issues.Perhaps we have failed to measure skills from the student perspective.If that is the case, then the appropriate interpretation of this difference might be that although students are aware of the issues, they are less able to act on that awareness than they think they are.
The students' perspectives, compared to that of the field instructors', may not represent disagreement about skill level, but rather, it may reflect the cultural competence developmental process.Students must have a great deal of awareness of cultural issues in assessment, intervention, and overall interactions with clients and others before they can put this knowledge into practice.As McPhatter (1997) put it, "Enlightened consciousness and a grounded knowledge base are the bricks and cement that build cumulative skill proficiency."(p.271, emphasis in original) Students reported that they have the necessary level of awareness.Instructors, on the other hand, reported a lower level of demonstrating use of this awareness and knowledge.
This study used data collected for another purpose, which presents both positive and negative issues.On the negative side are problems common to secondary analysis of other types of data, including the lack of control over what is asked and how questions are phrased.The existence of these data, however, is a positive step, as they evidence attempts being made to evaluate the effectiveness of social work education in producing cultural competence outcomes in graduates.One implication for social work education is that data such as these should be collected and evaluated on a regular basis and gathered from multiple perspectives.Although it was not practical in the program evaluation which provided the data for this study, attempts should also be made to obtain matching responses from students and field instructors.Finally, for program monitoring, data collection on the effectiveness of education for cultural competence should be an integral part of social work education and practice.For example, as a result of the program evaluation reported here, routine data collection instruments (in the form of student exit surveys, alumni surveys, and telephone surveys of employers of our graduates) were expanded to include items rating cultural and diversity competence.In addition, although items related to accomplishing culture and diversity learning objectives have been a regular part of the instrument used by practicum instructors to report evaluation of student learning in field instruction, items were revised to replace the original yes/no format with a Likert scale on each item so that changes over time may be tracked for quality improvement.Some important issues still need to be addressed in the ongoing research and evaluation of cultural competence outcomes.Beyond the issue of whose perspective is more accurate or what the various perspectives actually represent are the issues of how to measure skills and which skills are important to measure.If a practitioner were culturally competent, what would we see them do?The literature here is a bit vague and abstract.Is cultural competence a matter of attitude, awareness, understanding, experience, skill, or knowledge?Probably all of the above.Yet, the literature is not entirely consistent in the way that it deals with the cognitive/affective status of cultural competence.This study suggests the importance of distinguishing more clearly between attitudinal, conceptual, and skill components of practice for cultural competence and when measuring the effectiveness of our educational efforts.It also suggests the importance of focusing more attention on the practicum experience, where affect, attitude, experience, concept, and skill become integrated.Does education for cultural competence occur at the interpersonal level, the level of the agency, the community, social policy?Most authors define culturally competent practice largely in terms of interpersonal or direct practice with individuals, families, and groups.Some, however, like McPhatter (1997) and Sowers-Hoag and Sandau-Beckler (1996), suggest that culturally competent practice includes skills to intervene at every level necessary, identifying and removing barriers at the organizational, community, social, economic, and political levels and "correctly identifying and confronting issues of racism and discrimination."(McPhatter, 1997, p. 273).As helpful as the literature has been to enhancing our understanding of what it takes to educate culturally astute and diversity competent practitioners, clearly, much more needs to be done empirically, conceptually, and educationally to define (and develop professional consensus around) the concrete educational objectives and tasks that together operationalize the overarching goal of cultural competence.The ultimate aim, of course, is to provide students of the profession with the most effective learning experiences possible to meet this important educational/practice goal.

Table 1 :
Evaluations of Social Work Education for Practice with Diverse Populations

Table 2 :
Comparison of Retrospective Pre-Test and Post-Test Means

Table 3 :
Comparison of Student and Instructor Means