Identifying Trauma-Related and Mental Health Needs: The Implementation of Screening in California’s Child Welfare Systems

: A central aspect of trauma-informed care in child welfare (CW) systems is the use of a trauma-informed screening process. This includes the use of a broadly administered measurement approach to assist professionals in identifying current trauma-related symptomology or a history of potentially traumatizing events. With a high prevalence of unmet mental health needs among CW-involved children, screening can be a crucial step as systems strive to identify children impacted by trauma. This paper offers a summary of CW screening approaches in county-administered CW systems across California. Through a web-administered survey, 46 county administrators reported on their screening practices and perceptions. Information about ages of children screened and screening tools used, perceptions of screening implementation priorities, degree of implementation and satisfaction with screening processes is provided. Several implementation considerations for future trauma-informed care efforts are offered including maintaining a focus on childhood trauma, closing the science-practice gap, and evaluating the state of the science.

experiences across the lifespan (e.g., obesity, depression, heart disease, smoking, alcoholism; Felitti et al., 1998;Larkin, Felitti, & Anda, 2014), linking CW's trauma screening efforts with a thorough mental health (MH) assessment and trauma-focused interventions could contribute to meaningful public health benefits. As Ko and colleagues (2008) point out, "For the child welfare system to become increasingly trauma-informed, effective trauma screening and assessment protocols are needed at every level" (Ko et al.;p. 398).
In the U.S. general population, 61.8% of adolescents have been found to experience one or more potentially traumatic events (PTE), such as interpersonal violence, accidents, injuries, or witnessing others experience a PTE, with 32.7% experiencing 2 or more PTEs (McLaughlin et al., 2013). Almost all children who become involved in CW have been impacted by maltreatment, a specific subset of PTE (DHHS, 2016) and 61.8% of the children and youth involved in current investigations are estimated to have previous reports of maltreatment (Horwitz et al., 2012), indicating that the maltreatment has persisted over a period of time. Although experiencing PTE is fairly common in childhood, it is more common and more persistent for CW-involved children and youth, and children who experienced multiple PTEs are more likely to develop posttraumatic stress disorder (PTSD; McLaughlin et al., 2013).
CW systems are also well-positioned to identify children impacted by other forms of MH impairment. Almost one-quarter of adolescents (22.2%) in the general population have experienced a significant MH need during their lives (Merikangas et al., 2010). In contrast, approximately half (47.9%) of a nationally representative sample of children and youth investigated by CW services for child maltreatment were identified as having a significant MH need (e.g., emotional and behavioral problems; Burns et al., 2004), indicating a high rate of MH need among CW-involved children and youth.
Moreover, the second wave of the National Survey of Child and Adolescence Well-Being uncovered a stark underutilization of MH services among children and youth with significant MH need involved in CW services, with only one-third (33.3%) of these youth receiving MH services (Horwitz et al., 2012). A disproportionate underutilization of MH services has been well-established among children and youth of color with significant MH need who are involved in CW compared to their white counterparts (Garland et al., 2000;Garland, Landsverk, & Lau, 2003;Horwitz et al., 2012;Kim & Garcia, 2016). While underutilization of MH services is problematic in the general population (Merikangas et al., 2010), CW-involved children and youth, who are already connected to social services providers and might presumably have greater access to other MH services, are still much more likely to have unmet MH needs. This is particularly true for CW-involved children and youth of color.
With longstanding leadership from organizations like SAMHSA, NCTSN, and ACF, these findings have contributed to ongoing policy changes at federal, state, and local levels, resulting in widespread screening implementation efforts in CW systems across the nation. In California, similar to many other states, class action litigation has been spurred by advocacy groups on behalf of former CW-involved youth whose MH needs were unmet while involved in CW services (Kosanovich, Joseph, & Hasbargen, 2005 ] established the requirement for each of California's 58 county systems to implement screening procedures for identifying children and youth involved in CW with significant MH needs to help increase their access to MH assessment and treatment. Technical assistance and implementation materials, including monthly phone calls, a practice manual, and a review of screening tools (Crandal & Conradi, 2013), have been provided by the state through Pathways to Mental Health Services. State leaders have facilitated opportunities to support California's county leaders with the development of procedures to identify children and youth involved in CW whose well-being has been disrupted, particularly as a result of maltreatment, and ensure referral to assessment and treatment (see California Department of Health Services, 2017; California Department of Social Services, 2017 for more information).
Despite this new energy behind screening among children and youth involved with CW, there has also been considerable autonomy and some ambiguity for CW administrators and staff to implement the actual screening programs for the children and youth they serve, both in California and across the country. ACF has provided funding to technical assistance teams in multiple states across three cohorts to support state screening efforts by way of five-year grants (DHHS, ACF, 2011). However, in many CW systems, the mandates for screening led administrators to rapidly implement a screening process without piloting, technical assistance collaboration, or detailed planning. Questions remain regarding the constructs targeted by screening practices, the screening tools utilized, the ways tools are utilized, the ways screening practices are tied to intervention, how key decision points are prioritized, and how decision makers perceive their implementation efforts.
To date, there has been no systematic evaluation of how screening efforts have been implemented by CW leaders and staff. The California screening implementation under Pathways to Mental Health Services presents an opportunity to investigate how CW systems have responded to screening mandates and to offer new frameworks for other CW systems moving to implement similar screening processes. The current study provides an investigation into new screening implementation efforts in CW services by characterizing California's CW systems' screening approaches, offering examples of how this aspect of TIC has been implemented in practice, and evaluating the diverse attempts to implement and sustain screening from a trauma-informed framework.

Research Questions
 Following statewide implementation efforts, what are the characteristics of screening approaches used in county CW systems to identify trauma-related and MH needs?  In response to policy mandates accompanied by broad implementation support, what are CW leaders' perceptions of the selection and implementation of screening approaches for identifying trauma-related and MH needs?

Procedure
California's CW system is county-administered, with oversight provided by the state's DSS. The authors developed a web-administered survey to obtain information on MH and trauma screening practices in each county. The survey invitation was disseminated through an email from the state's association of human service directors to the association's representative in each county CW agency. The California DSS also endorsed the need for counties to complete the survey. The email recipient was asked to identify one respondent in their county with knowledge of screening practices for children involved with CW, from either the county CW or MH agency (based on the particular county system responsible for screening in that county) to complete the survey. In order to maximize the response rate, two reminder emails were sent by the state association. An in-person reminder was also provided at a standing monthly meeting attended by representatives from county CW agencies and staff from the California DSS. The survey was administered between February 24, 2016, and March 23, 2016.

Sample
The survey was completed by a respondent from 46 of the 58 counties in California. The breakout of respondents by position in their agency was as follows: director or chief, 13.0%; assistant or deputy director, 19.6%; manager, 47.8%; supervisor, 6.5%; and other, 13.0%. The participating counties represented 96.8% of the state's population according to 2010 census information (U.S. Census Bureau, 2015), and they represented 97.4% of the 46,261 CW case openings that occurred in all counties in the state between April 1, 2015 and March 31, 2016 (Webster et al., 2016). Based on the National Center for Health Statistics' 2013 Urban-Rural Classification Scheme for Counties (Ingram & Franco, 2014), the percentage of counties in the sample in each urban-rural category and the percentage of counties in that category statewide, shown in parentheses, were as follows: 17.4% were large central metro (13.8% of counties in the state), 15.2% were large fringe metro (13.8% of counties in the state), 23.9% were medium metro (22.4% of counties in the state), 13.0% were small metro (13.8% of counties in the state), 13.0% were micropolitan (13.8% of counties in the state), and 17.4% were noncore (22.4% of counties in the state).

Measure
The web-based survey contained questions on screening practices for identifying MH and trauma-related needs of children involved with the CW system. Respondents were asked to indicate the extent to which screening had been implemented in their county (e.g., full implementation, partial implementation, implementation being planned, or no plans for implementation), the degree of satisfaction with current screening procedures, and whether changes in screening procedures were being considered. The survey contained a list of tools commonly used to screen or assess MH and trauma-related symptoms (see Table 1), and respondents were asked to indicate which tools were being used or considered by their county. An open field was included so respondents could also report on tools that did not appear on the list. An additional question asked about the age groups of children being screened or being considered for screening (ranging from infants to transition-age youth). Finally, respondents were asked to rank order a list of 10 considerations (see Table  2) for determining a screening procedure/selecting screening tool(s) ranging from 1 (most important) to 10 (least important).

Screening Implementation Status
With regard to implementation status, 84.8% (n=39) of respondents reported that a screening procedure to identify MH and/or trauma-related needs of children involved with CW had been fully implemented in their county, 13.0% (n=6) indicated a screening procedure had been partially implemented, and 2.2% (n=1) reported that a screening procedure was being planned. Those who indicated that screening had been fully or partially implemented (n=45) were asked whether changes to screening were being considered (e.g., "We are thinking about changing our current screening tool(s)/procedures for MH and/or trauma-related needs") and about satisfaction with screening (e.g., "We are satisfied with our current screening tool(s)/procedures for MH and/or trauma-related needs"). Both questions were rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Out of the 45 participants who answered the question, 20% (n=9) agreed that they were thinking about changing current screening tool(s)/procedures, 26.7% (n=12) were not sure, 37.8% (n=17) disagreed, and 15.6% (n=7) strongly disagreed (M=2.51, SD=0.99). As for their satisfaction with current screening tool(s)/procedures, 20% (n=9) of respondents strongly agreed they were satisfied, 57.8% (n=26) agreed, 15.6% (n=7) were not sure, 4.4% (n=2) disagreed, and 2.2% (n=1) strongly disagreed (M=3.89, SD=0.86).

Screening Tools Being Used and Under Consideration
Of the 46 participants, 44 reported use of at least one of the tools listed in the survey or reported use of at least one other tool that was not listed to screen for MH and/or traumarelated needs. Information on reported use of each tool for these 44 participants is shown in Table 1. The list of tools was presented in alphabetical order in the survey, but in Table  1, the tools are grouped into categories: tools to assess MH symptoms completed by caregivers and/or youth; tools to assess trauma-related symptoms completed by caregivers and/or youth; tools completed by providers; and unknown. Approximately half of the participants (52.3%; n=23) reported use of provider-completed tools only, 40.9% (n=18) reported use of both provider-completed and caregiver-and/or youth-completed tools (MH and/or trauma), and 6.8% (n=3) reported use of caregiver-and/or youth-completed tools only (MH and/or trauma).
Almost a quarter (23.9%, n=11) of the 46 participants reported that at least one tool was being considered for use in their county to screen for MH and/or trauma-related needs. Information on tools being considered is also reported in Table 1. Of the 10 participants who answered the follow-up questions, 20.0% (n=2) were considering a providercompleted tool, 40.0% (n=4) were considering one or more caregiver-and/or youth-completed tools, and 40.0% (n=4) were considering both provider-completed and caregiver-and/or youth-completed tools. * number of respondents for these items=10 ** This row offers information on perception of use of SDM for screening purposes but is not representative of the proportion of counties who use SDM throughout the state for other purposes.

Considerations for Determining Screening Tool(s) or Procedures
Respondents' ranking of considerations for determining screening tool(s) or procedures are reported in Table 2. A list of 10 considerations was provided in the survey and the respondent was asked to place the items in order of importance. The item "It's evidence-based or supported by research" was ranked as most important by 56.5% (n=26) of respondents (Mdn=1.0), and the item "The tool(s) was designed to be completed by staff" was ranked as most important by 19.6% (n=9) of respondents (Mdn=3.0). Half of the items were ranked as most important by less than 10% of respondents, and three items were not ranked as most important by any of the respondents.

Discussion
This evaluation of the Pathways to Mental Health Services screening process implementation offers several findings germane to TIC. Most participants perceived their screening process as fully implemented and expressed satisfaction with the process. Generally, these processes included screening all or nearly all ages of children and youth served, most commonly with practitioner-rated tools that examine general child functioning, potential for risk, or family strengths and needs. Survey results also showed that a majority of respondents collectively viewed "evidence-based or supported by research" as the top consideration when determining a screening tool or process in their respective counties. Although these findings are specific to California's implementation of screening for CW-involved children, there are implementation considerations demonstrated here that can inform future attempts to broadly implement TIC.

TIC Implementation Consideration 1: Maintaining Focus on Childhood Trauma
After longstanding national emphasis on screening in CW as a crucial component of TIC, California's statewide screening implementation ultimately occurred as a result of class action litigation with an emphasis on access to MH services. A focus on MH services was intended to convey an inclusive array of services, targeting trauma-related as well as other MH needs a child may have. The California DSS offered a practice model and technical assistance which included emphasis on the importance and role of TIC in this implementation effort. Disconcertingly, however, an open-ended focus on broad MH may have diluted the salience of identifying PTSD-and trauma-related needs for the CWinvolved population at the county level, as evidenced by the 36 counties (78.3%) whose respondents reported no trauma-related content or results included in their screening tool approach.
For more than a decade, TIC researchers have highlighted the importance of linking trauma-related needs with research-based trauma treatment services (Kerns et al., 2016;Ko et al., 2008;Taylor, Wilson, & Igelman, 2006), rather than general MH services (Leslie, Hurlburt, Landsverk, Barth, & Slymen, 2004). Advocates of TIC need to ensure the mission and meaning of TIC is directly translated into specific policies and implementation practices in future efforts, particularly within large multilayered systems where aspirational aspects of an effort may become attenuated across layers.

TIC Implementation Consideration 2: Closing the Science-Practice Gap
Of the four most commonly used tools used for screening reported in this survey, three are completed by professionals involved in the case, rather than by direct report from the child or caregiver. While there are examples of professional-or practitioner-completed measurement tools performing well and providing meaningful data, they are generally less represented in measurement literature (Crandal et al., 2015). This may, in part, be due to the risk for measurement error because the targeted construct is distanced from the measurement source. For example, measuring a child's trauma history or symptoms from the point of view of a professional produces a score that represents the professional's knowledge of the child's history or symptoms, which can vary widely based on each professional's interactions with the child, familiarity with the child's case, understanding of screening procedures, or skill investigating childhood trauma. High caseloads, overburdened workers, and strained organizations or systems may exacerbate these potential sources for systematic measurement error (Podsakoff, MacKenzie, Lee, & Podsakoff, 2003).
Since the psychometric performance of the practitioner-completed measurement method is not clearly established, particularly in applied settings, there is cause for caution when interpreting results from professional-rated screening tools. Additionally, literature review of peer-reviewed published research on several of the most commonly reported tools used for screening in the respondent counties suggested these tools were not developed for use as MH or trauma screening tools and have not been evaluated for performance when used to assist professionals making screening decisions (i.e., cut scores, sensitivity, specificity, predictive power; Crandal & Conradi, 2013;Haynes, Smith, & Hunsley, 2011).
Given these measurement concerns, the commonly implemented screening tools are often those with uncertain performance. Nonetheless, a majority of survey respondents ranked "It is evidence-based or supported by research" as the highest priority consideration when determining screening tools or procedures. This illustration of a science-practice gap in CW has been noted by others (Burns et al., 2004;Raghavan, Inoue, Ettner, Hamilton, & Landsverk, 2010). In his study of the use of research evidence in public youth-serving systems, Palinkas (2015) found that systems leaders tend to rely on the availability of resources, local indications of a need, and their personal experience when deciding if they will use evidence to make changes. CW leaders were more likely than other system leaders to ignore evidence when making changes (Palinkas, 2015). Emerging literature on addressing the science-practice gap in CW calls for research-practice partnerships that build on cumulative and collaborative efforts to close the science-practice gap (Aarons & Chaffin, 2013;Palinkas, 2015;Palinkas, Short, & Wong, 2015;Testa et al., 2014). Future TIC implementation efforts in CW should address the science-practice gap beginning with an awareness of the need for collaboration among leaders with diverse science-and practice-related expertise.

TIC Implementation Consideration 3: Evaluating the State of the Science
While there are research-practice partnership and implementation models for expanding access to evidence-based practices for child-welfare-involved children Testa et al., 2014), these models have focused on increasing access to established evidence-based treatments. Evidence-based screening for CW-involved youth, on the other hand, is still taking shape. Evidence-based screening is commonly misrepresented as merely the use of tools that have produced data that was found to have strong reliability and validity, sometimes without regard for screening-specific test performance (e.g., sensitivity, specificity, predictive power).
There are measurement tools which have performed well when screening for general MH needs (Gardner, Lucas, Kolko, & Campo, 2007;Goodman, Ford, Simmons, Gatward, & Meltzer, 2000), as well as screening tools developed to detect trauma-related need or assess histories of childhood traumatic stress (Conradi, Wherry, & Kisiel, 2011;Crandal & Conradi, 2013). However, only recently have these tools been evaluated in broad CW contexts to identify children who might benefit from further MH assessment and treatment (Kerns et al., 2016). Nonetheless, "screening should involve a system, not just a test," (Raffle & Gray, 2007, p. 42) and a screening system that is evidence-based includes evaluation of the screening test performance as well as the outcomes for individuals involved in the process.
To date, TIC advocates do not have sufficient information about screening programs to determine how many children with positive screens are found to have serious MH conditions and benefit from treatment, how many children with positive screens are found to have no serious MH condition or do not benefit from treatment, how many children with negative screens later demonstrate significant MH need, or which services might be appropriate for children with negative screen results. Researchers, policy makers, administrators, and other stakeholders are embarking on a nascent application of a longstanding element of TIC. The barriers identified in the Pathways to Mental Health Services screening implementation process bring to light the importance of acceptability and feasibility in existing evidence-based screening models. Meaningful steps remain to understand and implement this crucial component of identifying children whose well-being has been disrupted by traumatic events and develop evidence-based screening processes that are feasible to implement in CW systems.

Limitations
The current study offers a point in time glimpse of an on-going, expansive implementation effort in a large state. There are a number of factors that have facilitated effective and ineffective aspects of implementation at the national, state, county, and local levels that are beyond the scope of this project. Furthermore, although attempts were made to identify the most appropriate respondent, data were collected based on report from a single representative from each county. Accuracy of the reported information was not assessed and the extent to which social desirability influenced responses is unknown. County screening programs are likely to involve multiple stakeholders with a range of perspectives and there may be additional information that was not captured based on the methodology used in this study. Respondents from different counties may have different perceptions of screening and the distinction of screening from assessment, which could contribute to imprecision in the survey responses. Finally, given the large countyadministered organization of California's CW services, these findings may not generalize to all other CW settings.

Future Research
Future attempts to explore broad screening implementation could include evaluations of fidelity and sustainment of the implementation effort (e.g., training and professional development among direct service staff, quality assurance monitoring, evaluating perceptions of the practice among staff providing the service) as well as evaluation of the effectiveness of the implemented TIC practices. Additionally, TIC implementation models would be useful to support CW organizations looking to implement specific practices. For example, Kerns and colleagues (2016) describe an approach to strengthening the CW workforce capacity to identify trauma symptoms for children and youth involved in services. This articles describes different screening approaches that were implemented or under consideration for implementation and leadership perceptions of those approaches in the context of a statewide effort. An implementation model, on the other hand, would examine the lessons learned from researchers and practitioners and concretize an implementation approach designed to be translated to different CW organizations. While further steps are needed to make trauma-related and mental health screening more accessible and practical for CW systems, this study offers information and considerations intended to encourage the progress of this crucial element of TIC.