Evaluating Community-Engaged Research in Promotion and Tenure

To advance and encourage partnerships between institutions and their greater communities, academic reward structures must be designed in ways that support those who choose to leverage their expertise, resources, and time to engage with community in meaningful and mutually beneficial ways. This study investigates how school-and department-level promotion and tenure committees define, understand, and evaluate faculty’s engaged research. Specifically, this study explored what goes into making evaluative decisions and how evaluative decisions are made (e.g., how review committees define and categorize faculty’s engaged research, what metrics are used to assess it). In this single case multi-site qualitative study 12 participants across five R1 institutions classified as engaged by the Carnegie Foundation participated in semi-structured interviews. All participants were tenured, engaged scholars with experience serving on a school-and/or department-level promotion and tenure review committee. Findings demonstrate that review committees struggle to define, categorize, and evaluate community engaged research in promotion and tenure, as they are forced to exclusively rely on a traditional set of metrics to evaluate the engaged work of their peers. Though universities are making strides to institutionalize engagement, appropriate recognition of engaged research within promotion and tenure is not yet a reality.


Introduction and Framing
Traditional academic reward structures must be designed to support those who choose to leverage their expertise, resources, and time to engage community.Research suggests that even though more institutions are actively working to cultivate an engaged campus identity, institution-level rhetoric praising community engagement and the rewarding of engaged faculty through promotion and tenure can be inconsistent (Diamond, 2005;O'Meara, 2002;Saltmarsh, Giles, et al., 2009;Sobrero & Jayaratne, 2014).The perceived misalignment between institutional rhetoric and rewarding engaged faculty through promotion and tenure is especially problematic when institutions seek external recognition and are heralded as engaged campuses by external entities, such as the Carnegie Foundation.As campuses work toward infusing community engagement into their institutional missions and strategic plans and are acknowledged for doing so, there is a need for research that explores this suggested dissonance between institution-level praise for engagement and how engaged faculty are rewarded through promotion and tenure.
However, appropriately rewarding engaged faculty should not be left solely to institution-level leadership.It is well documented that the values, beliefs, and personal experiences of school-and department-level promotion and tenure committees influence their likelihood to reward and promote faculty who pursue engaged research (Diamond, 2005;O'Meara, 2002;Sobrero & Jayaratne, 2014).Studies show that changes to institution-level promotion and tenure guidelines reflecting an increased acceptance of community-engaged research do not necessarily ensure a similar acceptance of such research in school-and department-level guidelines (Alperin et al., 2018;Saltmarsh et al., 2009).Though school-and department-level reward processes are undoubtedly influenced by written guidelines and committee members' values and beliefs, there is a gap in the literature exploring the evaluative processes school-and department-level promotion and tenure committees undertake when evaluating faculty's engaged research or how evaluative judgments are made.
Multiple resources (Able & Williams, 2019;Jordan et al., 2009;Wood et al., 2018) have been created to assist in evaluating faculty's community-engaged research.However, there is a lack of knowledge regarding whether or how such resources are being used.Community-engaged research often operates in historically non-traditional ways.It includes community members as co-researchers, seeks to produce additional scholarly products outside of peer-reviewed publications, and often favors local impact over national recognition.It cannot be evaluated in the same ways as traditional research (Boyer, 1990;Deetz, 2008;Ellison & Eatman, 2008;Zukoski & Luluquisen, 2002).As such, there is a need for a better understanding of how promotion and tenure committees at the school and department levels make evaluative decisions regarding faculty's community-engaged research.
The growing tension arising from the misalignment of rewarding engaged faculty is further compounded by changing faculty demographics.Research has shown that large percentages of baby boomer faculty are beginning to retire and are being replaced by more diverse groups of faculty who are more likely to pursue an engaged scholarly agenda, believing that they have a responsibility to apply their knowledge toward advancing community-identified needs and issues, even when it often runs counter to the institutional norms (Holland;1997;O'Meara et al., 2011;Saltmarsh & Wooding, 2016).These incoming faculty are increasingly more female and racially and ethnically diverse (Finkelstein, 2010;Finkelstein et al., 2016).Further, studies have shown that faculty of color are more likely than their White counterparts.Females are more likely than males to pursue an engaged research agenda, provide services for local communities, and leverage their scholarship to promote community-based change and equity (Colbeck & Wharton-Michael, 2006;O'Meara, 2002;O'Meara et al., 2011;Vogelgesang et al., 2010).It is clear that the current institutional systems of promotion and tenure disproportionally affect scholars of color and other minoritized identities.
This article shares findings from a multi-campus study that was guided by the following research questions: 1. How do school-and department-level promotion and tenure committee members evaluate faculty's community-engaged research?2. What guidelines, tools, and/or processes, or lack thereof, guide school and departmentlevel promotion and tenure committee members' evaluation of faculty communityengaged research (e.g., school/department-level guidelines and language, institution-level guidelines and language, peer review/letters, rubrics, other tools, etc.)? 3. How are community-engaged research processes and community-engaged research products (community-engaged scholarship) evaluated by school and department-level promotion and tenure committees? 4. How do school and department-level promotion and tenure committees differentiate community-engaged research processes (e.g., co-creation of study design, research questions) and products (community-engaged scholarship) when evaluating the engaged work of faculty?

Exploratory Qualitative Study
Due to the nature of this research, the desire to build upon prior scholarship, and the complex phenomenon of evaluation within promotion and tenure, an exploratory, qualitative study focusing on evaluating community-engaged research within promotion and tenure, was identified as the most appropriate approach to investigate the research questions.This IRB-approved study was intentionally bounded in terms of the identified phenomenon (evaluation of community-engaged research), sites (institutions), and participants (faculty who have served on review committees).Binding the case in this way was appropriate due to the exploratory nature of this research.Though the study is bounded in the aforementioned ways, it is important to acknowledge that school-and department-level committees are influenced and affected by a handful of other entities -including, but not limited to, institutional missions and guidelines, school, department, and institutional cultures, and external organizations and/or associations.

Institutions
Institutions for this study were first required to have received an initial classification or reclassification for community engagement from the Carnegie Foundation in 2020 classification cycle (N = 119).Narrowing the site selection to 2020, Carnegie classified institutions only involved institutions identified as the most advanced in institutionalizing community engagement across their campuses.The scope of this study was further narrowed by including only R1 institutions (N = 28).R1 institutions were intentionally chosen due to their significant emphasis on traditional research instead of teaching or academic service.Due to their heavy focus on research outputs, there is inherently more tension at R1 institutions to accept and place community-engaged research on par with traditional research.This study included five of the 28 R1, 2020 Carnegie classified institutions.The five participating institutions were included and requested to participate because professional relationships with Community Engagement Professionals (CEPs) at those institutions previously existed, significantly aiding in recruiting individual participants.Notable characteristics of each of the five participating institutions are identified in Table 1.

Participants
Study participants were recruited from the identified institutions with the help of CEPs on each campus.Acknowledging the need to narrow the participant selection in ways that allowed for a detailed exploration of the research questions, the following participant inclusion criteria were established: 1. Are they a tenured faculty member currently serving on their school and/or departmentlevel promotion and tenure review committee, or have they served on their school and/or department-level promotion and tenure review committee within the past 12 months?2. Do they have a primary appointment in either a social science field or STEM field (e.g., Science, Technology, Engineering, Math)? 3. Do they have some familiarity with community-engaged research as an approach to inquiry?
Table 2 outlines the key characteristics of each participant.

Data Sources: Participant Interviews
Individual participant interviews were the primary source of data.The interviews were semi-structured, which lasted roughly 60 minutes each and was conducted via Zoom.Each participant was assigned a pseudonym using a random name generator.Interview questions were constructed to address the central research questions and incorporated a series of structured, neutral probes to elicit additional information about the participants' experiences (Berg, 1995).
The interview protocol included ten major questions that were categorized into three specific phases: Phase One: Building Understanding, Phase Two: Evaluating Community-Engaged Research -Processes and Products, and Phase Three: Looking Forward.

Data Analysis
Data analysis of participant interviews consisted of the following phases: 1. Transcription of participant interviews: Transcripts were autogenerated by Zoom and reviewed for accuracy and completion by the researcher.2. Data exploration, review, and memoing: This phase included a review of all transcribed data from a holistic perspective to understand the breadth and scope of all data within single participants, single institutions, and across multiple institutions.3. Open coding and the development of raw codes: Open coding, or the development of raw codes to illustrate the major categories of information identified within the data, occurred after, and was influenced by, the more general data exploration and memoing phase (Creswell, 2007;Strauss & Corbin, 1990).4. Iterative, axial coding assisted by participant member checks: Focused axial coding involved the creation of additional codes and subcodes concentrated on specific ideas and concepts, which allowed for more in-depth theorizing about the original concepts (Strauss & Corbin, 1990).Identified themes were emailed to all study participants for feedback.Participant feedback was considered and influenced the next phase of data analysis.5. Selective coding, data reduction, and development of themes: This selective coding was more conceptual than the previous process of line-by-line coding and identified codes that frequently appeared throughout the data (Stake, 2010;Strauss & Corbin, 1990).6. Examine the data in light of current literature.

Ensuring Trustworthiness
During each interview, the researcher created notes, or memos, about each conversation, detailing thoughts, insights, and initial interpretations of the discussion with each participant.The exploration, review, and memoing phase of data analysis included a holistic review of all memos to understand the breadth and scope of the data to help ensure trustworthiness (Creswell, 2007).Additionally, member checks were employed as a systematic way to solicit feedback from study participants about collected data and initial conclusions to help improve the accuracy and credibility of the study (Lincoln & Guba, 1985;Merriam, 1998;Yanow, 2006).Member checks included taking initial conclusions, themes, and analysis back to participants via email to test for factual and interpretative accuracy and provide evidence of credibility (Lincoln & Guba, 1985).The initial member check highlighted issues needing further revision, definition, or expansion that required an ongoing analysis cycle and edits (Lincoln & Guba, 1985).
In addition to member checks, trustworthiness was sought by the use of thick descriptions to convey the study's findings.Geertz (1993) and Creswell and Creswell (2018, p. 196) encouraged the use of thick descriptions within qualitative research to "transport readers to the [study's] setting and give the discussion an element of shared experiences."Participant voice, via direct quotations, was woven into the acts of both data analysis and presentation, seeking to provide a full and revealing picture of the participants' individual experience and context.Further, institutional context (presented in Table 1) and participant characteristics (presented in Table 2) aid to ground findings in the unique environments and experiences of the study's participants.
Lastly, the triangulation of data, the examination of data across multiple institutional sites and participants, was used as a strategy to ensure trustworthiness and build coherent and justified themes (Creswell & Creswell, 2018;Merriam, 1998).Conclusions were drawn using multiple participant perspectives on the same issue rather than relying on a single participant voice (Denzin, 1978).Triangulation of data across multiple institutions and participants helped to refine the conceptual linkages within all data and strengthen the study's ultimate findings (Goetz & LeCompte, 1984).Data triangulation also helped ensure data saturation (Denzin & Lincoln, 1994).

Findings and Discussion
As evidenced by the experiences and voices of study participants, there is a growing desire among today's faculty to carry out an engaged research agenda that directly benefits their local communities, includes the voices and perspectives of community members, and responds directly to community-identified needs.However, study participants shared that there are many barriers standing in the way of appropriately evaluating their peers' engaged scholarship in promotion and tenure: 1.No articulated definition of community-engaged research or scholarship within school-or department-level promotion and tenure guidelines 2. Absence of "community-engaged research" or similar terminology within school and department guidelines 3. Narrow conception of research and scholarship that excludes community-engaged scholarship and incorrectly categorizes it as service 4. Reliance on traditional metrics to judge the quality of engaged scholarship 5. Inability to evaluate quality research processes and reliance on general quantitative metrics to judge the quality of research 6.Lack of support to assist committees understand and evaluate engaged scholarship (e.g., guidelines, definition sheets) 7. Institutional, school, and departmental culture that publicly praises community engagement but does not internally support engaged faculty Of all barriers cited, those that were identified as the most imposing on the participants' inability to effectively evaluate their engaged peers effectively were the narrow conception of scholarship and review committees' total reliance on traditional metrics to evaluate engaged scholarship leading to their reliance on general quantitative metrics to judge its quality.Out of all identified barriers (themes), all participants most frequently articulated these, often within the interview.The below sections elaborate on these three themes in more detail.

Engaged Scholarship is Incorrectly Categorized and Evaluated as Service
Across all participants and institutional sites, it is clear that engaged scholarship is almost always seen as something nice to have, supplemental addition to a faculty's dossier.While faculty are often given a pat on the back for engaging community in their research, the scholarly products that come from engaged research are rarely, if ever, identified by review committees as scholarly.This study identified that when it comes to evaluating faculty's engage scholarship, it is routinely understood by review committees as service and not valued highly, if at all: A community-based dissemination doesn't count.It would be counted as service.If it is not a peer-reviewed publication it is automatically seen as a second-tier product and is not considered scholarship.(Joyce, Social Science) Even looking at some of the creative products -art shows, videos… I've been part of research projects where that has been a primary output.Where there was a summit at the end, where we pull all the key stakeholders in the community together and had various presentations and arts-based research exhibits that came out of the research.But none of that gets accounted for.But yet, in some ways, those are almost more powerful as far as outputs and contributions to the … research field.When it generates new conversations that could have never been had otherwise.(Andrea, Social Science) Participants stressed that the metrics used to evaluate faculty engagement when placed in the service category are extremely weak and are given very little weight.Such metrics rely almost exclusively on simple lists and counts of events or community members involved.This aligns with O'Meara's (2005) previous exploration of promotion and tenure documents and faculty handbooks, finding that when evaluating faculty service, the quantity of engagement is considered, not its quality or impact on the non-academic community.The sheer involvement of external community partners in a research project, beyond their involvement solely as research subjects, instantly triggers committees to understand and label such work as service and not research.The depth of community partnerships was not shown to be a factor in the categorization or evaluation of faculty work by committees.Regardless of if faculty worked with their community partners to identify research goals or co-present findings in a community space, such involvement with non-academic peers instantly relegates the work as service and diminishes it.Though most participants did not personally agree with the mislabeling of engaged scholarship as service, they felt powerless to categorize and evaluate it as anything else due to the current structure of their department and school guidelines, policies, and procedures.As engaged faculty themselves, participants were incredibly frustrated with their inability to categorize or label the scholarly products of engaged faculty more appropriately.This disconnect and frustration led to many participants' perceived lack of agency in the review process.
One major finding from this study emphasizes that for review committees to value and reward faculty's engaged scholarship effectively, it must first be recognized within promotion and tenure guidelines as research.Without clear language legitimizing community-engaged research and its scholarly products as research, committees are powerless to value it as anything but service.However, this identified issue is further compounded when considered in light of Holland's (2005;2009) and O'Meara, Eatman, & Petersen's (2015) prior research, which found the most common approach to revising reward structures and guidelines was to make room for engaged scholarship in the service or outreach category, or encourage an "expanded view of service, administration, and outreach" (Doberneck, 2016)

Peer-reviewed publications
Recognized as the "gold standard" and only acceptable outlet for disseminating scholarly work.It is not inclusive of community-based dissemination outlets or other scholarship (e.g., community presentations, laws/public policy, delivery of products or services).

Funding
Only national funding is recognized and valued.Local/regional funding is not acknowledged as legitimate or valuable.

Reputation
Faculty reputations and accomplishments with local partners are not considered or valued.Only a faculty member's national/international reputation and reach are considered.

Impact
The impact is measured solely by journal impact factors.Community engagement journals typically have lower impact factors.Local/regional or community-based impact is not acknowledged.

External letters
Only the opinions of other academics hold weight.Community members are not seen as peers and are deemed unable to speak to the work of faculty appropriately.
Participants lamented that anytime the word research was used as a way to define faculty work, it automatically requires very specific and tightly defined outputs on which it must be evaluated: As soon as there's an "R" attached to it for research, then the committee is going to be looking at the standard metrics.They're going to be looking at the number of grants you've had, and the size, the number of publications and the impact factor the journals in which you're contributing, and your growing national and international reputation, which is more difficult to do in a community-engaged context.(Louis, STEM) Participants across all disciplines echoed this lose-lose situation.As Felix, Social Science, shared, "Community-engaged research is considered research, yes.It's research.But it still falls under the umbrella of research, and they look at it by, what's your return on investment on that research."The rigidity of traditional metrics to evaluate research within promotion and tenure is an issue felt across all participants and institution types at institutional and school/department levels.Participants shared that their department-and school-level guidelines were all quite detailed in outlining specific traditional scholarly outputs (e.g., peer-reviewed journal articles) required of any faculty work labeled research: Some of our promotion and tenure guidelines explicitly say, 'You will publish two research journal publication articles per year in order to earn the next step.'I've never seen a P and T guideline that says, 'You need to do so many workshops.'Or 'You need to do so many community-level events.'Those just don't count.(Douglas, STEM)

Traditional Metrics Lead to Reliance on General Quantitative Metrics
The evaluative metrics in Table 3 were identified by all participants, regardless of discipline or institution.Thomas, Social Science, shared: When it comes to promotion and tenure, it is very traditional criteria that completely dominate.How many publications and in which journals?And it's not that people reject community-engaged scholarship.I mean, that's nice.But that's not what people focused on in those kinds of decisions.Jerry, STEM, elaborated, acknowledging that their department and school lack the tools to evaluate community-engaged scholarship through a non-traditional lens effectively: I don't think we've reached enough shared understanding among lots of stakeholders here regarding either the timelines or even the full range of criteria that should and can be used to evaluate how effectively people are doing community-engaged research.Additionally, many participants noted that when evaluating faculty's scholarship, committees often do not have enough time to dive fully into understanding the type of scholarship, even if they wanted to.Instead, they must quickly identify the quality and quantity of faculty scholarship, which are usually reduced to a specific set of easily quantifiable metrics.All participants acknowledged that these somewhat superficial metrics are unable to fully account for the quality of non-traditional scholarship and put engaged faculty at a disadvantage.However, some participants noted that a short list of easily quantifiable, standardized metrics allows for a more efficient and less subjective evaluation process overall, regardless of what type of scholarship faculty produce: You can count peer-reviewed journal articles.It's very objective.If somebody says, 'Well, I made this video and it changed the way we operate in a pandemic.'…So what.
And then you have to say what is the 'so what.'And then it's up to a bunch of people to subjectively judge whether they buy your 'so what'... That's part of the fear that I understand for committees.(Julie, Social Science) Though a standardized set of quantifiable metrics may take away some of the subjectivity, bias, and ambiguity for committee members, participants noted that this approach usually always leads to "bean counting" and focuses more on the quantity of traditional scholarship as opposed to the quality and overall impact of one's work: I think at the college level… I just know from serving on, that there are some bean counters in there that are going to count up, 'They did this number of community presentations and they did this number of peer-reviewed journal articles.Oh, and these five at an impact factor of at least three.'You know, they're going to be like that… We have to be aware that there are people that are going to be counting... Sadly, there are going to be people that are going to count stuff.So, we have to be aware that.Playing the game is part of this whole deal.(Phillip, STEM) Further, many participants shared that, in their experience, when a number is assigned to something, it is inherently valued more -what is counted, counts.Not providing a clear way to evaluate or count engaged scholarship is inherently devalued within promotion and tenure.While most participants acknowledge, this devaluing is also felt by junior faculty.In line with Ellison and Eatman's (2008) study, a few participants shared that they know of candidates within their departments and schools who are nervous even to include engaged scholarship within their dossiers for fear that it could be seen as illegitimate or childish: It's this tension behind, 'I know what I have to hit… and I did something really cool, but I don't know how the committee is going to judge it.'The committee might just think it's something that kindergarteners can do.(Julie, Social Science) Regardless of the intention of a standardized, quantifiable set of metrics, when numbers are assigned to scholarly outputs and counted, those outputs are inherently more valued.By not considering the depth and nuance of engaged scholarship and instead holding it to metrics created to evaluate traditional scholarship, engaged faculty are put at a clear disadvantage.

Reliance on Traditional Metrics Leads to the Devaluation of Community-Engaged Scholarship
Due to the fact that the measures used to evaluate traditional research are too narrow in their focus to judge quality engaged scholarship, it is engaged faculty who are pushed to change their focus, methods, and products, rather than promotion and tenure committees adopting additional evaluation measures.As noted by almost all participants, it is extremely difficult for engaged faculty to morph their engaged scholarship in ways consistent with the tenants of communityengaged research and aligned with the traditional metrics used by committees to evaluate scholarship.Thus, applying traditional measures to evaluate engaged scholarship often deters faculty from pursuing engaged scholarship entirely or pushes them to carry out their work in a way that hinders the collaboration, insights, and needs of community -fundamental components of engaged research.
For faculty who pursue an engaged research agenda, presenting their engaged scholarship in a way that adheres to traditional metrics is much like attempting to fit a square peg into a round hole.Any attempt of a candidate to document their engaged scholarship in non-traditional ways is extremely risky and jeopardizes their chance of promotion and tenure.All participants echoed this sentiment: I think where I see some of the problem areas in terms of even just trying to figure out how to evaluate engagement, is because we oftentimes are traditionally looking at published articles or grant dollars that are coming in.We can frame it around engagement as long as there's publications and grant dollars that are coming in with it.
But if there's alternative ways of output and products, it gets a little bit tricky to try to figure out if that's valuable… When faculty start having alternative ways to demonstrate impact or to share their research, there's kind of that cultural transition that hasn't quite taken effect yet, in terms of being able to see how we evaluate, or how can we effectively accept that type of work in place of the traditional articles, publications, research output, that individuals are easily measured.(Andrea, Social Science) As all participants have an engaged research agenda, many voiced frustrations with how their non-engaged colleagues generally misunderstand community-engaged research and its nontraditional processes.The consensus of participants was that their non-engaged peers view community-engaged research to be not as rigorous as traditional research, even though major investment in engaged research by funders in specific disciplines (e.g., medicine, public health, education) illustrates its increased rigor and relevance (Balazs & Morello-Frosch, 2013).
Acknowledging that their colleague's misinformed beliefs are primarily the product of a misalignment between traditional metrics and non-traditional, engaged processes and scholarship, Felix, Social Science, lamented: People think you're making an excuse for not being as productive as other people.And part of me is like, 'I don't have a minute in the day that I'm not spending on something related to [engagement].'So, it's not that I'm not being productive, but it's not in the metrics that you're focused on.
The use of these traditional metrics to evaluate all forms of faculty scholarship is highly problematic.Suppose engaged faculty can clear the first hurdle of categorizing their engaged scholarship as research and not service.In that case, they are then met with a much higher hurdle to overcome -the evaluation of their engaged scholarship with traditional metrics.As local dissemination in non-traditional outlets for non-academic audiences does not easily fit within the current evaluative structures, faculty's engaged scholarship is misunderstood and often dismissed by promotion and tenure committees.This study supports previous research by Sobrero and Jayaratne (2014), which found that department heads are unlikely to promote or tenure faculty who pursue non-traditional avenues of scholarship and dissemination because they find it difficult to understand and evaluate.Because engaged scholarship cannot be appropriately understood or evaluated by traditional metrics, it is often not considered research and either (a) not believed to be relevant to decisions for promotion and tenure, (b) labeled as "nice to have" and not factored into decisions of merit, or (c) recategorized as service and heavily devalued.

Recommendations and Next Steps
As defined by the Carnegie Foundation, community engagement cannot be fully institutionalized if a campus lacks clearly defined pathways to reward, promote, and tenure faculty who conduct engaged research.Previous research suggests that the Carnegie Foundation's community engagement classification has "enhanced both the prominence and promise of community engagement in higher education" (Driscoll, 2008, p. 41) and contends that there is "clear evidence of institutional change as a result of applying for the classification" (Noel & Earwicker, 2015, p. 55).However, findings from this research suggest that for the R1 institutions included in this study, who have recently received the initial classification or have just been reclassified, the label of an "engaged" campus, there is still much to be desired when it comes to rewarding engaged faculty through the traditional structures of promotion and tenure.This points to a dangerous gap between the public recognition of engaged institutions and the policies and structures leveraged in practice.
When considering how to best chip away at and open the current structures of promotion and tenure, this study provides four clear policy and infrastructure-based recommendations for interested institutions and their leadership to consider, as illustrated in Figure 1.
Though classifications, rankings, and public image might open the door and entice an institution to commit to establishing a culture of engagement (Step 1), this is not the final step.Further, creating offices, centers, institutes, and/or committees to lead the work, often in silos, is also a step (Step 2).To appropriately evaluate and reward faculty for their engaged research and the scholarship it produces, change to department-, school-, and institution-level guidelines is required to expand the definition of scholarship and what "counts" as scholarly work (Step 3).However, changes to written institution-, school-, and department-level guidelines are not the final step.
Most importantly, establishing referenceable metrics upon which review committees may properly evaluate the work of their engaged peers is essential if real change is to be made (Step 4).Including community-engaged research and/or scholarship in promotion and tenure guidelines is useless if review committees continue to evaluate it with traditional metrics.It is important to note that while the alignment of departmental policies and metrics is critical (Step 4), there are additional supporting reforms within departmental structures that likely will need to be considered prior to changes to evaluative metrics (e.g., expanding who can be a peer/external reviewer, reconsidering the definition of impact).Such cultural considerations that will likely live within departmental units are further explored in Table 4. Though the order of steps is suggested to build on each other, they likely will be done concurrently and inform each other.Though change at the institution-level to build structures and supports, broaden the definition of scholarship, and create guidelines and referenceable metrics upon which to evaluate engaged scholarship is much needed, change must simultaneously occur at the school-and departmentlevels to be sustained.There is a great need for schools, departments, and their leaders to cultivate a culture that values engagement within their units, both in word and in practice.
Though such a culture cannot be created overnight, it is essential and must occur in addition to any changes made at the institutional level.Academic disciplines and department cultures are known to heavily influence the type of scholarship faculty conduct (Buzinski et al., 2013;Doberneck & Schweitzer, 2012;Sobrero & Jayaratne, 2014;Wade & Demb, 2009;Ward, 2005).
Even with structures, policies, and procedures in place, as O'Meara (2002) identified, "unconscious values and beliefs held by faculty facilitating the reward system can prevent newer forms of scholarly work from being accepted and rewarded" (p.76-77).In conjunction with changes to structures and policies, a culture accepting the engaged work of faculty must be created.
Further, valuing the engaged research of faculty is an important issue of equity.The suggested recommendations are only made more urgent and necessary when considering this need through the lens of faculty diversity, equity, and inclusion.When considering the current structures of promotion and tenure and the type of faculty work that is valued and that which is not, failing to recognize and reward the engaged work of faculty has clear implications for faculty of color, women, and other marginalized groups who are often more engaged than their White male peers (Finkelstein, 2010;Finkelstein et al., 2016, Settles et al., 2022).Suppose institutions of higher education do not explicitly reward and value the engagement of its faculty in local/regional communities working to address community-identified issues, often through a social justice lens.
In that case, institutions are inherently pushing forward an agenda of systemic inequality.As detailed by participants in this study, engaged faculty are often required to forgo an engaged research agenda, pursue a non-tenure track appointment, or produce two lines of scholarship to achieve promotion and tenure.How scholarship is currently valued within promotion and tenure disproportionately affects faculty of color, women, and other marginalized groups.In today's climate, higher education is at a point of reckoning.With the upheaval caused by the COVID-19 pandemic and nationwide protests for racial justice, institutions are poised to reorient and reimagine the ways they leverage and reward the engaged work of their faculty.

Define and Categorize Community-Engaged Research as Research
Creating a culture and establishing policies, procedures, and guidelines to support and fuel the developing culture go hand in hand.One cannot be done without the other.When it comes to actions that can be taken by school and department leaders, this study suggests that the first step must be the expansion of what "counts" or what is defined as scholarship.Before appropriate metrics can be created to evaluate engaged scholarship, community-engaged research must be identified and defined as research, and not service, within the formal school and department promotion and tenure guidelines.Further, the products of community-engaged research, community-engaged scholarship must be understood as scholarly and acceptable research outputs within the guidelines.As evidenced in this study, without clearly defining engaged research and its scholarship as research in school-and department-level guidelines, review committees are ultimately required to define, categorize, and thus evaluate it as service.Only when properly defined, understood, and categorized as research, can engage research and the scholarship it produces be appropriately evaluated by review committees.
Within the current structures of promotion and tenure, the placement of engaged work within promotion and tenure guidelines and categories is one of debate among scholars.There is no consensus around where to place engaged work in its own "community engagement" category or integrated throughout the established categories of teaching, research, and service.Some scholars believe it is essential that engaged scholarship be explicitly recognized within the research category and viewed as parallel to traditional research (Saltmarsh et al., 2009;Weerts & Sandman, 2008).However, others believe that it is best to avoid the rigid divisions of teaching, research, and service and instead integrate all forms of engaged intellectual work into its own space (e.g., a fourth "engagement" category) that recognizes the public purpose of academic work (Colbeck & Wharton-Michael, 2006).
Within the current structures of promotion and tenure at the R1 institutions included in this study, the research category reigns supreme.For faculty on the tenure track, anything other than research is second-tier and devalued compared to what is understood as research.Thus, creating a fourth "engagement" category would run the risk of becoming second tier for tenure-track faculty.When recognized as something other than research, such work will continue to be devalued or be reserved as a way only for non-tenure track faculty to provide evidence of scholarship.Thus, community-engaged research and the non-traditional scholarship it produces must be recognized and included within the current research category.Without such recognition, it will continue to be devalued in a fourth space.It cannot and should not be siloed into its category and othered.Community engagement is a strategy for faculty to accomplish their teaching, research, and service.Creating a fourth category and distinguishing it as something other than teaching, research, and service ultimately negates this thinking.

Construct Appropriate Metrics to Evaluate Community-Engaged Scholarship as a Product of Research
Recognizing community-engaged research and its scholarly products as valid forms of research within promotion and tenure guidelines is only half the battle.When engaged research is categorized as research, review committees are required to evaluate it as such, and they only have one minimal set of metrics upon which to assess it.This study illustrates how challenging review committees are to evaluate engaged scholarship using the current metrics constructed to assess traditional scholarship.Earlier, Table 3 outlined the traditional metrics used by committees to evaluate all forms of scholarship and the challenges of evaluating engaged scholarship through the same lens.To appropriately evaluate community-engaged scholarship, it must be judged against a set of metrics constructed to assess its unique contributions and rigor.
In Table 4, alterations to the current metrics are proposed to evaluate community-engaged scholarship more appropriately.It is important to emphasize that non-traditional, engaged scholarly products must still be subject to peer review.However, as this study illustrates, the review of engaged products requires alternative metrics to evaluate.Further, adjusting traditional metrics to evaluate engaged scholarly products better will likely vary by school/department and differ based on unique school/departmental culture.

Recommendations for Future Research
Future research related to the evaluation of engaged scholarship in promotion and tenure would continue to enhance and push forward the conversations about meaningfully institutionalizing and leveraging community engagement as a strategy for higher education's overall success.The need for identifying and clarifying additional metrics to effectively evaluate community-engaged research and its engaged scholarship is evident.More specifically, research exploring the diverse ways to illustrate and assess the processes (e.g., generating and sustaining meaningful partnerships, ensuring trust with community partners, identifying shared research goals) and impact of engaged research is warranted.Further, research investigating alternative ways to document and assess the intangible processes of community-engaged research would shed light on how to bring engaged research processes most effectively into the conversation of faculty reward.
The findings also support calls for future research to address and clarify ways to successfully document and evaluate the impact of faculty's community-engaged research.Though the proposed adjustments to current evaluative metrics and resources to assist review committees with the process of evaluation outlined in prior sections provide a foundation upon which to initiate change, questions remain specifically around the issue of defining, documenting, measuring, and evaluating impact.The understanding of impact must move beyond journal impact factors.However, engaged faculty and promotion and tenure committees continue to wrestle with questions that future research could explore, such as: • How can engaged faculty document the impact of an engagement project or program?
• What constitutes "appropriate" or "valuable" scholarly impact within engagement?
• How can community partnerships' depth, strength, and impact be measured?
• How can community voice be appropriately leveraged to assist with illustrating impact?Further, there is a need for additional exploration into how the expert voice of community partners can be best leveraged within the evaluative aspect of promotion and tenure.The need to improve how community partners are leveraged as experts and external reviewers is evident.Further research is needed to investigate the most advantageous ways to bring in and value the voice of community partners (e.g., through individual external letters, via an institutional or external resource of vetted partners who can serve as reviewers).Some limitations of this study could also be addressed through future research, specifically including the perspectives of faculty of color to provide additional insights into the nuance of how issues of diversity, equity, and inclusion are impacted by and fueled through the current limitations of promotion and tenure is necessary.As this study did not explicitly seek to include participants of color and was limited by the availability and willingness of engaged faculty to participate, a more nuanced approach to such research, specifically exploring the intersection of faculty reward and diversity, equity, and inclusion, would lead to important revisions to policy and practice.

Study Limitations
As this study included only a select number of R1 institutions that received the Carnegie community engagement classification or reclassification in 2020, the transferability or applicability of findings is less strong for non-R1 institutions that have not received the classification (Lincoln & Guba, 1985).This study intentionally narrowed its site and participant selection to extract deeper, more focused experiences and minimize the variance of institutions and individual participants within the study sample.The further narrowing of participants to those within either a social science or STEM field does not include all academic disciplines, departments, or schools.Still, it provides a balanced sample of faculty perspectives across multiple disciplines.Though the emphasis on recently classified R1 universities and additional criteria required for participants was warranted, it does open the door for future research exploring other institutions and faculty types.
It should be noted that the goal of this inquiry was not to make universal generalizations or construct a conclusive theory about the evaluative practices of promotion and tenure committees.Instead, the intent was to generate new knowledge and provide insights into how promotion and tenure committees understand, evaluate, and make judgments regarding faculty's engaged research and to encourage additional research and evaluations of practice.
A clear limitation of this study is its lack of a more racially diverse, non-white participant group.Though the recruitment of participants was not restricted to any specific racial or ethnic group, it was limited by the availability and willingness of faculty within the identified institutions.Only white faculty members responded to invitations to participate in this study.Though it is not certain as to why only white faculty were interested in participating, research has illustrated that faculty of color are more frequently asked to spend their time and energy serving diversity-focused campus initiatives and mentoring students, creating an often invisible cultural taxation that leaves them with little leisure time (Guillaume & Apodaca, 2020;Jimenez et al., 2019;Matthew, 2016;University of Oregon Social Sciences Feminist Network Research Interest Group, 2017).Including the perspectives of faculty of color would greatly enhance this study and provide additional insights into the nuances of how issues of diversity, equity, and inclusion are impacted by and fueled through the current structures of promotion and tenure.

Conclusions
With more and more institutions committing to advancing an engaged agenda and being recognized for doing so, we must ask ourselves what constitutes an institution as engaged.As higher education and the academic labor market continue to evolve and respond to the increasing needs of our communities, faculty are faced with the choice of where to employ their time and expertise.Undoubtedly, faculty will continue to engage with their local communities meaningfully.The question, rather, is a matter of how quickly, if at all, institutions will recognize the need to align their public commitment to community engagement with the appropriate internal systems, structures, and policies to recognize and reward their faculty who are doing it.With the upheaval caused by the COVID-19 pandemic, increasing political tensions, and dwindling social trust in higher education, institutions are poised to reorient and reimagine the ways they leverage and reward the engaged work of their faculty.The last few years have provided a unique perspective and call to reevaluate the public purpose of higher education.There is a real opportunity for change as our country, institutions, faculty, and communities continue to reckon with what it means to commit ourselves to furthering the public good.Higher education must be up for the challenge.products of public scholarship in research and creative activity.IUPUI Scholarworks.Retrieved from http://hdl.handle.net/1805/25254Yanow, D. (2006).Interpretation and method empirical research methods and the interpretive turn.M. E. Sharpe.

FIGURE 1 .
FIGURE 1. Recommended steps for institutions working to appropriately recognize and reward community-engaged research and scholarship within promotion and tenure

TABLE 1 .
Institutions Engagement in inst.mission" refers to if the campus mentions community engagement (or similar terminology) in their institutional mission."Engagement in strategic plan" refers to if the campus has a specific goal area or category within its institution-level strategic plan focusing on engagement with the community."Highest engagement leadership" provides the most senior leadership position within the campus whose scope of work explicitly includes community engagement (or similar terminology) within their title.

Metrics to Evaluate Faculty Research Do Not Work for Community- Engaged Scholarship
. There is very little mention of rewarding engagement in alignment with faculty research.Results from this study clearly state that to ensure faculty members' engaged scholarship is appropriately rewarded within promotion and tenure, departments, schools, and their institutions must more broadly define what constitutes scholarly work and what can be categorized as scholarship.However, if given the choice, faculty are seemingly stuck between a rock and a hard place.If they elect to categorize their engaged scholarship as service, it would not be weighted as heavily as other non-engaged scholarly products.If they categorize their engaged scholarship as research, it is held to traditional research standards and metrics, which pose unique challenges.The need for promotion and tenure committees to judge the quantity and quality of faculty scholarship quickly and efficiently in the most unbiased manner has caused committees to utilize a very specific, standardized set of metrics to evaluate faculty scholarship.All participants in this study referenced a similar list of traditional metrics used by review committees to evaluate engaged scholarship.Table3below synthesizes and identifies what participants cited as the most prominent metrics used by review committees to evaluate faculty's scholarship and the unique challenges they pose when attempting to use them to assess engaged scholarship specifically.

TABLE 3 .
Traditional metrics used to evaluate faculty scholarship and challenges when applied to evaluation of community-engaged scholarship

TABLE 4 .
Current traditional metrics used to evaluate faculty scholarship and proposed adjustments to more appropriately evaluate community-engage scholarship