Confirmatory Factor Analysis for a Service-Learning Outcomes Measurement Scale (S-LOMS)
DOI:
https://doi.org/10.18060/23920Keywords:
Service-learning, Hong Kong, student developmental impacts, measurement instrumentsAbstract
Service-learning was introduced into Hong Kong over a decade ago, yet there is a research gap about the self-perceived developmental outcomes for students, partly due to the lack of a reliable measurement instrument across course disciplines and types of service-learning. This study validated a recently created Service-Learning Outcomes Measurement Scale (S-LOMS) through confirmatory factor analysis (CFA) with data from 629 students. S-LOMS measures self-perceived student development through 56 items, which cover outcome domains under four overarching categories: knowledge application, personal and professional skills, civic orientation and engagement, and self-awareness. Alternative measurement models were compared in this validation exercise, with the results indicating that although a model with 11 domains and without overarching categories was preferred, there was also support for a model with 10 domains subsumed under the four abovementioned overarching categories. Multi-sample analyses indicated that both models were stable across gender. The practical implication of our findings is that for the purpose of measuring the developmental impacts on students of engaging in service-learning, S-LOMS offers investigators a number of options besides using the entire 56-item scale. Some administrative options are described at the end of the paper.
References
Astin, A. W., & Sax, L. J. (1998). How undergraduates are affected by service participation. Journal of College Student Development, 39 (3), 251–263. Retrieved from https://digitalcommons.unomaha.edu/slcehighered/7.
Astin, A. W., Vogelgesang, L. J., Ikeda, E. K., & Yee, J. A. (2000). How service-learning affects students. Los Angeles, CA: Higher Education Research Institute, UCLA. Retrieved from https://digitalcommons.unomaha.edu/slcehighered/144.
Bentler, P. M. (1990). Comparative fit indices in structural models. Psychological Bulletin, 107, 238–246.
Bentler, P. M. (2006). EQS 6 structural equations program manual. Encino, CA: Multivariate Software, Inc.
Blunch, N. J. (2016). Introduction to structural equation modeling using IBM SPSS Statistics and EQS. London: SAGE.
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). New York, NY: Guilford.
Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods and Research, 21, 230-258.
Byrne, B. M. (2008). Structural equation modeling with EQS: Basic concepts, applications, and programming (2nd ed.). New York: Psychology Press.
Celio, C. I., Durlak, J., & Dymnicki, A. (2011). A meta-analysis of the impact of service-learning on students. Journal of Experiential Education, 34(2), 164–181. https://doi.org/10.5193/jee34.2.164.
Conway, J. M., Amel, E. L., & Gerwien, D. P. (2009). Teaching and learning in the social context: A meta-analysis of service
learning's effects on academic, personal, social, and citizenship outcomes. Teaching of Psychology, 36(4), 233–245. https://doi.org/10.1080/00986280903172969.
Driscoll, A., Holland, B., Gelmon, S., & Kerrigan, S. (1996). An assessment model for service-learning: Comprehensive case studies of impact on faculty, students, community and institutions. Michigan Journal of Community Service Learning, 3, 66–71.
Einfeld, A., & Collins, D. (2008). The relationships between service-learning, social justice, multicultural competence, and civic engagement. Journal of College Student Development, 49(2), 95–109. https://doi.org/10.1353/csd.2008.0017.
Eyler, J. S., Giles, D. E., & Braxton, J. (1997). The impact of service-learning on college students. Michigan Journal of Community Service Learning, 4(1), 5–15. Retrieved from http://hdl.handle.net/2027/spo.3239521.0004.101.
Eyler, J. S., & Giles, D. E. (1999). Where's the learning in service learning? San Francisco, CA: Jossey-Bass.
Eyler, J. S., Giles, D. E., Stenson, C. M., and Gray, C. J., (2001). At a glance: What we know about the effects of service-learning on college students, faculty, institutions and communities, 1993–2000 (3rd ed.). Nashville, Tenn.: Vanderbilt University Press.
Felten, P., & Clayton, P. H. (2011). Service-Learning. New Directions for Teaching and Learning, 128, 75–84. https://doi.org/10.1002/tl.470.
Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling: guidelines for determining model fit. Electronic Journal of Business Research Methods, 6, 53–60.
Hu, L.T. and Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55.
Hurley, A., Scandura, T., Schriesheim, C., Brannick, M., Seers, A., Vandenberg, R., & Williams, L. (1997). Exploratory and confirmatory factor analysis: guidelines, issues, and alternatives. Journal of Organizational Behavior, 18(6), 667-683.
Jacoby, B. (1996). Service-learning in higher education: concepts and practices. San Francisco: Jossey-Bass.
Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The sources of four commonly reported cutoff criteria. Organizational Research Methods, 9(2), 202–220. https://doi.org/10.1177/1094428105284919.
Lo, K. W. K., Kwan, K. P., Chan, S. C. F., & Ngai, G. (2016). Cross-cultural validation of the global citizenship scale for measuring impacts of international service-learning in Hong Kong setting. Paper presented at the 3rd International Conference on Service-Learning, Hong Kong.
Lundy, B. L. (2007). Service learning in life-span developmental psychology: Higher exam scores and increased empathy. Teaching of Psychology, 34(1), 23–27. https://doi.org/10.1080/00986280709336644.
Ma, C. (2018). Service-learning development in higher education in Hong Kong. In T. W. Lim, & W. X. Li (Eds.), Studying Hong Kong: 20 Years of Political, Economic and Social Developments (pp 43–61). New Jersey: World Scientific. https://doi.org/10.1142/9789813223554_0004.
Ma, C. H. K., & Chan, A. C. M. (2013). A Hong Kong university first: Establishing service-learning as an academic credit-bearing subject. Gateways: International Journal of Community Research and Engagement, 6, 178–198. https://doi.org/10.5130/ijcre.v6i1.3286.
Ma, H. K. C., Chan, C. W. F., Tse, P. H. (2019). A common outcome measurement for service-learning in Hong Kong. Journal of Higher Education Outreach and Engagement, 23(3), 3–19.
Ngai, S. S. (2009). The effects of program characteristics and psychological engagement on service-learning outcomes: A study of university students in Hong Kong. Adolescence, 44, 375–389. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/19764273.
Novak, J. M., Markey, V., & Allen, M. (2007). Evaluating cognitive outcomes of service learning in higher education: A meta-analysis. Communication Research Reports, 24(2), 149-157. https://doi.org/10.1080/08824090701304881.
Prentice, M. (2007). Service learning and civic engagement. Academic Questions, 20, 135–145. https://doi.org/10.1007/s12129-007-9005-y.
Rama, D. V. (1998). Learning by doing: Concepts and models for service-learning in accounting. AAHE's series on service-learning in the disciplines. Washington, D.C.: American Association for Higher Education. Retrieved from https://eric.ed.gov/?id=ED449726.
Richard, D., Keen, C., Hatcher, J. A., & Pease, H. A. (2017). Pathways to adult civic engagement: Benefits of reflection and dialogue across difference in higher education service-learning programs. Michigan Journal of Community Service Learning, 23(1), 60–74. http://dx.doi.org/10.3998/mjcsloa.3239521.0023.105.
Savalei V., & Bentler, P. M. (2005). A Statistically justified pairwise ml method for incomplete nonnormal data: a comparison with direct ML and pairwise ADF. Structural Equation Modeling, 12, 183–214. https://doi.org/10.1207/s15328007sem1202_1.
Schumacker, R.E., & Lomax, R. G. (1996). A beginner’s guide to structural equation modeling. Mahwah, NJ: Lawrence Erlbaum Associates Inc.
Shumer, R., Stanton, T. K., & Giles D. E., Jr. (2017). History and precursors of service-learning theory, development and research. In Robert Shumer (Ed.), Where’s the wisdom in service-learning? (pp. 1-32). NC: Information Age Publishing.
Simons, L., & Cleary, B. (2006). The influence of service learning on students’ personal and social development. College Teaching, 54(4), 307–319. https://doi.org/10.3200/ctch.54.4.307-319.
Siu, P. Y. C., Tang, H. H. E., & Lai, C. O. A. (2013). Teaching and learning global citizenship through service-learning: The success story of cross-border service-learning summer institute, Lingnan University, Hong Kong. Paper presented at 4th Asia-Pacific Regional Conference on Service-Learning: Service-Learning as a Bridge from Local to Global: Connected World, Connected Future. Hong Kong and Guangzhou, China. Retrieved from https://commons.ln.edu.hk/cgi/viewcontent.cgi?referer=https://scholar.google.com/&httpsredir=1&article=1073&context=osl_conference.
Snell, R. S., & Lau, K. H. (2020). The development of a service-learning outcomes measurement scale (s-loms). Metropolitan Universities, 31(1), 44-77. https://doi.org/10.18060/23258.
Stevens, J. P. (2009). Applied multivariate statistics for the social sciences (5th ed.). New York: Routledge. https://doi.org/10.4324/9780203843130.
Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Boston: Allyn & Bacon.
Tay, L., & Jebb, A. (2017). Scale creation. In S. Rogelberg (Ed.), The SAGE Encyclopaedia of Industrial and Organizational Psychology (2nd ed.) (pp. 1381–1383). Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781483386874.n242.
Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington D. C.: American Psychological Association.
Tong, X., Zhang, Z., & Yuan, K. H. (2014). Evaluation of test statistics for robust structural equation modeling with nonnormal missing data. Structural Equation Modeling: A Multidisciplinary Journal, 21(4), 553-565, https://doi.org/10.1080/10705511.2014.919820.
Warren, J. L. (2012). Does service-learning increase student learning? A meta-analysis. Michigan Journal of Community Service Learning, 18(2), 56–61. Retrieved from http://hdl.handle.net/2027/spo.3239521.0018.205.
Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: a content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127.
Xing, J., & Ma, C. (2010). Service-learning in Asia: Curricular models and practices. Hong Kong University Press. https://doi.org/10.5790/hongkong/9789888028467.001.0001.
Yorio, P. L., & Ye, F. (2012). A Meta-analysis on the effects of service-learning on the social, personal, and cognitive outcomes of learning. Academy of Management Learning & Education, 11(1), 9–27. https://doi.org/10.5465/amle.2010.0072.
Downloads
Published
Issue
Section
License
Copyright (c) 2020 Ka Hing LAU, Robin Stanley SNELL
This work is licensed under a Creative Commons Attribution 4.0 International License.
