Household Higher Ed Cancellation Plans Fall 2020 Semester

Chart showing break-out of households with one or more residents cancelling plans to attend fall semester 2020, with break-out categories by race, level of education (high school, some college, bachelor's degree), income level, and whether the household had one or more residents using SNAP (food stamps), who are unemployed, or who are depending on only regular income to the household.

Chart showing break-out of households with one or more residents cancelling plans to attend fall semester 2020, with break-out categories by race, level of education (high school, some college, bachelor’s degree), income level, and whether the household had one or more residents using SNAP (food stamps), who are unemployed, or who are depending on only regular income to the household.

A Proposed Revision of the UCO Classroom Evaluation Process for Tenured and Nontenured Faculty

by John R. Wood, Associate professor, UCO¹

In this white paper, I would like to make a case for the University of Central Oklahoma to revise their classroom evaluation process for tenure and nontenure faculty. We need to address the need to change the classroom evaluation requirements for UCO tenure track and non-tenure-track faculty because summative evaluations of faculty create unneeded stress for faculty and students. Regrettably, many, if not most, UCO faculty are required to conduct summative classroom evaluations in every class, every year.

Simply, for a summative evaluation, the quantitative measure’s value is limited to “information needed to make a personnel decision – for example, hiring, promotion, tenure, merit pay” (Chism, 2007,  5). Theoretically, the correlation between higher grades and higher evaluations would indicate more learning; however, this is not necessarily the case (See Marsh and Roche 1997, 2000). Unfortunately, “an evaluation often tells more about a student’s opinion of a professor than about the professor’s teaching effectiveness” (Williams, 2007, 171). Student evaluations also tend to be more positive toward instructors who display a certain level of warmth in their behavior (Best & Addison, 2000). Moreover, faculty perceived as caring receive higher numerical outcomes on evaluations, and this also affects student perceptions of their own cognitive learning, signifying that an instructor’s personal characteristics can influence evaluations (Teven & McCroskey, 1997).

Education scholars also find that internal factors, i.e., professor’s sexual orientation, ethnicity, gender, or in the course content, or the nature of the course itself, the rigor of the course and the faculty member’s teaching style, less demanding courses, influence numerical outcomes in these types of numerical evaluations. Internal factors consider such aspects, as the professor’s sexual orientation, ethnicity, gender, or in the course content, or the nature of the course itself, can influence the numerical outcomes of the evaluation (Basow & Howe, 1987; Hobson, 1997; Sidanius & Crane, 1989; Williams, 2007; Flaherty, 2018). According to Centra (2003, 498), there is a bias “when a student, teacher, or course characteristic affects the evaluations made, either positively or negatively, but is unrelated to any criteria of good teaching, such as increased student learning.” For example, prior work has found that student evaluations are related to grading leniency (Greenwald & Gilmore, 1997) and the gender of the instructor (Basow & Silberg, 1987). Other factors can also influence these numerical outcomes, such as the rigor of the course and the faculty member’s teaching style. Likewise, the personal appearance of the instructor, including student perceptions of gender or race of the instructor, can influence these evaluations. Findings in the broader literature support claims both for and against the notion that the appearance of the instructor influences student scores of instructor effectiveness (Campbell, Gerdes, & Steiner, 2005). There is also research suggesting that instructors who teach fewer demanding courses receive evaluations that are abnormally high (Overbaugh, 1998). In addition, not surprisingly, a student’s class grade & Gillmore, 1997; Tang, 1999).

In fact, the evidence opposing the validity of these evaluations is compelling.

Two types of evaluation of teaching – summative and formative.

1.    Indirect assessment (summative) of teaching captures students’ perceptions of their learning (UCO’s SPIES). It is a proxy for student learning

2.    Direct assessments (formative) of student learning reflect a demonstration/evidence of student learning (Maki, 2010). Indirect measures are tests of validity in the sense that there is a discrepancy between the judgmental measure (usually a rating of achievement) and the criterion measure (a score on a standardized achievement test). Direct measures are a more direct test of validity in that teachers are directly asked to estimate the achievement test performance of their students. On the whole, the results revealed high levels of validity for the teacher-judgment measures. Best practices cited in Maki (2010) are teaching portfolios (including a teaching philosophy and evidence of practice (samples direct assessment).

While summative classroom evaluation is helpful to assess faculty, it does little in terms of learning and reflecting on improving teaching (Chism, 2007). However, a formative evaluation “describes activities that provide teachers with information that they can use to improve their teaching” (Chism, 2007, p. 5). In balance, Lewis and Benson (1998; Youmans & Lee, 2007) find that summative faculty evaluations persist to provide balance to the hotly debated question whether student evaluations are either valid or reliable indicators of either the faculty’s effectiveness or the course’s quality. However, as long as universities continue to regard student input as important, instruments measuring student perception, inadequate they may be, will presumably remain part of the process. Even so, they should not be the primary and only process of class evaluation, which then presents a circular argument to support a second imperfect evaluative process, i.e., peer evaluation among faculty. Revising the nature of faculty peer evaluations would retain the feedback benefits to the observed while potentially also generating more instructional dialogue among faculty (Chism, 2007). Similarly, the American Association of University Professors’ (AAUP), argue that any system developed for evaluation are best directed toward constructive measures for improvement (Euben, 2005).

 

Thus, the following suggestions for revision should be considered:

Tenure track and nontenure track faculty should be assessed with a summative classroom evaluation only in the 1st, 3rd, 5th, and every third year for post tenure review and for promotion to full professor. Summative evaluation focuses on information “needed to make a personnel decision – for example, hiring, promotion, tenure, merit pay” (Chism, 2007, 5).

a. Often quantitative

b. For public inspection

c. Determine quality of teaching performance compared to peers

d. Conducted at given intervals, i. e., Annual or promotion and tenure reviews

 

Faculty should supplement summative classroom evaluation with a formative classroom evaluation with combination of peer review and a teaching portfolio. Formative evaluation provides teachers with information they can use to improve their teaching (Chism, 2007, 5).

a. For personal use, private and confidential.

b. Qualitative

c. Peer reviews & Portfolios

d. Focused on continuous learning & improvement

During the 2nd, 4th, and every year summative classroom evaluations are not required. The faculty member would create a teaching portfolio that will create a conversation with their respective department chair and dean on how to promote continuous improvement.

The faculty member can use peer evaluation, their own questionnaire, etc. The teaching portfolio will not be a part of an official document unless the faculty member so desires.

In this way, a combination of a summative evaluation will make administrators happy with accountability in mind and easy numerical measures to manage a large number of faculty.  In balance, a formative evaluation is best when the faculty continuously learn about their students and facilitate innovation in the classroom.

References:

Basow, Susan. A., and Karen. G. Howe. 1987. “Evaluations of college professors: Effects of professors’ sex-type, and sex, and students’ sex.” Psychological Reports 60:671-8.

Campbell, Heather, Karen Gerdes, and Sue Steiner. 2005. “What’s Looks Got to Do with It? Instructor Appearance and Student Evaluations of Teaching.” Journal of Policy Analysis and Management, 24 (3): 611-620.

Centra, John A. 2003 “Will Teachers Receive Higher Student Evaluations by Giving Higher grades and Less Course Work?” Research in Higher Education 44(5) Oct. 03: 495-518.

Chism, Nancy Van Note. 2007. Peer Reviewing of Teaching: A Sourcebook. 2nd Ed. Indiana University-Purdue University Indianapolis: Anker Publishing.

Euben, Donna. 2005. “Post-Tenure Review: Some Case Law” AAUP Counsel https://www.aaup.org/issues/tenure/some-case-law (accessed May 30, 2020).

Flaherty, C. 2018. “Same Course, Different Ratings Study says students rate male instructors more highly than women even when they’re teaching identical courses. Inside Higher Ed. https://www.insidehighered.com/news/2018/03/14/study-says-students-rate-men-more-highly-women-even-when-theyre-teaching-identical

Greenwald, Anthony. G., & Gilmore, Gerald. M. 1997. “Grading leniency is a removable contaminant of student ratings.” American Psychologist 52, 1209-1217. http://faculty.washington.edu/agg/pdf/Gwald_Gillmore_AmPsychologist_1997.OCR.pdf (accessed December 20, 2019).

Hobson, Suzanne. M. 1997. “The impact of sex and gender-role orientation on student evaluations of teaching effectiveness in counselor education and counseling psychology.” Ed.D. diss. Western Michigan University, Kalamazoo.

Lewis, Jerry M. and Denzel E. Benson. 1998. “Section Eight. Course Evaluations,”: 99- 114 in Tips for Teaching Introductory Sociology, edited by Jerry M. Lewis. Belmont, CA: Wadsworth

Maki, Peggy. 2010.  Assessing for learning: building a sustainable commitment across the institution. 2nd edition. Sterling, VA: Stylus Publishing.

Marsh, Herbert. W., and Lawrence. A. Roche. 2000. “Effects of grading leniency and low workload on students’ evaluations of teaching: Popular myth, bias, validity, or innocent bystanders?” Journal of Education Psychology 92: 202-28.

Overbaugh, Richard. C. 1998. “The effect of course rigor on preservice teachers’ course and instructor evaluation.” Computers in the Schools 14:13-23.

Sidanius, Jim, and Marie. Crane. 1989. “Job evaluation and gender: The case of university faculty.” Journal of Applied Social Psychology 19:174-97.

Stumpf, Steven. A., and Richard. D. Freedman. 1979. “Expected grade covariation with student ratings of instruction: Individual versus class effects.” Journal of Educational Psychology 71:293-302.

Tang, Shengming. 1999. “Student evaluation of teachers: Effects of grading at college level.” Journal of Research and Development in Education 32:83-8.

Teven, Jason. J., and James. C. McCroskey. 1997. “The relationship of perceived teacher caring with student learning and teacher evaluation.” Communication Education 46:1-9.

Williams, Dana A. 2007. “Examining the Relation between Race and Student Evaluations of Faculty Members: A Literature Review.” Profession: 168-173.

Youmans, Robert J. and Benjamin D. Lee. 2007. “Fudging the Numbers: Distributing Chocolate Influences Student Evaluations of an Undergraduate Course.” Teaching of Psychology 34 (4): 245-247.

 

¹ This white paper was created after an UCO CETTL workshop on the book by Chism, Nancy Van Note. 2007. Peer Reviewing of Teaching: A Sourcebook in the fall 2018.

My Transformative Experience While Teaching

[Written by Therese Williams, PhD]

As a mature white woman newly teaching at the university level, I have been afforded opportunities that were not frequently offered while I spent many years working in industry and government.  I have participated in numerous book groups sponsored by the 21st Century Pedagogy Institute (21CPI) at UCO and learned from each experience.

In the fall of 2019, I participated with a group reading and discussing “ain’t i a woman” by bell hooks. Although I was aware of some of the horrors women in slavery experienced in this country, and I knew that racial discrimination has continued long after the Emancipation Proclamation,  the 13th Amendment to the U.S. Constitution and the Civil Rights Act of 1964 were enacted; I was not aware that white women actively discriminated against women of color during the Women’s Suffrage Movement in the late 1800s and early 1900s.  During this group, the book “White Fragility: Why It’s So Hard for White People to Talk About Racism” by Robin DiAngelo was recommended to me.  It went on my list, but as it happens, it became a group in the spring of 2020.

This book significantly helped me to understand things that I ‘thought’ I knew and things that I probably didn’t know and I finally understand the ‘white privilege’ mentality that is so pervasive in our society, in all of our lives, and especially in corporate and government power structures.  As part of this book group, for 21CPI, I wrote a reflection that committed to continuing my education on these topics and to support, in ways that are possible for me, the goals of groups that are struggling to change our society to become more inclusive and provide a level playing field for ALL people.

And then George Floyd was murdered.  I have attended a webinar supported by white women in the Women’s March movement who want to help others become educated on the issues, a webinar sponsored by Black Lives Matter OKC that featured Black Women who spoke about their struggles and changes they believe will be effective, a webinar sponsored by UCO – “Navigating Racial Trauma Panel” which spoke to helping yourself and others to handle racial trauma, and bits and pieces of other opportunities including reading a newly recommended book “How to be an Anti-Racist” by Ibram X. Kendi.  I’ve learned that it’s not enough to just not be a racist but that one must actively be an anti-racist.  One of the many quotes attributed to Martin Luther King is “There comes a time when silence is betrayal,” and I am taking this quote to heart.

I teach primarily technology classes in the College of Business.  As such, there is not much opportunity to slide social issues into the classroom.  As a result of some of the transformation I have been going through, I am working on a personal inclusion statement to be included in my classes.  At first, I thought I would include it in the syllabus, but as all instructors know, only a few students really read the syllabus!  Therefore, I am going to include it as a separate document.  I will discuss my journey, the organizations I am supporting, and include a recommended reading list culled from books recommended to me and other lists I have seen.

When I was an undergraduate student, although I read a great deal, I was not exposed to ideas on social issues other than in my family.  And in my family, we did not (and still don’t) discuss issues beyond the basics of kindness, fairness, and truth.  My hope is that, perhaps, someone in a class will start to think about these issues beyond what they have already learned, and it will become transformational for them.

Addressing Racial and Social Injustice

Graphic logo for University of Central Oklahoma's Center for Excellence in Transformative Teaching and Learning

The Center for Excellence in Transformative Teaching and Learning (CETTL) includes the 21st Century Pedagogy Institute (21CPI), Student Transformative Learning Record (STLR), and Educators’ Leadership Academy (ELA). Collectively, we serve faculty and students at UCO and beyond.

In times that underscore the ongoing need to not just denounce racism but to be anti-racist, CETTL, 21CPI, STLR, and ELA commit to more than simply continuing what we do now, such as our ongoing work over the years like past development opportunities around race and sexual identities, and faculty-led book groups on titles like White Fragility and Teaching Queer, and our STLR work with Global & Cultural Competencies. (Please visit Selected Professional Development Resources about Social Injustice for Faculty and Staff.) We further commit to accepting the challenge that we, UCO, and the nation must meet to bring justice and equity to our campus and our society for all Black people and People of Color as well as other marginalized and disenfranchised groups and individuals among our fellow colleagues and citizens.

Perhaps no other college or university is more strongly called to meet this challenge. Because UCO centers itself around Transformative Learning (TL), we must in these times follow its process, one that literally defines personal learning and growth as prompted by disorienting dilemmas that lead to critical self-reflection followed by perspective shifts that help learners change themselves and their conceptions to embrace new and better ways of being.

We’re a university. Learning is what we do. Past learning and growth to become less racist is part of who we are, but now we are challenged to learn more, to do more.

Protests across the nation are calling attention to what for many in the privileged class may be the disorienting dilemma that is our country’s current and long-standing failure to ensure equity in treatment for Black people and People of Color, LGBTQIA+, and other groups suffering unequal treatment. Among our fellow citizens and colleagues who suffer the inequitable treatment dilemma daily, there is always the dilemma of how to make a co-worker or a group or a nation listen, to understand, to change.

We commit to listen. We will work to understand, and we will change as a result of our critical reflection and with the help and guidance of colleagues. To that end, CETTL, 21CPI, STLR, ELA are initiating among ourselves TL-focused processes with the intent to improve ourselves to become better colleagues and models for others. We hope our journey will produce transformative realizations in service to adding to the broader solution so necessary right now.

This is in addition to the work we do daily to support diversity and inclusivity and in providing resources and learning opportunities to support the transformative realizations and the resulting behavior and mindset changes that others may seek.

Jeff King, Ed.D., Executive Director, Center for Excellence in Transformative Teaching and Learning (CETTL)
Jody Horn, Ph.D., Director, 21st Century Pedagogy Institute (21 CPI)
Cary Williams, Director, Educators’ Leadership Academy (ELA)
Camille Farrell, M.Ed., Assistant Director, Student Transformative Learning Record (STLR)
Mark Walvoord, M.S., Assistant Director, Student Transformative Learning Record (STLR)
Brenton Wimmer, Ph.D., Assistant Director, Transformative Learning (TL) Assessment
Jon Hicks, Administrative Assistant III, CETTL


Horizontal logo graphic of Student Transformative Learning RecordLogo graphic of 21st Century Pedagogy Institute  Logo graphic of Educators' Leadership Academy

100 N. University Dr., Box 212, CTL 200 • Edmond, OK 73034 • Phone (405) 974-5570 • cettl.uco.edu21CPI.uco.edustlr.uco.eduela.uco.edutlconference.uco.edu

Selected Professional Development Resources about Social Injustice for Faculty and Staff

Addressing Racial and Social Injustice

In times that underscore the ongoing need to not just denounce racism but to be anti-racist, CETTL, 21CPI, STLR, and ELA commit to more than simply continuing what we do now…We further commit to accepting the challenge that we, UCO, and the nation must meet to bring justice and equity to our campus and our society for all Black people and People of Color as well as other marginalized and disenfranchised groups and individuals among our fellow colleagues and citizens… Read Full Letter


Upcoming Professional Development Book Groups on Topics of Racial & Social Injustice

 

Front cover of the book How to Be an Antiracist, by Ibram X. FendiHow to Be an Antiracist, by Ibram X. Kendi

  • Open to UCO faculty and staff
  • E-books will be provided to registrants in the order in which participants register, until our stock of books is depleted. If you want to join and the e-books are gone, please feel free to purchase your own copy and join the group.
  • Attendance at two out of three sessions and the creation of critical reflection is required for receiving a book.
Time, Dates, Location:
2:30-3:45 p.m. | July 14, July 21, July 28 | Zoom Meeting
FacilitatorMichelle Johnson, Adult Education and Safety Sciences

Front cover of the book Deconstructing Privilege: Teaching and Learning as Allies in the Classroom, by Kim Case

Deconstructing Privilege: Teaching and Learning as Allies in the Classroom, by Kim Case

  • Open to all full time and adjunct faculty
  • E-books and hardcover books will be provided to registrants in the order in which participants register, until our stock of books is depleted. If you want to join and the e-books are gone, please feel free to purchase your copy and join the group.
  • Attendance at two out of three sessions and the creation of critical reflection is required for receiving a book.
Time, Dates, Location:
1:30-3:00 p.m. | June 30, July 14, July 28 | Zoom Meeting
FacilitatorsMatt Hollrah and David Macey, English

Register here for Deconstructing Privilege book group


Selected Additional Resources

Inclusive Reading List in Political Science (with thanks to Victor Asal, Journal of Political Science Education Editor in Chief and the JPSE editorial team for the help organizing this list)

Documentaries to Watch

  • 13th, Netflix
  • When They See Us, Netflix
  • I Am Not Your Negro, Netflix, Amazon Prime
  • The Kalief Browder Story, Netflix
  • Stay Woke: The Black Lives Matter Movement, Amazon Prime or YouTube
  • Freedom Riders, Amazon Prime

Movies to Watch

  • Just Mercy, presently made free by the production company to educate, on YouTube, Amazon, Apple TV, Google Play
  • American Son, Netflix
  • Do the Right Thing, Amazon Prime
  • If Beale Street Could Talk, Hulu
  • Fruitvale Station, Hulu
  • Selma, available to rent
  • Wyatt Cenac’s Problem Areas, HBO
  • BlacKkKlansman, HBO Max

TV Shows to Watch

  • Dear White People, Netflix
  • Watchmen, Amazon Prime
  • Atlanta, Hulu
  • Self Made: Inspired by the Life of Madam C.J. Walker, Netflix
  • Luke Cage, Netflix


Anti-Racist Pedagogy Reading List (In part compiled by Andrea Aebersold, Ph. D – University of California, Irvine)

  • Akamine Phillips, Jennifer; Risdon, Nate; Lamsma, Matthew; Hambrick, Angelica; & Jun, Alexander (2019). Barriers and strategies by white faculty who incorporate antiracist pedagogy. Race and Pedagogy Journal: Teaching and Learning for Justice, 3(2).
  • Alexander, M. (2020 – updated edition) The New Jim Crow: Mass Incarceration in the Age of Colorblindness. New York: The New Press
  • Ash, A. N.; Hill, R.; Risdon, S. & Jun, A. (2020) Anti-racism in higher education: A model for change. Race and Pedagogy Journal: Teaching and Learning for Justice, 4(3).
  • Baldwin, J. (1963, December 21) “A Talk to Teachers.” The Saturday Review, 42-44.
  • Blackwell, D.M. (2010) Sidelines and separate spaces: Making education antiracist for students of color. Race, Ethnicity and Education, 13(4) pp. 473–494.
  • Blakeney, A. M. (2005) Antiracist Pedagogy: Definition, Theory, and Professional Development. Journal of Curriculum and Pedagogy, 2 (1) pp. 119–132
  • Case, K.A. (2013). Deconstructing privilege: Teaching and learning as allies in the classroom. New York: Routledge
  • Case, K. A. (Ed.) (2017). Intersectional pedagogy: Complicating identity and social justice. New York: Routledge/Taylor & Francis Group
  • Cole, C.E. (2017) Culturally sustaining pedagogy in higher education: Teaching so that black lives matter. Equality, Diversity and Inclusion: An International Journal, 36(8) pp. 736–750.
  • Condon, F. & Young, V.A. (eds) (2017). Performing antiracist pedagogy in rhetoric, writing, and communication. Fort Collins: The WAC Clearinghouse
  • D’Angelo, R. (2018). White fragility: Why it’s so hard for white people to talk about racism. Boston: Beacon Press.
  • Douglass Horsford, S., Grosland, T. J., & Morgan Gunn, K. (2011). Pedagogy of the personal and professional: Considering culturally relevant and antiracist pedagogy as a framework for culturally relevant leadership. Journal of School Leadership, 21(4).
  • Freire, P. (2000). Pedagogy of the oppressed. New York: Continuum
  • Giroux, Henry A. (2003). Spectacles of race and pedagogies of denial: Anti-black racist pedagogy under the reign of neoliberalism. Communication Education, 52(191-4), pp.191-211.
  • Haynes, C. (2017). Dismantling the white supremacy embedded in our classrooms: White faculty in pursuit of more equitable educational outcomes. International Journal of Teaching and Learning in Higher Education, 29(1), 87-107
  • Haynes, C., & Patton, L. D. (2019). From racial resistance to racial consciousness: Engaging white stem faculty in pedagogical transformation. Journal of Cases in Educational Leadership, 22(2), 85–98. https://doi.org/10.1177/1555458919829845
  • hooks, b. (1994) Teaching to transgress: Education as the practice of freedom. New York: Routledge.
  • Howell, A. & Tuitt, F. (2003) Race and higher education: rethinking pedagogy in diverse college classrooms. Cambridge: Harvard Educational Review
  • Inoue, A. B. (2015). Antiracist writing assessment ecologies: Teaching and assessing writing for a socially just future. Fort Collins: Parlor Press/WAC Clearinghouse
  • Inoue, A. B. (2019). Ccccc chair’s address: how do we language so people stop killing each other, or what do we do about white language supremacy. CCC 71.2
  • Jenkins, C. (2016). Addressing white privilege in higher education. Academic Exchange Quarterly, 20(4), 121-126. Retrieved from http://hdl.handle.net/1969.1/158792
  • Jenkins, C. M. (2018). Educators, question your level of cultural responsiveness. Journal on Empowering Teaching Excellence, 2(2), 15-23. https://digitalcommons.usu.edu/jete/vol2/iss2/4
  • Jenkins, C. (2018). Intersectional considerations in teaching diversity. In Carter, N & Vavrus, M. (Eds.), Intersectionalities of Race, Class, and Gender with Teaching and Teacher Education: Movement Toward Equity in Education. Leiden. The Netherlands: Brill/Sense. DOI: https://doi.org/10.1163/9789004365209_003
  • Jenkins, C., & Alfred, M. (2018). Understanding the motivation and transformation of White culturally responsive professors. Journal of Adult and Continuing Education, 24(1), 81-99. https://doi.org/10.1177/1477971417738793
  • Joseph, N. M., Haynes, C., Cobb, F. (Eds.). (2016). Interrogating whiteness and relinquishing power: White faculty’s commitment to racial consciousness STEM classrooms New York: Peter Lang Publishing.
  • Kailin, J. (2002). Antiracist education: From theory to practice. New York: Rowman & Littlefield Publishers, Inc
  • Kandaswamy, P. (2007). Beyond colorblindness and multiculturalism: Rethinking antiracist pedagogy in the university classroom. Radical Teacher, 80(6)
  • Kendi, I.X. (2019). How to Be an Antiracist. New York: One World
  • Kendi, I.X. (2017). Stamped from the beginning: The definitive history of racist ideas in America. New York: Bold Type Books
  • Kishimoto, K. (2018) Antiracist pedagogy: from faculty’s self-reflection to organizing within and beyond the classroom. Race Ethnicity and Education, 2(4), pp.540-554, DOI: 10.1080/13613324.2016.1248824
  • Lawrence, S. M. & Tatum, B. (1997). Teachers in transition: The impact of antiracist professional development on classroom practice. Teachers College Record, 99, 162–180
  • Milagros Castillo-Montoya, Joshua Abreu & Abdul Abad (2019) Racially liberatory pedagogy: A Black Lives Matter approach to education. International Journal of Qualitative Studies in Education, 32(9), pp., 1125-1145, DOI: 10.1080/09518398.2019.1645904
  • Oluo, I. (2019). So you want to talk about race? New York: Seal Press.
  • Phillips, C.B. & Derman-Sparks, L. (1997). Teaching/learning anti-racism: A developmental approach. New York: Teachers College Press.
  • Picower, B. (2009) The unexamined Whiteness of teaching: How White teachers maintain and enact dominant racial ideologies, Race Ethnicity and Education, 12:2, 197-215, DOI: 10.1080/13613320902995475
  • Pierce, Andrew J. J. (2016). Interest Convergence: An alternative to white privilege models of antiracist pedagogy and practice. Teaching Philosophy, 39(4), pp. 507–530
  • Saad, L.F. (2020). Me and white supremacy: Combat racism, change the world, and become a good ancestor. Naperville: Sourcebooks
  • Schick, C. (2000) ‘By Virtue of Being White’: Resistance in Anti-Racist Pedagogy. Race Ethnicity and Education, 3(1) pp. 83–101.
  • Sue, D. W. (2015) Race talk and the conspiracy of silence: Understanding and facilitating difficult dialogues on race. Hoboken, NJ: Wiley
  • Tatum, B.D. (2017). Why Are All the Black Kids Sitting Together in the Cafeteria? And Other Conversations About Race. New York: Basic Books.
  • Torino, G. C. (2015) Examining biases and white privilege: Classroom teaching strategies that promote cultural competence. Women & Therapy, 38(3-4) pp. 295–307
  • Tuitt, F. (2003b). Afterword: Realizing a more inclusive pedagogy. In A. Howell, & F. Tuitt (Eds.), Race and higher education: Rethinking pedagogy in diverse college classrooms (pp. 243–268). Cambridge, MA: Harvard Educational Review.
  • Tuitt, F., Haynes, C. & Stewart, S. (Eds.), (2016). Race, equity, and the learning environment: The global relevance of critical and inclusive pedagogies in higher education. Sterling, VA: Stylus
  • Tuitt, F., Haynes, C. & Stewart, S. (2018), Transforming the Classroom at Traditionally White Institutions to Make Black Lives Matter. To Improve the Academy, 37, pp. 63-76. doi:10.1002/tia2.20071
  • Wagner, A. (2005). Unsettling the academy: Working through the challenges of antiracist pedagogy. Race Ethnicity and Education, 8(3), 261-275.

Past/Current/Future CETTL-21CPI-STLR-ELA Activities

Still to Come Book Groups

  • Oklahomo: Lessons in Unqueering America by C. Mason, several facilitators and groups, Jessica Appleby (Modern Languages) and others. This book is a collaboration with the Gender and Sexualities Conference sponsored by the BGLTQ+ Student Center;
  • Against Common Sense:  Teaching and Learning Toward Social Justice, by K. K. Kumashiro, facilitator J. David Macey (English)

Related Past Book Groups and Workshops

  • White Fragility by Robin DiAngelo
  • ain’t i a woman: black women and feminism by bell hooks
  • Antiracist Writing Assessment Ecologies: Teaching and Assessing Writing for a Socially Just Future by Asao B. Inoue
  • Teaching Queer: Radical Possibilities for Writing and Knowing by Stacey Waite
  • Bandwidth Recovery: Helping Students Reclaim Resources Lost to Poverty, Racism, and Social Marginalization by Cia Verschelden (author is formerly of UCO)
  • Generative Knowledge Institute and Embodied Reflection by Melissa Peet
  • Intercultural Development Inventory by IDI, LLC in Collaboration with the Center for Global Competency
  • Beyond the Asterisk: Understanding Native Students in Higher Education, edited by Heather Shotton et al.
  • Allyship to Advocacy: Sexuality, Gender, and Transformative Learning in Collaboration with the Office of Cultural Competency
  • Teaching the Whole Student: Engaged Learning With Heart, Mind, and Spirit by David Schoem
  • Grit: The Power of Passion and Perseverance by Angela Duckworth
  • Daring Greatly: How the Courage to Be Vulnerable Transforms the Way We Live, Love, Parent, and Lead by Brené Brown
  • Trans* on Campus: Opportunities for Inclusion in Collaboration with SAFE and the Office of Cultural Competency
  • Cultural Competence in the Classroom Series in Collaboration with the Office of Diversity and Inclusion, SAFE, and the Center for Global Competency
  • Connecting Students with Global Opportunities

Past and Ongoing Global Cultural Competencies Campus Support

  • Iraqi Scholar Visitors (each year 5-6 visiting faculty since 2014)
  • Transformative Learning Scholar Post-Doctoral Faculty Research from Universidade Presbiteriana Mackenzie, São Paulo, Brazil
  • Student Transformative Learning Record (STLR) Faculty Staff Training on How to Incorporate and Assess the Central 6 Tenets (including Global and Cultural Competencies; STLR staff have facilitated training for 775+ faculty and staff in completing a 6-hour workshop and additional sessions)
  • STLR Financial and Overall Support to Black Male Initiative & Black Male Summit, Hispanic Success Initiative & Primeros Pasos, Native American Success Initiative, and LGBTQ+ Success Initiative (in Collaboration with the Office of Diversity and Inclusion and BGLTQ+ Student Center)
  • STLR-tagged Credit and Campus-Wide Partnerships for Global and Cultural Competencies Assignments, Events, Student Groups, Projects, and Campus Locations
  • International Transformative Learning Collaborative Working Partnerships with:
        • Universidade Presbiteriana Mackenzie | Sao Paulo, Brazil
        • University of the Free State | Bloemfontein, South Africa
        • Durban University of Technology | Durban, South Africa
        • University Of Eldoret | Eldoret, Kenya
        • Koya University | Koy Sanjaq, Iraq
        • Kathmandu University | Dhulikhel, Nepal
        • Methodist College Kuala Lumpur | Kuala Lumpur, Malaysia
        • Singapore Management University | Singapore
        • Murdoch University | Murdoch, Australia
        • Massey University | Manawatu,  New Zealand with Māori Cultural Education
        • Technological University of Dublin | Dublin, Ireland
        • Collège La Cité | Ottawa, Canada

Incorporating Transformative Learning into a Spanish Course

Introduction

Stacy SoutherlandI attended UCO’s Student Transformative Learning Record (STLR) training in the summer of 2019 and subsequently used information from that instruction to redesign one of my fall 2019 Elementary Spanish I culture activities. Motivating my effort was interest in refreshing course assignments, helping students meet course and UCO transformative learning objectives in new and diverse ways, and providing students an opportunity to earn STLR credit. This endeavor also allowed me achieve two of the five faculty learning outcomes (FLOs) that form part the faculty development framework offered by UCO’s 21st Century Pedagogy Institute. FLO 1 requires faculty to demonstrate the ability to redesign part of a course incorporating Transformative Learning Theory using evidence-based principles of teaching and learning. FLO 3 requires faculty to demonstrate the ability to use the STLR rubric effectively to measure student transformative learning.

A Culture Assignment Enhanced

STLR training provided me with the opportunity to redesign an existing assignment used to promote global and cultural awareness and understanding in my Elementary I Spanish courses. It enabled me to align the learning activity even more closely with course learning objectives and render it more meaningful to students. I had first developed the assignment for use in my campus-based Elementary I courses and later used it in my online sections of the same course. I also shared the assignment with other online instructors by incorporating it into the online course template I designed.

Culture activities like the one I planned to reimagine for this STLR initiative serve the purpose of increasing learners’ awareness of topics of local and global cultural importance. As such, assignment questions were initially crafted with the intent of fostering critical thinking about complex and controversial cultural topics, promoting an open mindset, challenging personal beliefs and biases, and promoting reflection about how we engage with others and our communities. Learners’ responses to assignment reflection questions indicated the success of the assignment in increasing their awareness of various facets of complex cultural and global issues and, in some instances, indicated a transformation in their worldview. My grading experiences with the activity and the information I had read about the STLR initiative led me to believe that the assignment would be particularly well suited for STLR tagging. Insights gained through STLR training validated this thought and provided me with the tools to transform the assignment to align with STLR tenets.

Time spent attending STLR training sessions and subsequently applying principles learned in those meetings afforded me dedicated time for reframing assignment reflection prompts and grading criteria in a way that would make the relevance of the assignments and the course to students’ daily lives more readily apparent to learners. Helping students discern the relevance of coursework potentially increases their dedication to engaging with course content, renders the course and it assignments more rewarding for learners and the instructor, and increases the overall value of the learning experience.

STLR certification also allowed me to enhance and refine my feedback practices for this assignment and others. I already routinely provided thorough, relevant, and meaningful feedback to students on course assignments, but the examples of feedback and rubrics provided in STLR training materials inspired new ideas for, and offered guidance on, ways to increase the value of feedback for students. In addition to comments specific to question responses as they affect grade outcomes that I have always provided, I have been able to leverage STLR tenets and concepts to offer feedback that includes actionable items to promote learners’ future intellectual growth and community engagement that are specific to the interests and experiences individuals express in their assignment responses. STLR principles and grading processes bring efficiency to this effort to offer an increased level of feedback customization.

The integration of STLR into my courses has served to increase the value of the learning experience for students, increased learners’ motivation to complete and do well on the assignment, and provided a more standardized method for assessing learners’ transformative growth resulting from the assignment.

Assignment Details

My STLR assignment reworked an existing culture activity that requires students to read an article describing the experiences and challenges of immigrants to the United States and their path to citizenship from the perspectives of an immigrant and those who work to help immigrants navigate legal and social dilemmas. For this assignment, students write a summary of the article and respond to reflection questions related to the article and the immigration issues it explores. The questions encourage learners to share personal perspectives on this complex and controversial topic at the center of many debates in students’ local and global community. They also prompt students to consider the topic from immigrants’ viewpoint, examine their own personal beliefs about immigration, and articulate their thoughts and opinions about this complex, multifaceted, and often divisive issue.

The reading and the accompanying assignment reflection questions challenge learners’ belief systems, increase students’ awareness of the struggles many immigrants face, and stand to promote a deeper understanding of and greater appreciation for the unique challenges inherent in the immigrant experience.

The article for this assignment presents multiple perspectives on the immigration issue. The reading and the accompanying assignment reflection questions challenge learners’ belief systems, increase students’ awareness of the struggles many immigrants face, and stand to promote a deeper understanding of and greater appreciation for the unique challenges inherent in the immigrant experience. The activity prompts students to think critically about what they hear from news outlets and others in their community about this increasingly complex issue and consider the broader consequences of rhetoric surrounding the immigration debate. Moreover, it encourages learners to form and express their own informed opinions and has the potential to prompt them to engage in informed, responsible, and appropriate civic actions that may advance civil debate and conversation on the topic and empower positive solutions to various facets of the issue.

Scaling the Assignment to Other Sections

The process for integrating my first STLR assignment into online Elementary Spanish I and II courses that I designed and coordinated, but that were taught by multiple additional instructors, proved effective and efficient. These courses form part of UCO’s core curriculum and online sections of the offerings comprise approximately 33% of UCO’s Elementary Spanish courses in fall and spring and approximately 75% in summer. This effort to integrate a STLR-tagged assignment into all sections of online Elementary Spanish I and II expanded the impact of the assignment by affording more students than those in sections I teach the opportunity to participate in this STLR-tagged learning experience.

This implementation strategy also expedited the integration of a STLR-tagged activity into all online Elementary Spanish courses and introduced STLR training to all instructors of those sections. At the time of this initiative, the majority of the online Elementary Spanish instructors had not been STLR certified and did not use a STLR-tagged formal course assignment. To establish a timeline for the initiative while allowing instructors time to plan accordingly, I asked all who had not previously completed the training to do so before the spring 2020 semester. They each completed it in the summer of 2019, enabling us to launch the initiative and implement the assignment in all fall 2019 sections of the courses.

Although we standardized the assignment and evaluation information across all online sections of these courses for the initial STLR assignment, instructors completed their own assessments of student achievement. Feedback from instructors indicated that they appreciated the example of an assignment and assistance with the first STLR activity.

Advancing the STLR Culture

This assignment proved successful for meeting the vision I had for it and for promoting student achievement of learning outcomes. The reworked questions promoted better learner responses than those submitted in past semesters and reflected learners’ sincere efforts to provide thorough and thoughtful responses to the assignment’s question prompts. The majority of students demonstrated evidence of integration of the goals of UCO’s Global & Cultural Competencies tenet. Some students indicated interest in civic engagement inspired by the assignment and reflection activities that, if acted upon, could provide evidence of attainment of STLR transformation credit.

The reworked questions promoted better learner responses than those submitted in past semesters and reflected learners’ sincere efforts to provide thorough and thoughtful responses to the assignment’s question prompts.

Moving forward, instructors for all sections of the courses will create their own STLR-tagged culture activities. They may use this assignment as model to work from for redesigning and adapting the other existing culture assignments in the courses or creating entirely new ones. Some instructors requested the option to continue using the article assignment created for use in all courses in the short term, but will develop their own assignments over time. This diversity in STLR culture activity topics across sections promotes academic integrity while ensuring that all learners in the courses enjoy STLR assignment opportunities.

STLR Training Outcomes

STLR training was one of the most productive and worthwhile professional development activities I have experienced. The facilitator used our time well and was knowledgeable, well organized, and helpful, particularly with regard to my questions about how to integrate my STLR-tagged activity into multiple sections taught by other instructors. Information covered in the sessions stayed on topic and proved relevant as I developed the assignment. Materials provided during training advanced my understanding of the criteria to consider when developing STLR-tagged assignments and associated evaluation rubrics and served as a reliable and easy to understand reference during and after training. The D2L resource site for STLR-trained faculty members continues to serve as an invaluable and comprehensive guide for all facets of integrating STLR into my courses.

A Call for Higher Education to Critically Reflect on the Coronavirus Disorientation

Economist Milton Friedman’s statement in an introduction to a reprint of his book, Capitalism and Freedom (originally published in 1962), has been often quoted:

“Only a crisis — actual or perceived — produces real change. When that crisis occurs, the actions that are taken depend on the ideas that are lying around.”

The quote can clearly connect to the current COVID-19 pandemic and the crises it is producing. But one wonders whether “the ideas that are lying around” include the kind of thinking that produces generative change as an outcome of these crises.

As faculty and students adapt to the pandemic’s forced at-distance modalities of teaching and learning, many in higher education are asking whether real change in how colleges and universities operate will result from the pandemic. In the short-term, real change was forced when institutions scrambled to convert face-to-face classes to at-distance delivery and dealt with implications such as the sudden loss of students’ ability to interact with each other on campus, among many other disruptions.

graphic of covid-19 viroid particle with question-mark overlaying

In the long-term, the pandemic will cause financial and operational disruptions. At-distance teaching may be the only kind of teaching possible even into the 2020-2021 academic year.

In Transformative Learning (TL) terms, higher education has been suddenly and dramatically forced to confront a disorienting dilemma.

TL’s foundational theorist, Jack Mezirow, counted disorienting dilemmas as one of the key experiences leading to transformation. Such a dilemma forces the learner (in this case, higher education writ large) to reflect on how and why the new situation does not fit into the learner’s existing worldview. Then, a perspective shift or expansion must occur to accommodate the new reality, thereby launching a revised worldview.

For higher education, the existing worldview — even in the face of increasing pressures in recent decades — has stubbornly remained the residential learning experience on a college campus where students attend classes and accumulate 3-credit-hour chunks of learning based on seat time and successful exam performance until they complete a prescribed program that qualifies them for graduation. Then the pandemic barreled into higher education like a tsunami, and almost overnight the prevailing worldview became extremely fragile.

But TL also offers a solution, exactly as described above: disorienting dilemma (i.e., disruption), followed by reflection, followed by the adoption of a new worldview and subsequent action taken based on the new perspective. Applying this process to higher education can be refined by addressing the challenge as a learning organization.

Peter Senge, in The Fifth Discipline: The Art and Practice of the Learning Organization (1990), counts mental models (“worldviews” in our discussion) among the disciplines of a learning organization (pp. 6-10):

  • Systems Thinking
  • Personal Mastery
  • Mental Models
  • Building Shared Vision
  • Team Learning

From Senge’s list, “mental models” includes necessary reflection to determine what the mismatch is between the reality and organizational worldview. For higher education administrators, trustees, and others, the pandemic has forced reflection about the definition of a university. Because the old definition (the existing worldview) is no longer possible, then reconciling that perspective with what is possible requires adjusting the current definition in order to expand the perspective.

For higher education administrators, trustees, and others, the pandemic has forced reflection about the definition of a university.

Those wedded to the existing definition of a university are frozen in hope-space — that tenuous existence without agency and totally dependent on the old reality somehow returning. But those whose reflection yields a new definition for “university” are far better equipped to take action.

Assuming those of us at our own university have adopted a new definition for what we are right now, then building a shared vision around that new reality is the next important step. Senge (p.9) says it’s necessary to unearth “shared ‘pictures of the future’ that foster genuine commitment and enrollment” to succeed in this process.

As we work at distance through coming weeks and possibly months, personal reflection about our own worldview of higher education can be a productive activity by prompting an expanded definition of a university. Our resulting new worldview can accommodate the current reality while at the same time motivating us if we tap into the reasons we came to higher education in the first place.

 

Reference

Senge, P. (1990). The fifth discipline: The art and practice of the learning organization. New York, NY: Doubleday.

Three Techniques for Using Micro-Credentials to Engage Learners

Think of a time when you completed a college course, a webinar, or a workplace seminar…

There were most likely records of your attendance and completion…but…how were you able to communicate the skills you gained with others?

Did you receive a paper certificate? A standard transcript?

Micro-credentials are gaining popularity in education and workplace settings as a promising strategy for helping people demonstrate their knowledge, skills, and abilities to others.

This article describes the purpose of micro-credentials and explores three techniques for using micro-credentials to engage learners.

What are micro-credentials?

There is some level of confusion around the definition of a micro-credential.

Micro-credentials are becoming more common as a method for helping people demonstrate their knowledge, skills, and abilities in new and more visible ways. Digital Promise defines micro-credentials as “digital certifications that verify an individual’s competency in a specific skill or set of skills.”

graphic of 4 people standing at a map with a large computer with digital badges on screen

One of the more popular methods of issuing micro-credentials is through digital badges. A digital badge is an electronic representation of achievement from a pre-defined set knowledge, skills, or abilities.

Digital badges can be posted on social media platforms, linked on websites, or displayed on résumés. They provide a level of evidence that a person claiming they have a set of skills has demonstrated the claimed competencies in a verifiable way.

One note of particular importance is the distinction between curriculum and the micro-credential. A curriculum program or offering is primarily concerned with helping people build their knowledge, skills, and abilities. This can be accomplished through classes, workshops, readings, webinars, or any other type of instructional strategy. In contrast, a micro-credential is primarily focused on verifying and communicating the achievement of knowledge, skills, and abilities independent of how they may have been acquired. This allows micro-credentials to be a strong signal for verifying that a person does indeed possess the competencies they are claiming.

Engaging Learners Using Micro-Credentials

Micro-credentials present an important opportunity to enhance engagement as part of an overall learning strategy. The following list describes three options for using micro-credentials to support engagement.

1. Recognize Learning Milestones

Learning is an ongoing process that can take on many formal and informal paths. Micro-credentials allow learners to recognize key milestones along a learning journey.

Take earning a college degree, for example. This process can take many years with relatively few opportunities for recognition of skills along the way. Micro-credentials can help learners demonstrate and recognize their achievements as they are earned instead of when the overall degree is awarded.

This enhances motivation and increases opportunities for the learner to benefit from investments made throughout the education experience.

2. Create Positive Challenges

Micro-credentials can provide learners a positive challenge, or goal, to pursue.

For learners who are motivated by pursuing clearly defined goals, micro-credentials can outline a pathway and create a positive sense of challenge to achieve a defined objective. A micro-credential can create a destination that highlights opportunities for growth and obstacles that can be overcome through learning and growth.

3. Build Capability “Stacks”

We grow our competencies and capabilities every day. When we learn a new skill, that skill augments our existing complement of capabilities.

Micro-credentials allow people to visualize their unique combination of knowledge, skills, and abilities. Opportunities for future growth can also be identified by seeing how an existing set of knowledge, skills, and abilities, might be augmented through growing new capabilities. This allows us to “stack” our knowledge, skills, and abilities together to create a unique mosaic of competencies and capabilities.

Concluding Thoughts

Micro-credentials are emerging as a new way of recognizing learning achievement. With thoughtful planning and design, micro-credentials can be used to help people share their expertise with others as well as increase engagement in transformative learning experiences.

For more information and help with creating your own micro-credential, contact the Institute for Learning Environment Design.

 

Additional Resources

Where Do Graduates Live After Graduation?

graph showing where college graduates live by institution type (online, elite, community college, and state university)

From: Sentz, R., Metsker, M., Linares, P., & Clemans, J. (2018). How your school affects where you live: Where do graduates move after college? Ivy League grads move to the big cities. Community college grads stay close to home. But what about everyone else? Report from Emsi. Available at https://www.economicmodeling.com/how-your-school-affects-where-you-live

Transformative Development Necessary to Counter Technology’s Side Effects

4-year-old child (in a demanding voice) to ubiquitous electronic assistant: “Alexa, play music I like.”

Alexa: “Can you be more specific?”

4-y-o child (angry voice): “You’re stupid, Alexa. Don’t you know anything?”

Alexa: “I will try some music you may like.” (Tune from “Sesame Street” begins playing.)

4-y-o child (angry voice, now screaming): “You stupid, stupid person, Alexa!”

In her article, “The Challenge of Scaling Soft Skills” (2018), Lynda Gratton briefly considers why soft skill development, such as how to interact well with other humans, is now and will continue to be so very important. She also considers some reasons why soft skills seem to be in short supply. The hypothetical exchange between a child and Alexa may be one reason technology giveth, yet it taketh away, too — sometimes with a vengeance.

Artificial intelligence and technological replacement of tasks that used to require humans to accomplish will grind forward. There are studies that put numbers to these things, but the basic tenet seems obvious. Tom Freidman (2018) shared a couple of examples, quoting education-to-work expert Heather McGowan:

In October 2016 Budweiser transported a truckload of beer 120 miles with an empty driver’s seat. . . . In February 2017, Bank of America began testing three ‘employee-less’ branch locations that offer full-service banking automatically, with access to a human, when necessary, via video teleconference.

One ramification of spreading artificial intelligence (AI) and robotization is that the tasks machines can’t do become that much more valuable in human employees. Yet the Alexa-child interaction imagined above might give pause regarding the child’s development of empathy, which is pretty high on the list for many employers when seeking new hires. As Gratton says, “. . . machines are generally poor at understanding a person’s mood, at sensing the situation around them, and at developing trusting relationships” (2018).

Humans, however, can learn to do these things. It’s a great irony, though, that technology (which cannot produce machines to do these things) can actually make it more difficult for humans to learn these key “soft skills.” Would the child in the scenario above be as rude to a librarian when asking for help (if the child is asking a human librarian and not the robot at the circulation desk)? One hopes the child will have learned to be a better human, but enough Alexa-training and a concurrent dearth of human interaction can make you wonder.

Are the sweet technological advances resulting in the cell phone causing detrimental trickle-down effects on developing humans’ abilities to interact well with others, to identify others’ moods based on facial expression and body language? Maybe we are becoming a society not so good at empathy and the ability to relate to others yet at the same time able to select the perfect emoji to attach to a text message.

Technology giveth; technology taketh away.

Gratton piles on some more in her brief 2018 article about pending technological dangers to the workplace and society. She points out the effect stress has on learning.

Imagine you’re a new employee. Your soft skills are lacking. In that scenario, you’re going to be stressed, maybe even very stressed. That kind of stress lowers your ability to learn to read others’ intent, needs, and mood. Now your ability to succeed as a new employee by learning fast is compromised.

If you don’t come into the job with being-a-decent-human skills already developed, you may be in trouble in that job. Learning to be a decent human via on-the-job training is probably more difficult than learning a technical skill required on the job. After all, a successful learning curve for this kind of development generally takes years as children grow into adulthood.

But imagine what Transformative Learning (TL) could do as a disruptor to the bad disruptions technology is causing. In the fight between AI and TL, at least in an arena where humans are the players, TL should triumph for a pretty basic reason: TL helps humans reflect on relationships and their own development.

When guided and facilitated by someone who knows how to scaffold the occasional disorienting dilemma and then prompt for the kind of reflection that expands empathic understanding, humans develop. In particular, they develop the kinds of skills that robots and AI, at least for the foreseeable future, do not possess as members of a functioning team.

Technology giveth; technology taketh away; TL giveth back.

 

References

Freidman, T. (2018, January 17). While you were sleeping. The New York Times, p. A19.

Gratton, L. (2018, August 6). The challenge of scaling soft skills. MITSloan Management Review. Retrieved December 27, 2019, from https://sloanreview-mit-edu.cdn.ampproject.org/c/s/sloanreview.mit.edu/article/the-challenge-of-scaling-soft-skills/amp