Think Before You Trust: Why College Practitioners Must Scrutinize Research Claims

(A. Solano)

As higher education faces external attacks, we must also confront self-inflicted wounds. I’ve never shied away from calling out unproductive practices—and I often pair that with practical resources. As a fierce defender of community colleges and broad-access institutions, it's time to sharpen our critical thinking even more.

That said, countless college practitioners have experienced the negative impact of flawed research firsthand. That’s why I continue to urge practitioners: Do not take research about your sector at face value. Ask critical questions.

-What is the exact methodology?
-What’s the sample size, and is it truly representative?
-Are the research limitations clearly defined and acknowledged?
-Is the data analyzed through a single, narrow lens?
-Are findings or frameworks from large universities or small liberal arts colleges being applied—sometimes inappropriately—to community colleges?
-Is the research relying on superficial methods (e.g., word counts on compliance documents) to make sweeping assumptions about your practice?
-Is the research truly peer-reviewed by independent, critical thinkers from various fields, or is it reinforced by a theoretical groupthink lens?

A red flag: If you, as a practitioner, finish reading a research article or listening to a research presentation about your sector and think, "I have no idea what to do with this information."

Rewarding mediocre research practices is a disservice to the field and an insult to scholars who rigorously uphold high standards. Those dedicated to producing high-quality research take great care to cite, ensure integrity, and a commitment to doing no unnecessary harm to colleges. (Not to mention those who also charge exorbitant consulting fees to cash-strapped colleges—delivering little to no meaningful outcomes in return. For example, $95K for three workshops and six hours of racial equity training.)

Higher education and DEI are under attack. We cannot afford to passively accept research that misrepresents or undermines the precious community college and broad-access institution sector—especially in today’s climate, where malevolent actors are quick to weaponize information against us.

Stay sharp. Stay critical. Onward.

***

Also visit: The Policy, Research & Practice Disconnect