Gateway Course Placement with Dr. Melinda Karp

LISTEN TO THE EPISODE:

Learn about a study of placement as a catalyst for institutional transformation.

In this episode, I interview Dr. Melinda Karp, Founder & Principal at Phase Two Advisory, and former Assistant Director at the Community College Research Center, Columbia University. It's great to have her back! Check out her other episode focused on holistic student supports. 

Phase Two Advisory's library of resources.

Placement-As-Transformation

New Placement Approaches in Broad-Access Institutions

(Scroll down to access the transcript.)

We discuss the following topics:

07:22 Goals of the gateway course placement study.

13:37: Gateway course placement study methodology.

23:57: Study findings.

32:54: Navigating institutional change.

51:39: Why shared values matter and the importance of being practical about equity in the work.

Select Dr. Karp quotes:

"The study had two goals. To find the renegades who are playing with the next frontier of placement. The other goal was to understand if it is possible to use placement reform to push transformation of an institution."

"Historically, developmental education reform has been sort of three buckets. One was placement reform. The other, curricular reform. And the third was the pedagogy. What we're finding is that you really no longer can do that. Like most institutions in the vanguard, they are combining placement with something else to the point where we have some colleges that don't even do placement anymore."

"We were able to come up with three categories of innovation. One of them takes multiple measures and pushes it. We call that a reflective algorithm. Typically, something like a grade point average coupled with a test or coupled with a student's course taking that in some way through a formula or an algorithm tells us where students will end up in the curricular sequence. These reflective algorithms go a bit further in that they integrate some version of student reflection, whether that's a student survey on basic needs, whether that's a student providing their own assessment of how confident or competent they feel about doing coursework. Those self-assessments are combined with a breakpoint average where traditional multiple measures, data and an algorithm then spits out a student's placement. 

The second bucket is what we call guided self-placement. Students are asked to think about their grades, their experiences with English and math, their access to things like computers and wi-fi. But then the college gives them information about the courses that are available to them. The students are provided with the opportunity to self-place given the guidance the institution provided. So it takes that algorithm and flips it. The students get to make the choice, but it also flips the onus on providing that information guidance onto the institution. 

The final version, and this is where we really found our radical innovators--and I will say we only have three of those in our sample--were what we called diagnostic just-in-time support. Essentially, what these colleges do is provide universal access to college credit courses. Not even necessarily co-req, but college credit courses coupled with an intentional provision of just-in-time support to all students."

About Dr. Melinda Karp
Melinda is a nationally recognized expert on improving students’ transitions to college and supporting them once there. She founded Phase Two Advisory after nearly twenty years conducting research and working with colleges on education reform as Assistant Director at the Community College Research Center, Teachers College, Columbia University. The proud granddaughter of refugees, Melinda works with national and institutional leaders, campus-based faculty and staff, and philanthropists to ensure that all individuals have the opportunity to realize the intergenerational mobility higher education provides. She frequently writes, speaks, and comments on education reform at professional meetings, in the media, and in academic forums. Melinda holds a BS in human development and family studies from Cornell University; and both an MA and a PhD in sociology and education from Columbia University. She chairs the Effective Advising Practice Guide Panel for the Institute on Education Science’s What Works Clearinghouse, and she is a member of the inaugural editorial team for the Journal of Postsecondary Student Success.

About Dr. Al Solano
Al is Founder & Coach at the Continuous Learning Institute. A big believer in kindness, he helps institutions of higher education to plan and implement homegrown practices that get results for students by coaching them through a process based on what he calls the "Three Cs": Clarity, Coherence, Consensus. In addition, his bite-sized, practitioner-based articles on student success strategies, institutional planning & implementation, and educational leadership are implemented at institutions across the country. He has worked directly with over 50 colleges and universities and has trained well over 5,000 educators. He has coached colleges for over a decade, worked at two community colleges, and began his education career in K12. He earned a doctorate in education from UCLA, and is a proud community college student who transferred to Cornell University.

 

Podcast Homepage

AS [00:00:48] For today's podcast, it's a pleasure to have Dr. Melinda Karp, again, at the Student Success Podcast. Melinda is a nationally recognized expert on improving students transition to college and supporting them once there. She founded Phase Two Advisory after nearly 20 years conducting research and working with colleges on education reform. As assistant director at the Community College Research Center, Teachers College, Columbia University. The proud granddaughter of refugees, Melinda works with national and institutional leaders, campus based faculty and staff and philanthropists to ensure that all individuals have the opportunity to realize that intergenerational mobility higher education provides. Melinda holds a B.S. in Human Development and Family Studies from Cornell University, my alma mater too, and both an M.A. and a Ph.D. in sociology and education from Columbia University. She chairs the Affective Advising Practice Guide Panel for the Institute on Education Science's What Works Clearinghouse, and she is a member of the inaugural editorial team for the Journal of Postsecondary Student Success. Welcome back again, Melinda.

Melinda Karp [00:02:00] Thank you. Go Big Red. I'm so glad to be here. Nice to be here. And actually spent three weeks up in Ithaca this summer and thought of you a little bit, Al, as were wandering around campus.

AS [00:02:13] That's so cool. Well, you know, my next question is going to be. If you wouldn't mind sharing a story, a hobby or anything beyond work. So yeah, I'd love to hear about Ithaca, if that's part of the story or anything else.

Melinda Karp [00:02:28] Well, I think it would be that it can be, Yeah. Yeah. We like to head out there and sit by the lake. And actually, one of the things about outside of work is the extent to which I get to read when we're out there. And that's one of the things I do in my free time. And as an academic, you know, reading is our job. But I have found that I've been reading really extensively lately well beyond the higher ed literature, partly because it makes me better at my job. And so reading a lot of memoirs, a lot of political analysis, things that aren't technically higher ed, but I think really inform the way, you know, you understand the world. So it's fun and it's interesting, but it shows up in my work sometimes, too. The kids always joke that my books, they don't want to read my bookshelf. All my books are too smarts, which isn't true, but that's what they think. So I'm going to go with that.

AS [00:03:24] Is there a particular topic that's really resonated with you?

Melinda Karp [00:03:28] The memoir, as an innate curiosity, so I love the stories, I find memoir is really useful because so much of what we do is is really about having to understand how to walk in other people's shoes when you can't inherently walk in everyone's shoes. Right. Like, part of what we do as educators is meet students where they are. And if you don't know their lived experiences because the diversity of want experiences are so broad, memoir is really helpful. I think I let people all I want I'm doing a lot of is really thinking about how we got to this political and economic moment. And because I think it helps us understand how to navigate some of the challenges that folks in the higher space have been grappling with. Right? So I read this book, American Resistance by David Rothkopf, which really digs into what do you do when the world around you doesn't like you fully without your values and and it doesn't tactically have anything to do with higher ed, But I find increasingly that story and that thinking about what do do resonates because we are all constrained when we work in public education.

AS [00:04:38] Yeah, we can do a whole podcast just on that.

Melinda Karp [00:04:44] Yes, we could, as well right now. But I've been, you know, reading extensively has I think at least helped me feel like I have some tools from other spaces. And as we work with, I work with the college leaders to figure out how to navigate some of the trickier moments that we've all face in the past couple of years. And I imagine are coming our way over the next 6 to 12 months.

AS [00:05:07] Yeah. You know, I think it's wonderful that you're doing that. Too many of my colleagues, they're researchers in higher ed. It's good that you can become an expert in an area and you can dig deep, but over time it can give someone a really myopic view of how to approach things. And I'm finding that increasingly troublesome. Actually, they're not willing to learn other fields. They're very stuck on, well, this is my theory and I'm going to stick with it and I'm going to be unapologetic about it. But you're being unapologetically unproductive when you're working with colleges, when you only have one view of how things ought to operate. So kudos to you. Thank you for doing that and expanding your mind beyond what all the higher ed stuff we read. An Ithaca. I haven't been there since 1990 [mumbles the remaining years]. How is Ithaca?  Beautiful as ever?

Melinda Karp [00:06:13] Beautiful as ever. It's different. You got to get back. That's old folks. They changed arms a little bit. But I just, you know, I, I don't live on the East Coast anymore, but I'm a New Yorker, and New York State born and bred. And when it's nice to get home, seemingly, you know, we were there in July, which is great. I always joke, I have no interest in being there in February. Anyone knows Western New York or Central New York everyone knows is not the time to go, but three weeks in the summer. And it's great, great food, great wine, the campus delightful.

AS [00:06:46] Nice. Yeah. I got to head back there. So last time you were here, we discussed the holistic student support student success teams, and I shared your guide widely. I even used that at one campus to help them really think through the different types. And so thank you for that. And then you're returning back to the student success podcast because you've done some really meaningful work around placement. So can you tell us a little bit about that journey, the why? Why did you choose placement?

Melinda Karp [00:07:22] Yeah. And thanks for using our stuff and I'm always happy to share with your listeners the resources that we have on holistic student support  and middle leadership. But this project is really about placement and institutional transformation, which is what we do at Phase Two Advisory.  Our working assumption is that our broad access institutions, particularly community colleges, have grown up to have certain structures and ways of doing things that come from a very long tradition, set of at a time with rational reason, right. And now don't work terribly well for many of our students, particularly if we think about this from an equity perspective. And so it's not enough to just reform or improve what we're already doing, although that's important. But it really is fundamental about transforming, like really shifting how we do business, and that comes in lots of forms. And so we were talking with a funder in this project is funded by the Ascendium Education Group. And I want to give them immense thanks for their flexibility and a curiosity in funding this work because they sort of said go out and find cool people doing cool things and talk about it, which is really an immense pleasure as a grantee to have an under with that perspective. And so we were thinking about all of the work that's been done in the developmental education reform states, and it has been immense and it has been extraordinarily successful, right? If you look at the data, many more students are getting into and through their gateway English and math classes than was the case 10 or 15 years ago. In many instances, that's between racial minoritized and or low income students, and their more advantaged years are closing, although they are not loads. It has been perhaps one of the most widespread and successful reform efforts that we have seen in our sector. And, it's not nearly enough, right? Because many, many students are still not getting into them, through gateway courses, equity gaps are not closing. We are starting to see some emerging evidence from some leading states, California and Florida, notably that while gateway course outcomes are improving, collection and transfer are not. So something's happening there, right? Because the goal is to get students through gateway, math and English. Get them into the finish line. So there's sort of this causal and at Ascendium we were just sort of playing with this idea of like, well, what's happening here? What do we do that takes the successes of the developmental ed reform movement and pushes the next envelope? And we really took inspiration from some of the early reformers. If you go back 10, 15 years and thinking about the folks at Long Beach City College and thinking and some other folks who really started putting out this notion of multiple measures and people thought they were out of their minds. People thought that was radical, how could you possibly not place students using the standardized tests or using our high school grades like you have lost it? And these brave reformers said, no, we think this is better. We have the data, we are going to do this and we're going to run around the country. We're going to get other people to try it such that more people can try it or people can test it and we can build the evidence base. And today, multiple measures, so that idea of using GPA or using a combination of grades and of course taking those students is not radical at all. Our thought was, well, who is the next version of that? Like, who's the next radical thing? How do we find them? How do we write about them and sort of seed the field with these interesting things that then can be tested? And if they seem to work, we can try and scale that so that ten years from now we push the envelope even further. So this is really an exploratory study. It was literally go out and find the next wave of people. And we were really focused on placement. Although when we get to findings, I'm sure we'll talk about the ways that placement reforms now dovetail with other efforts. But we were really trying find these renegades that we can talk about to people like you, Al, and to other folks to try it. This second hypothesis around this was that placement reform is not going to be enough. Right. That really and this goes back to these two's notion of transformation placement is great because it improves something you already have, but it doesn't fundamentally change how we do business, right? So placement reforms historically have focused on being more accurate, getting the right students into the right places. They don't interrogate by design. How do we teach? How do we view college readiness? We think about the systems within our institution. And those are the things that I think most of us know that you really got to push on. If we push on pedagogy, it's going to be real hard to change student outcomes because the classroom is where students spend the bulk of their time. If we don't change how we talk to students, if we don't change how we support them,  being more accurate in placement isn't going to be insufficient. Those core functions of the institution has been the most resistant to reform over the past decade or two. And so we had this idea of maybe placement can be bigger than accuracy. Maybe there's a way to use placement reform, which, as hard as it is, is a relatively small  reform as compared to, like, changing all your teaching. Maybe there is a way to use placements to push on these core functions, and we call that placement as transformations. The idea that by changing the practices around how you place students, you might also change the broader functions of the college. And so we had this hypothesis that that might be happening out there. And if we could understand that, it might give us an additional lever to push on the broader notion of institutional transformation that we think is necessary to really get to a place of strong equitable outcomes. So, we had two goals.  I'm just going to summarize, one goal was to find these renegades who are playing with placement, that next frontier. The other goal was to understand if it is possible to use placement reform to push transformation of an institution.

AS [00:13:30] Thank you for giving us that context. Tell us a little bit about the methodology and then we'll will dig into findings.

Melinda Karp [00:13:37] Yeah. So this was an exploratory study so I want to be really clear, we were not really looking at outcomes outside of anything an institution had had on its own. That was not our purpose. What we wanted to know was sort of describing what folks are doing, how they were doing it, what they think the implications might be for their students and our institutions. So we used a technique called scalable sampling, and we set up a set of criteria that sort of said, these are the kinds of colleges we're looking for, right? So colleges that are engaged in something beyond sort of traditional placement. So bigger or different than multiple measures primarily or using a standardized test. We were looking for institutions that had equity in mind, and we were looking for institutions that at least on the surface, were trying to use placement to do something beyond accuracy. So we put out a call to our networks. We just emailed everybody. We probably emailed you, Al, we emailed all of our friends. Put it on, I think at the time we were still using Twitter. We don't do that anymore. We put on LinkedIn and just said, If you know of anyone, send them our way. And then we just started screening institutions. We tried to make sure that we had a combination of colleges in states that were doing developmental education reform at a policy level and some that weren't. We were looking for geographic diversity. We actually were really looking for rural institutions in particular because they are often underrepresented in national studies. But we know that many low income and racial minoritized students live in rural areas too. So we wanted to make sure we captured it in small and rural and urban and all the things you did. And we ended up with 15 colleges throughout the country and 13 states. So that was exciting. We do have some very rural institutions as well as some more urban ones. And then at each college we talked to between 3 and 5 individuals. We talked to the person who was sort of in charge of this reform work. And so that varied a little bit from college to college. We called them our key informants. So they were the ones who kind of knew the whole thing and then talked to a handful of folks who were implicated, as we say, by the change in placement. And that depended a little bit, again on the college's approach. But it was typically a faculty member or someone in advising. In some instances it was folks in their IT office or because they had to rethink how they do all of their back end SIS work. So it really varied, a bit of the placement office, tutoring office, various things like that. We did what we call semi-structured interviews. So about 45 to 60 Minutes with a set of questions that then sort of followed where the conversation went, took notes and then did all of the analysis. And we did a couple of kinds of analysis that we can get into if that's of interest. I don't want to go too far down the logical rabbit hole. But we ended up talking to about almost 50 folks across the institutions.

AS [00:16:29] Thank you for unpacking the methodology. By the way, how novel to actually talk to community college people to get information. I'm seeing this, we're going to go look at compliance documents and count words and make assumptions about people. Thank you for actually taking the time to talk to community college professionals. That's the way to do it.

Melinda Karp [00:16:52] I will say before we go, the one thing we didn't do, and it hurts me a little bit and I would want you future research. We did not talk to students and actually when we get to the findings, you'll see that that is something that feels very glaringly absent from this research. And that was because the study is designed to look at institutional access. But I think it's really critical to acknowledge that most of what we uncover really tries to elevate agency for students. And we didn't talk to students, so we actually don't know how the students, all the things we have to say, like we don't know if the students agree with what the institutions believe. And I think it's important to acknowledge that there was a good reason for not talking to students. But to your point about wanting to really get to the authentic ness of what's happening in a college. It is so critical to talk to practitioners, but the students they know things. The study didn't include their voices.

AS [00:17:47] Yeah, it could be a follow up. Right. But still, you dug in with 15 institutions in 13 states, over 50 people. That's a lot of work. So thank you for doing that. Yeah. I mean, the thing about students.. I'm just having a flashback. So I was a reentry student. I just left the military and we're planners, right? In the military. We just learned to plan so I was like, all right, let me, I want to go back to school. And so I went to the nearest community college just to do recon, if you will, some reconnaissance. How does this work? Right. And then the first thing they told me was, you have to take some tests. I was like, shit. And hadn't taken math in, I don't know, five years and I wasn't that great at it. I was working full time when I was in high school. I actually worked like 40 hours a week when I was in high school. Crazy. But anyway, I just felt like, wow, okay, I have to take tests. Okay. Fortunately for me, I had a little bit of money. I went to some tutor. It was a tutoring center and at the end of the day, I didn't really learn that much. They taught me kind of some test taking tricks. And then I took the placement and I was very, very fortunate to have placed in to transfer level English and math. I was so, so  fortunate. What's really interesting is that I think, yeah, I spent five semesters. I was a spring student. Of all the students that I knew that were not in transfer level English and math because I was very active. I was part of clubs and everything when I was transferring and not one of them made it out. Not one, not one. I remember, this was before, I'm giving you the retroactive student perspective, right? So I remember that this was before Rate My Professor. The Internet wasn't really popping back then. Yeah. I don't even think it was. Yeah, it was early 90s. Right.

Melinda Karp [00:19:55] We didn't have the internet back then, Al. Sorry.

AS [00:19:58] Yeah. But I remember at the Veterans Center there was a list on, on a board and they're called it friendlies. You want to guess what that is, Melinda?

Melinda Karp [00:20:10] Those were the professors to go to who weren't going to be awful.

AS [00:20:15] Yes.

Melinda Karp [00:20:16] Oh, oy.

AS [00:20:17] Yes. So I was like, wait, so these are the faculty who are the ones who care? And I said, well, wait a second. Are you telling me they're faculty that don't care? But it just occurred to me, there's a list. And then when I remember choosing the faculty, I remember I went to one of them that was a math instructor and at office hours  I was just just introducing myself and he was really focused. He said I'm  planning for today. So basically it was lesson planning. So I'm going to put you in groups and then I'm going to do this and I'm going to do that. And then right after class, I have something in the other room for 30 minutes with a student who had an A in this class, and he's going to help you all. And he was basically doing the early version of SI. So the reason I mention all this, because you kind of gave me a flashback here is that if I hadn't been a planner, if I hadn't had the money, there's a good chance I wouldn't even be here.

Melinda Karp [00:21:13] Yeah. Wow.

AS [00:21:14] And the second thing and it really came down to I was able to overcome the the placement hurdle, but it came down to caring, it came down to quality instruction.

Melinda Karp [00:21:27] Yeah. And your story breaks my heart. This is a different study, so I won't spend much time on it. But we also recently did a study on the students about how they plan their courses. And one of the things that they told us was they used rate my professor and they know it's not fairly accurate, but they are very focused on finding professors who are, quote unquote, good. And I think many folks hear that and think, they're looking for the easy 'A.' But what they're looking for is exactly what you describe. They're looking for professors who care. They need to know that they're going to be successful. Our students do not have time to waste. They do not have money to waste on classes that will not lead to actual credit, on time. And so they're looking for instructors who will who are good. And that means that we'll get them to the finish line. Not necessarily easy. But are not going to get in their way. And that's separate from the study we're talking about. But I think it's so critical to hear that, because the assumption you make is good equals easy. And that is not what students are saying. They're asking for this base level of care that you just described.

AS [00:22:39] That's right. And the sad thing, I think you're aware, I, I develop a six step process, inquiry and action. And I bring teams together. I work primarily with faculty. I would say 60 to 70% are faculty. And I take them through this process to have these beautiful a-ha moments about their practice to continually improve. And then sometimes I get in these teams is really fascinating. Our faculty who have a deficit mindset about their fellow faculty. For example, "Oh, well. They like going to her class because she's easy and she grades easy." And I'm like, wait a second. How denigrating is that? That you just made an assumption that she's easy when she works her ass off to continually improve her craft. And when she grades, she looks beyond points. She's really looking for learning. So it's a very, very interesting culture in higher ed still, but I know it took a little detour but that's okay. I think you you'll have that student perspective. I'm sure it could be part of a follow up study. Let's unpack the findings. What did you find?

Melinda Karp [00:23:57] Yeah. So let's start with just sort of what we found in terms of the innovations and we'll get into transformation again if that works for you. So a couple of things sort of at a high level. One thing and this this won't be surprising to anyone who works in the field, historically, developmental education reform has been sort of three buckets of reform.  And maybe institutions and states were doing more than one of those. But at any given time, they were sort of discrete. One was placement reform. One was the curricular reform. Corequisites or pathways changing that curriculum into and through gateway classes. And the third was really the pedagogy, making sure that the pedagogy we're using, particularly in developmental courses, is not the same old, same old students have experienced in field with before. So those three things were sort of separate, even if, you know, some institutions were trying to combinations of those, that those were three buckets of reform. What we're finding is that you really no longer can do that. Like most institutions in the vanguard are combining placement with something else to the point where we have some colleges where they're like, well, we don't even do placement anymore. So it's almost a misnomer to call this a  study of placement because a couple of our colleges don't do placement for anything because they don't have it. So that was interesting. The other thing that we found sort of a high level was on increasingly we're hearing discussions of integrating sub populations of students who have been separate from the developmental education reform. And so, for example, historically, dev ed reform was really focused on credit bearing degrees, seeking programs of study. And so students who are coming in through a GED or adult basic education, students who are an English language acquisition programs often had separate and weren't part of the dev ed reform conversation. This varies a little bit from state to state, but in broad strokes, dev ed reform was really on sort of your quote unquote, traditional degree seeking students. You're starting to hear a lot more discussion about how to integrate, particularly English language learners and adult basic ed or GED students into the dev ed reform space so that whatever change is happening around placement is being used not just for students in degree and credit programs, but actually for GED students or for ELL students. So there's a single door which feels like a really important equity innovation to us because it means that students, regardless of their lived background, have the same access to those pathways. The final thing we were surprised by was the extent to which we weren't finding innovation. So we found colleges that were doing this, but it was harder than we thought. Our sense is that many places are institutions are really just doubled down on doing multiple measures, which is important. I don't wanna denigrate that. But that is a huge important step away from a single test. But there were fewer folks trying to go beyond that than we had expected. We have some hypotheses as to why. I don't know that any of them are particularly good, but it it took us a lot longer to find 15 innovative institutions that we expected. And I think, again, if you think of reform and change as sort of an arc or a continuum, I am super glad that so many institutions have moved towards multiple measures. I am interested in the fact that many institutions, they think multiple measures is the end because if you look at the data, it's a really important intermediate step. I'm not convinced it's where we want to end. And so that was just something, you know, for all of you, who want to be renegades like there's a lot of renegades in the renegade space. And so those are sort of our high level takeaways in terms of those 15 innovating institutions. We were able to come up with three sort of categories of degrees of innovation. One of them sort of takes multiple measures and pushes it. And we call that a reflective algorithm. Essentially what this does is it takes the notion, and I'm making the assumption that most of your listeners are familiar with multiple measures, but if you're not a multiple measures approach, essentially says we're going to look at more than just a test or it's typically something like a grade point average coupled with a test or coupled with a student's course taking that then still in some way through a formula or an algorithm sort of tells us where they're going to end up in whatever the dev ed curricular sequence or sequence of college offers. These reflective algorithms go a bit further in that they integrate some version of student reflection, whether that's a students survey on basic needs, whether that's a student providing their own assessment of how confident or competent they feel about doing coursework. Often students are asked to just look at some sample problems and say like, Hey, you know, do you think you could do this? Do you think you could do this to some? How do you think you would really struggle? And then those self-assessments are combined with breakpoint average where traditional multiple measures, data and an algorithm then spits out a student's placement. So it's allowing a bit more of the student to come in. But the college is still maintaining a bit of control over where the students like. The second bucket is what we call guided self placement and in higher ed terms, like everybody's got their own definition of that it's and directed self placement student directed self placement. We're going to call it guided self placement because what we are finding in these institutions is that this is an opportunity for a college to restructure how students are placed without letting it be a free for all. And so in this system, what we call guided self placement, it's often very much the same data as used in those reflective algorithms. Students are asked to think about their grades, their experiences with English and math, their access to things like computers and Wi-Fi. But then the college gives them information about the courses that are available to them. And then the students are provided with the opportunity to self-place given the guidance the institution provided. So it takes that algorithm and flips it. The students get to make the choice, but it also flips the onus on providing that information guidance onto the institution. The final version, and this is where we really found our radical innovators--and I will say we only have three of those in our sample--were what we called diagnostic just in time support. And this is like super varied. But essentially what these colleges do is provide universal access to college credit courses. Not even necessarily co-req, but college credit courses coupled with an intentional provision of just in time support to all students. So that means that students are not coming in as ready or college, not college ready, and only the quote unquote, not ready. Students are getting support. Anyone can be college ready. Anyone can be not college ready at any given time. And then you get the support you need. And that's done in a couple of different ways. In our sample, we have one that used to boot camp. We have one that used a traditional Co requisite, but they didn't start it until week three of the semester, which gave the students the opportunity to figure out if we thought they needed the support. And then we have one really radical one that did all sorts of wild, really cool braiding with their adult basic education. So, that bucket is both the most diverse and the most radical, but those are the three kinds of categories these found innovation within the licenses.

AS [00:32:04] And I find that, and my sample size is is rather small, from my experience as a coach, I find that the third one. You have to have the coalition of the willing. You got to have at least two to math faculty, two English faculty to even get something started like this. What's universal access, because incorporating just in time support into their pedagogy is really flipping the way they've been taught math, right? They've been taught math and English in graduate school and often in very antiquated ways. And this is why I don't have a faculty deficit mindset, because they tend to do the things that were done to them and they don't know any better. So they do the same things to their students.

Melinda Karp [00:32:54] Absolutely. These are the most transformative. You can't do this just in time support, as you just alluded to, without really fundamentally shifting how people do their work, how they structure their day, who they engage with, how you view students and yourself like you, you kind of have to transform the institution and you have to be very brave to do this just in time work. Because the idea that everybody's college ready and nobody's college readily. Just that philosophical underpinning. That is not the paradigm we have worked under for many, many years.

AS [00:33:31] Yeah, and I go back to my experience as a student here, as a faculty member who was taking the time to lesson plan to figure out how he was putting us into groups, how he was going to teach, how he's going to provide support. Right. And I hadn't done math in five years. And I was a C-plus student in math, and then I passed it.

Melinda Karp [00:33:52] Yeah. Yeah.

AS [00:33:54] Right. So it's the pedagogy and the caring really does matter. Now, here's what I find, I don't know if you found the same thing because it's usually a small coalition of the willing and it takes time to bring more people on board, even with the coalition of the willing. We find often that the course success rates decrease. So let's say that we're at 70% and now they went down by ten, 15 points. But the throughput rate is still much better than putting them in 2 or 3 classes below transfer level. I mean, significantly better, right? So I have to remind some faculty because they say, well, yeah, but look at the data. Of course success rates are going down. I said, Yeah, but you know what? There's more students, especially students of color, who are completing English and math than ever before. So what we got to do is go back to that 65, 70% and not just takes continually working on your craft, right? So yeah, I find that third one to be the most difficult one. And course, success rates tend to take a dip, but throughput rates are fantastic. Are you finding the same thing?

Melinda Karp [00:35:09] So not entirely. Yeah. And I will say to you, we were pretty bumbed that most of our institutions didn't have great data yet. And part of that was we collected the data about a year ago, a little less. Many of these innovations were put in place in part either due to or in conjunction with the pandemic. So they were very new, which meant that they didn't have a ton of data to go on and the data they did were and pandemically colored, which, you know, makes it real hard to interpret. But these just in time colleges, at least one of them was and this is our most radical, this one that was mushing adult basic ed and all the things they were finding that they were having very strong success because it was such tailored support. Right now, this one you couldn't do with just the coalition of the willing. Our just in time support institutions. At least two of the three really needed broader base support because it is so radical that you can't only do this in a section or two. If you're asking all your adult basic instructors to push into your gateway courses, that's way more than than coalition of the willing. And that is an interesting conversation. But because of that, the students were getting what they needed when they need it, which means that you can, and you have data systems to do that, which means you can touch many more students before they fail. So they were not really seeing a dip in course completion rates. And what was interesting is that this institution and their and the one I'm probably most obsessed researchers don't admit it, but we get obsessed with certain parts of our datasets. And this is the college that I'm currently obsessed with because they're doing such interesting, radical things. This was an institution that had scaled their placement for all the way to their very large prison reentry program. So their prison reentry students were also going directly into gateway courses with pretty intensive just in times support. When I talk about expanding sort of the envelope of who's included institutional change, that's what we're talking about here. And they were seeing really good success. And again, not every prison reentry student was successful from the get go because we know that typically students in those programs are going to come in with a lot of challenges in and out of the classroom. But they were getting what they needed when they need it and seeing strong success. And they didn't have to leave this institution, at least as we historically have known it. What they did, just to put a finer point on it, is they essentially took their adult basic ed, and this is a state that adult basic ed is offered through the community colleges but offered through different funding strains. They took those faculty who are traditionally really taught around pedagogy and used them to enhance either through coaching or offering free tutoring or offering workshops and the instruction in traditional gateway courses and other first level courses as well. Which meant that students could get what they needed when they needed it. Right. And they leveraged their adult basic education. To mean that no one needed to do dev ed, that they would just get support from trained instructors when they need it, which I thought was really cool.

AS [00:38:35] Yeah, that's rare in my experience.

Melinda Karp [00:38:38] Yeah, it's wild. It's. They had to bring in their IT folks. They had to rebuild their SIS to keep track for all of their grant funding. So when I talk about transformative, they really did have to rethink all of how they did business and it has not been easy but they are seeing success and students are not in dev ed.

AS [00:39:02] That's beautiful. So you you've had all these interviews from an implementation standpoint because you know that the devil is and forever will be in the details when it comes to implementation. Did you find a theme of who were the most critical people? To ensure the smoothest transition from planning to implementation.

Melinda Karp [00:39:30] So it's fascinating. So we did not we don't even see a theme in terms of which discipline. So in some institutions it was both math and English and some institutions. This kind of innovation was happening in only one or other disciplines. And I will say we have a number of institutions where it was the math department that was innovating and the English department was resisting, which is kind of goes against like the old axiom in higher ed, that math is the problem but that's not really what we found. That is very campus dependent. We have some really innovative and thoughtful math departments and some really innovative and thoughtful English departments. What we did find actually was that almost uniformly, I think in about 12 or 13 of our colleges, there was some sort of external impetus. That was a forcing measure, whether it was state policy, whether it was the pandemic, whether it was engagement in something like achieving the dream. There was an external impetus that got folks thinking that there had to be a better way. And then it was a matter of getting some folks, you know, as you always say, that coalition of the willing who are the ones that put their necks out. To think about how they would engage the rest of the institution or the rest of their department. And so, even beyond the usual, like you have to communicate and you have to make the case. We saw a fair amount of, the successful institutions, and granted, they were all successful because they all did this. But there is a throughline, the leaders really started with individual and institutional values and priorities and really making the case not just that, you know, our data aren't great, but that this change, this integration of student agency, this integration of acknowledging that students have strengths that they bring with us is aligned with our values and figuring out what are the values of both the institution writ large, but also those individuals with whom they needed to like needed to get them on board to really think about like, how do we start with shared values and build from that? And I thought that was a really important insight that I don't see a lot in change literature, right? We hear a lot about communication data, you know, using data to make a case. But this idea that our values are about students and so we're going to do this for for whatever that looks like, whatever sort of category name we're doing because it aligns with our values. And let's talk about that. And I thought was really an interesting and important throughline.

AS [00:42:05] So transformation, you mentioned at the beginning of a podcast that placement can serve as a catalyst for for transformation, but not, it's not in and of itself. So can you unpack a little bit about that theme of transformation?

Melinda Karp [00:42:22] For sure. So we had a hypothesis. I want to be clear. We didn't know for sure. We had a hypothesis that you could use placement as almost like a sneaky way to get the other stuff to get to transformation, that there's a way to do placement that's focused on accuracy and that is really important. But there's also maybe a way because, when you go to a college and you're like, we're going to change the way everybody teaches. But that's terrifying for a lot of folks. And you get resistance. And that's not the way to honor the work that's being done. So our hypothesis was that if you enter the discussion around institutional transformation with something perhaps a little less scary, like we're just going to change this, then there might be a way to get to those other core functions. How do you support students, how to teach them? We didn't know. It was just our hypothesis, and part of that was really our funder was like, well, we know we need to get to these core functions. And, you know, we haven't been able to crack that but yet. Can you help us see if this is a way? So one of the things that we did was we sort of built a definition or a set of indicators of what it would even mean to be transformative, because that's like a big, gnarly, amorphous word. And what we were really trying to get out was to what extent does placement move beyond this notion of accuracy to start thinking about placement as a way to address equity gaps and recognition barriers, as a way to get the college to change its underlying structures or the behaviors that folks engage in on campus, or those norms like how do you use students and or the extent to which placement has what we think of as like tentacles. So, placement can be a thing that just happens in the testing office or it can have far reaching impacts into all sorts of departments. I alluded to like IT and the SIS earlier, and so we were like, if that's happening, those are indicators that placement can and is being used for something more than a reform to the placement process. It's about transformation. So we had no idea. We tried to find colleges that looked like maybe they were doing this, but you don't really know. And it turned out that out of our 15 institutions, eight of them actually were using placement in a way that was transformative. And seven were sort of the more traditional. We're going to do an algorithm to make sure the right students are in the right parts of our sequence, which again, I want to keep saying, that's not bad, that's good. It just to me, we're looking at this bigger other thing. What was interesting about this transformation piece, these eight colleges that we're using placement transformatively, is that they were using all different kinds of those categories I went through. So obviously that college I mentioned like placement was transformative for them. But we had a number of institutions that were using a reflective algorithm, which remember is still an algorithm. It just had a little bit of additional kind of data in it. Then a multiple measures algorithm and a number of them were making really big transformative changes off of just that tweaks their algorithm. We actually had one college that didn't end up changing its placement system. They had thought they were going to change placement once they embarked down their discussion of new placement approaches. They realized that wasn't the problem and they were rethinking their curricula. So placement was being used as transformation because they realized they had to transform that change placement. So there's a real wide range of how you get to this idea of placement as transformation. But it is possible to use placement reform to push these other changes, which was exciting for us to find.

AS [00:46:20] So I want to go back to that question I had about key people and implementation. Because, for example, that college that said, well, we're not going to change our placement, but we're going to work on the curriculum. But who made that decision? What body made that decision? Because. Fine, if maybe it's a, I don't know, a five step process for them. But if you don't change placement and you still put students in developmental ed. Even if you have 70%. And just this what I tell math teachers--when you do the math, if you take 3 or 4 courses below transfer level and 70% of the first one pass and then the remaining move to the next one and then 70% pass and move on to the next class and then 70% you're left with a handful of students. So working on changes are fine, but ultimately we got to, so I'm just from a process and decision making perspective. I'm wondering, and maybe you don't know who it is, but I'm just wondering who  who makes those decisions?

Melinda Karp [00:47:36] Yeah, that's a great point. So two things that I want to unpack and what you just said, why most of these institutions with the transformative work, it really was coming from either a department chair and or the department chair like the provost or the process. Right. So those decisions, those conversations are being shepherded by folks with a fair amount of authority, at least within their departmental sphere, if not the whole institution. They also these two colleges that ended up not doing classroom reform in favor of something else. In both of those instances, I do not believe they had very, very long sequences, so they had already changed some particular pieces. One of the institutions had wanted to move to a new placement system and then ended up being stymied by their state policy that insisted that they do a more traditional multiple measures. And so in that instance, what happened with the departure was like, well, okay, then what else? How can I use multiple measures to force these other discussions in my department around normative change, how we use students, how we teach students? The other one was this institution that I mentioned. They were really focused on their English language learners and they had thought that they would try and rethink how they place their English language learners to place them more accurately. What they realized was that for them, placement wasn't the issue. It was the fact that English language learners are completely shut out in the state, from credit bearing sequences. And so they shifted from placement into thinking about how they could take a more biliterate bicultural approach to their curriculum. So they're thinking about things like can they offer key courses bilingually so that students engage in learning English while also earning college credit? So again, placement isn't really the thing, it's what are we doing correct early, but that stemmed from placement discussions. And so you're right, like if you have four levels of pre college credit, stepping away from classroom reform is not going to help. That wasn't the instance in these institutions, partly because we were looking for institutions that had sort of done some of that work by definition.

AS [00:49:45] Got it. Thank you. You know, you should do another study of 15 shitty, I mean, not that good institutions and find out why they are not doing. It's a really fascinating, you know I tell this to a lot of the colleges that I work with because they'll say something like, well, faculty don't want to do this. Or they'll say, you know, administration is not for this. Right. And then. I say, look, we go to committee meetings, and then one faculty, it's usually the loudest one says. You know, I don't want to do this. You know this person's against it, right? Then what happens is people stay quiet. Then they leave the meeting and they all put their arms in there in the air and go. Well, see, now, faculty don't want to do this. Shit. It was one person. One. Same thing happens with administrators, right? One Dean says something in a meeting, well administration don't want  to this, this is one person. So it's a reminder to those listening that when you're trying to make change, be mindful of that dynamic. It happens too much in higher ed. As we wind down here, Melinda, I was wondering, for those that listen to this podcast, they're all over the place, but you have very innovative colleges that are doing remarkable work with developmental ed reform. You have some that are just starting to work, some that are kind of midway there. Given all that rich knowledge that you have from this study, do you have any? I don't know, kind of, more specific lessons learned, like be mindful of this or don't do this and be and do that. Do you do you happen to have some kind of lessons you might want to impart?

Melinda Karp [00:51:39] Absolutely. And I think it goes to what you were just saying about this idea of how we not work around but work with folks who maybe are resistant to change. So part of what we heard is what I said before, which is so much of this is coming at the work from what are our shared values. So, so much of what we heard was people who wanted to do this work, sitting with their colleagues, and instead of taking no at face value, seeing let's understanding where our shared values are and how we can work on those and B, let's understand what you're resisting. So for an example, we have one institution and where you're sitting and let's create problems altogether to get there. So we have an institution in our sample that I alluded to before, they do use a coreq model, but they call it a late start model. And so everyone goes into gateway English at the beginning of the semester, and then at about  week 3 or 4  in partnership with faculty, students are given the opportunity to add the corequisite at that juncture. So they've already experienced the course. The instructor has gotten a sense of who might need more support. And then the coreq starts because what they do is they leverage their college's late start calendar to offer the coreq on the late start, which is really interesting because that they're doing is they're leveraging structures that already exist. When they started this model, the woman who was designing it wanted to start. I want to say she wanted to start the coreq at week 4. In the registrar was like, absolutely, you can't do that. After having some conversations about the why and the shared values and what goes into the late start model and the job of the registrar, they realized that it didn't make sense to start at week 4. Pedagogically it did, but structurally it did it. So they started it week three, they came to a meeting of the minds. Creative problem solving from a place of shared value. And I see so often our institutions are so Byzantine. That we hear no. And we throw up our arms because we don't have the full locus of control that we would like. And so many of our successful institutions did. The tack they took was this idea of let's sit down and talk about it, and B, let's figure out what we can control and not let the perfect be the enemy of the good. But let's come to a solution, right? So that was lesson number one. I think the other lesson. Is really this idea of transformation is a series of small and cumulative changes. And that's why placing this transformation has the potential to be really powerful. So I'm thinking of a different college in our in our sample in a really rural and conservative states that used a reflective algorithm. You can't talk about equity in this state in the terms that we are accustomed to in some other states. Like you just can't. And frankly, the lead  was like, listen, no faculty member wants to go to a meeting and hear how bad their pedagogy is, that they're not equity minded, that they're not doing it right. Like no one wants to hear that. So what she did was when they started to explore this notion of a reflective algorithm, which recall includes things like putting in a survey about what students might have in terms of basic needs requirements or their experiences with English in this instance. She sat with her department. She sat with her colleagues and they went through the survey together. She was like, Well, which questions do you think we should include? And when they say, Well, I want to know if they have access to a computer, she would say, Well, why? Why does that matter? And then back into a conversation about let's talk about basic needs. Let's talk about how you support and teach and use appropriate pedagogy with students who maybe in rural areas you don't have access to stable Internet. So rather than starting from the equity conversation, they started with a what do you mean? And then backed into it and led departmental discussions about equity minded pedagogy that way. She will tell you that as a result, over the past five years, the culture of their department has changed. She was able to institute new hiring requirements. After a couple of years, they started to shift who they prioritized when hiring from, like the professor of rhetoric to people who understood community college students. She was able to institute, early light touch assessments, which we know are good practice over time. She shifted the culture and structure of the department by starting from a place of let's talk about why you care about this in the algorithm and less together, learn how to support our students. So she transformed it not by being like we're all going to transform, but she started in a small space. And I think that story is really resonant to from a really practical, tactical way. You run those departmental meetings, you have that control. You may not control your whole college, but you control what we talk about, Right? You control how you discuss equity in your meetings and how to go.

AS [00:57:07] Yeah, I want to go back to what I mentioned earlier about how some people have a very myopic view of how to approach things. And unfortunately, because they're my allies, we're all part of the same team. I see this too much of my racial equity colleagues who think that they can go to this institution you just mention and unapologetically call them white supremacists. And think that that's going to change them. It doesn't work. And I've gotten shit over the years for not starting first with equity and I'm like, man, you just don't get it, do you? You have to have them do some of the work and have discussions and onboard them to the equity work. The outcome is equity. The approach matters so much and the approach is going to be different in the Central Valley of California, as it would be in in the Midwest, in a rural college, as it would be in San Francisco. In New York. You have to be really practical and smart of how are you going to approach this equity work. And so where you know that the language triggers, you can still get them to do equity work, you just kind of have a pedagogy to it. So thank you for sharing that. I really appreciate that.

Melinda Karp [00:58:21] Yeah. And it comes back to shared values because I refuse to believe that the majority of instructors don't care about all of their students. Right. So if you start with that shared values, you will get to equity in both places. Right. Because it is, to me doing equity work, make sure that is that every student is able to be successful regardless of where they come from or what they earn. And if you come with that shared value and you start talking about pedagogy and see who are our students and what do they need and how do we teach them. That's the conversation that this algorithm or this placement reform enabled colleges to have. So it wasn't going, this is just a technical change to placement. It's why do you need to change placement that opens you up to all of those critical, transformative conversations? Who are our students? How do we teach them? How do we engage with our colleagues in student services? How do we structure our institution Not because placement is broken per say, but because we need to talk about where students are and how we get them to the finish line, whatever that finish line might be.

AS [00:59:28] Right, we can get people to do equity work without them knowing it. And I think that's one of the reasons why there are some states, where legislatively, culturally, they're getting rid of DEIA. And I've work with many of these states. But what I find really, really fascinating because when I work with teams, we dig into the data by department. And what I find is that some of these states like I'm thinking of Florida and it varies, right? So I don't want to make a blanket statement that this is the average, but I find so many programs that where there's anti-DEI I efforts, their outcomes for students of color are better than, let's say, California. You know why that is. Because they do the work. You just do the work.

Melinda Karp [01:00:22] Yeah. And, you know, and I don't want to shy away from the fact that, look, assessment testing has a racist past so we can't deny some of our histories. And I think it's particularly important to go back to what we talked about at the beginning. This moment in time, we have to think about how do you, as you say, do this work? In ways that invite people in because otherwise we will won't get to do it at all. And so this idea of placement as transformation, as placement as a tool or a lever to push folks to think more critically about their institutional structures and practices with equity in mind, how ever a state defines that feels really promising right now, because we have to be honest that there are people who want to do this work who can't say it out loud and the work doesn't end. So the strategies we uncover, the idea that instantly someone said, it's all this sneaky, this, this place and this transformation phrase is a sneaky way to get to transformation. And that's not meant to be like a bad sneaky. It's just it's quieter. And that's okay because we have eight institutions in our sample who quietly created transformation on their campuses nearly by saying, we're going to change how we place students into their classes.

AS [01:01:40] That's right. Just again, just do the work. There's just too much talking. The 15 that you were able to find is because they move beyond the discussions, let's have more discussions. Let's look at data. Well, I need more data. Now I want to know what high schools our students are coming from. Like that matters. What are you going to do with that data? Well, I still have concerns. Well, let's plan to plan and let's do more planning. And you're familiar with my three month rule, right?

Melinda Karp [01:02:14] I cited it all the time.

AS [01:02:16] You may have three months in a year to really do major priority work at a campus. That's it. It's usually October. Maybe part of March and maybe part of April. It's not even three months altogether. Right. So you got to overcome that. But they're doing the work.

Melinda Karp [01:02:32] What are some other tangible findings? I will say one of them is to your point that we don't have a lot of time and we don't have time for more committees. These colleges did not let a good crisis go to waste. And I hate to ascribe anything good to Covid, but a lot of them, it's Covid. Like they had been in for a long time and then they didn't have choice. And so I think for folks who want to engage to do this kind of work as well, your point about no more committees. Yes. And I would add time to crisis that you can leverage, because that really is the impetus in many places to stop talking and start doing because you don't have a choice. Whether that's policy, I hope it's not another pandemic, but something because it was almost every institution in their campus that had an external kick in the rear end that made them move from talking about it to acting.

AS [01:03:28] Yes. And it took a pandemic to get rid of wet signatures.

Melinda Karp [01:03:33] Yeah.

AS [01:03:34] Well, Melinda, if you don't mind, I want to ask you one last thing to tie this up here. I want to go back what you mentioned earlier, what you've been reading outside of the higher ed literature, you're expanding the ideas, the the thoughts, the the writings of other people, of other fields. Is there anything in particular that you want to bring back from exploring all that other literature that ties in to what we've been discussing?

Melinda Karp [01:04:05] Yeah. And I'm not sure the ties in complete because I actually find this study really optimistic in this idea of we can transform even in difficult moments the fact that these institutions can use placement to do bigger things. But I just you know, I work with colleges all over the country. And I am just mindful that it is August of 2024. And as you said in the intro, I am the granddaughter of refugees who fled Hitler's regime. And so I want to go back to that. I think a lot about that book I mentioned, American Resistance by David Rothkopf. And it talks a lot about how do you make change when you're inside a system that feels not right. So he actually writes about the Trump the first Trump administration, and I guess they do call it that now and that there's a chance there will be a second. And it's not about politics per say. It's about what do you do when you are a public official and others and you are constrained in what you can do to fight against something. And I want to name that for many of our colleagues, this moment feels that way. So as optimistic as our study is, I think there are many folks who are feeling really deep down. And I come back to this idea of, do you stay or do you go? Do you fight or do you not fight? What does fighting look like when you can't wave your arms around and scream that you hate everything about what you're being asked to do? Right. And I think it's American Resistance, and I think folks should read it partly because it is just amazing. And where are those lines is how you work within the system and when do you decide that the system has gone too far? One of the messages is at some point there will be a red line and you should know your own values. And so I don't mean to get on a political soapbox. So what I want to tie this in a bow is by naming that some of your listeners may be struggling right now with where their line is. And B, many of your listeners are probably struggling right now with just the constraints of higher ed, and there's a lot to be learned from other sectors around how you can be a good actor even within a system that doesn't always feel good. Our students deserve it. I think, again, the study shows the ways that our students can be supported in lots of different ways. But I do want to name that this is not a great super fun moment to be in an equity minded professional in higher ed, and so I have really taken heart in some of these more political books, not about their politics with a small P, but more about just how to navigate complicated, messy things that don't have clear answers.

AS [01:07:03] Thank you for sharing that. For colleges to achieve student success, college educators need to come together and help each other be successful, and in these kinds of times. So really appreciate you again, Melinda. Really thankful for the study that you've completed here on placement and thank you for the podcast we did on holistic student support. I'm looking forward to having the show notes with links to your resources and everything. Thank you so much for participating in the Students Success Podcast.

Melinda Karp [01:07:42] Thank you for having me twice. It's an honor and it's always so much fun.