[William Engfer - NROC] 13:00:15 Hello, everyone. Welcome, welcome. We're going to give it another minute or 2, just let people filter in. I'm making an announcement right now. If you can't hear us. Please feel free to reach out to our support. Drop a message in the chat. [William Engfer - NROC] 13:00:30 Welcome, everyone. Gonna get started in just a minute. [William Engfer - NROC] 13:01:05 Alright, I'm seeing some more people filtering in. We're gonna give it just another minute or so. Let some people come in. We're making an announcement right now. Hopefully you all are able to hear us. [William Engfer - NROC] 13:01:17 We'll get started in just a minute. [William Engfer - NROC] 13:01:50 All right, well, let's get this webinar started. Thank you all for joining us today for our February edition of our NROC webinar series. We do these webinars about once a month here at the NROC Project, um, and we collaborate with a pretty wide variety of our partners. [William Engfer - NROC] 13:02:08 Today, we'll be talking with Dr. Vance Gray from the Pennsylvania Institute of Technology in Media, Pennsylvania. [William Engfer - NROC] 13:02:16 And the theme for our webinar today is Rethinking Placement, How Pennsylvania Institute of Technology, or PIT, utilizes EdReady to improve student outcomes. Thank you all for joining us. We'll run through a little bit of housekeeping first and foremost, just to go over some logistics, and we'll dive into the rest of the webinar. [William Engfer - NROC] 13:02:36 just to get started here. If you have any questions at any point, please feel free to drop those into the chat or the Q&A box. We'll get to those sort of in the order that they've been submitted. We'll also have a Q&A section at the end of the webinar. So if we don't get to it while. [William Engfer - NROC] 13:02:51 you know, presenting, we may follow up with it at the end. Um… This webinar is being recorded and it will be posted on our NROC.org website. It will also be posted with the presentation slides as well. So if you have any interest in exploring those, those will all be available. You'll receive a follow-up email, most likely tomorrow. [William Engfer - NROC] 13:03:12 Uh, that will include a link to this recording, uh, also with closed captioning as well. So if you have, but also if you have any issues with Zoom as the webinar is going on, please feel free to reach out to community at therock.org, and we'll help you get that sorted out. [William Engfer - NROC] 13:03:30 Alright. First and foremost, as per usual with our webinars, we always love to get to know who's joining us today, so please feel free to put down your name, the institution you're coming from, maybe what your role is in the chat, that way everybody can get to know each other and get a sense of where everybody's coming in from today. So, um… Feel free to please put those in the chat as we get started. [William Engfer - NROC] 13:03:55 I'm gonna dive a little bit into the history of the NROC project and EdReady, our platform, um, just to start off here, this should be pretty quick, but the NROC project was actually founded way back in 2003, so we're over 20 years old, um, as a nonprofit. [William Engfer - NROC] 13:04:11 Uh, originally, we sort of explored the open education resource space as we started developing those open education resources, we moved forward into realizing that we really needed a more robust platform to distribute those resources to students, and to help educators and administrators really get a sense of what students were learning or what they needed help with. [William Engfer - NROC] 13:04:33 That led to the development and launch of EdReady in 2014, so we're well over a decade out from the launch of EdReady now. [William Engfer - NROC] 13:04:43 And that, uh, EdReady is an adaptive learning platform that helps students master math and English skills. Today we'll be specifically talking about how that integrates into the placement process, but in case any of you aren't familiar. [William Engfer - NROC] 13:04:58 Uh, EdReady really starts with an initial diagnostic. Students take that, it takes about 30 minutes, maybe an hour, and then it gives students a sense of where they are with different aspects of material with math and English, right? If they're ready, they need review. [William Engfer - NROC] 13:05:13 They're doing well on it, or if they already have the subject mastered. [William Engfer - NROC] 13:05:17 From there, EdReady builds them a customized learning pathway so that they can work through all the areas that they still need to master, and eventually get up to their target score. We'll talk a little bit about that in its integrated process here with placement as well. [William Engfer - NROC] 13:05:34 Um, this is sort of essentially what a breakdown of that looks like after a student's taken the diagnostic, you can see here that a lot of these areas, they need improvement on, right? And this is… Pretty easily color-coded. You can see with the red, that's where students need still to put in some work to get themselves up to readiness. And yellow still needs review, green, you know, indicates mastery. So, uh, as we get started here. [William Engfer - NROC] 13:06:01 I'm gonna now push it over to our partner, Dr. Vance Gray from the Pennsylvania Institute of Technology, and he's going to talk a little bit more about how they've integrated EdReady into their placement process to really approach doing placement differently. So Dr. Gray, welcome. [Vance Gray] 13:06:23 you know, still giving that, uh, mute button is still an issue for everyone. But, uh, thank you for having me today. I'm really excited to be talking with you about how we're rethinking placement in general at the Pennsylvania Institute of Technology. [Vance Gray] 13:06:40 We no longer look at placement, really, as a technical issue, but it's really an ideological issue, and we really want to make sure that we are doing everything that we can for student success, academic integrity. As many of you are doing at your campus. [Vance Gray] 13:06:58 So we really wanted to do that. And so today I'm going to take you through what I call the seven principles of placement. Like, what do we really do in that instance when we're really thinking about it doesn't happen often at many of our institutions. [Vance Gray] 13:07:15 But where it does, through the governance process, we really need smart thinkers in that area. So I'll ask for the next slide. [Vance Gray] 13:07:27 So here we really want to determine number one, what is the purpose that we do placement or placement tests at our institution. Are we really trying to sort students efficiently? [Vance Gray] 13:07:42 Are we trying to identify what their needs are? Are we trying to see if we're going to use credit for prior work being sent into for students to do at our institution? Are we trying to protect the rigor? [Vance Gray] 13:07:58 Really, for our gateway or our harder courses. So we really want to do that in a way that tells us what the goal is. Like, what problem are we really trying to solve with our placement exam, as we know that's really different. [Vance Gray] 13:08:14 For every institution, or every community, whether you're at an independent college or a private institution, or a community college or four-year institution provided for by the state. So that's really important. So again, it's really about understanding what our purpose is. [Vance Gray] 13:08:34 Like, again, what our goal is. Next slide. So I really wanted to provide for those of you that think about measures. I really wanted to provide some real data for you to look at and to consider, and maybe even write down. But one of the questions that I think about every day when I'm thinking about validity. [Vance Gray] 13:08:57 and reliability, and whether or not our measures work. [Vance Gray] 13:09:00 What are we trying to do? Like, what have been the results of our efforts? These, what you see on the screen here is by academic year are our pass rates. [Vance Gray] 13:09:15 For our remedial classes in both English and Math. And you know, I'm being transparent. I'm putting these numbers out so you can see them. And so we see that we have really risen in English and in math. [Vance Gray] 13:09:30 per se in our remedial English and our remedial math classes. Of course, we could do better right? So we really want to think about like, is this really predicting this course? [Vance Gray] 13:09:44 What we do in our actual classes each year as they're going, what are we introducing? What are we doing? Is that helping? Because we had some really challenges with our placement tests in the past. [Vance Gray] 13:10:00 Which, coming out of COVID, I'm sure many of you have experience, if not tried to work through yourself. So again, how well do our classes represent what our real results have been in the class? [Vance Gray] 13:10:16 And then two, do alternative measures sort of like multiple measures for high school work? Or is it the coursework portfolios? Would those do a better job? And then. [Vance Gray] 13:10:30 Excuse me, what trade-offs are we willing to accept? For example, we know that if students are cheating before the test. [Vance Gray] 13:10:42 They're teaching in the class, and they're cheating after the test. So really are working to find what is the best way for us to do this. And that's really important for us on what we do. Next slide. [Vance Gray] 13:10:57 So, as we're thinking about that, we really need to think about, is there equity, bias, or differential effect? This is really point number two, or theme number two, because we all know that placement is not neutral. [Vance Gray] 13:11:13 It may be a learned behavior, it may be something that we're trying to overcome, but we know. [Vance Gray] 13:11:21 It's empirical that placement is not neutral. It's built into our system. [Vance Gray] 13:11:27 We know that standardized tests often don't reflect the difference in prior opportunity more than differences in ability. We know that the number one outcome or predictor of success in college. [Vance Gray] 13:11:43 It's high school preparation, and not all high schools are created equally. [Vance Gray] 13:11:49 We also know that first generation students often, people of color and students from under-resourced schools are disproportionately placed into remedial education. [Vance Gray] 13:12:04 help us figure that out. We know that empirically, and that may be a belief that I hold, but I hold it sincerely, and I do think there are areas for opportunity for most schools to be able to break through. [Vance Gray] 13:12:18 We also know that remediation, like getting students out of those systems of schooling as traditionally designed are typically a barrier. They're not necessarily a breach. They helped, they closed gaps, but should they have been there in the first place? [Vance Gray] 13:12:37 So again, this is really about how do we design a system that supports equity rather than hinders it or produces inequality in our system? [Vance Gray] 13:12:48 Next slide. So, again, looking at the evidence, looking at what I was able to see over the course of the last year, we introduced Ed Ready into our system in summer of 2024. So just over. [Vance Gray] 13:13:06 six terms ago, or a semester, a year and a half ago on what we do. And we see in these charts what what I'm looking at as the mean score for our students that are in those remedial classes. [Vance Gray] 13:13:22 So we looked at them doing a test in the first week of courses and them doing a test in the last week of courses. And we know based on enrollment and based on their success, that students typically gain. [Vance Gray] 13:13:38 Um, scores. The beginning score and their ending score by using it ready, and that's a system of really taking our students through in math and in English. [Vance Gray] 13:13:52 all of the things that they need to be college ready. And so, again, this chart suggests that in English 110, which is our remedial course. [Vance Gray] 13:14:02 that we're seeing gains, that we're still seeing gains from fall of 2024 and a fall of 2025. We see those same gains in our math courses. [Vance Gray] 13:14:17 And that's, you know, we have seen enrollment growth over that period of time at the college itself, but we have also seen that what's staying constant are our students in those remedial class. [Vance Gray] 13:14:32 scoring better on the second test than they do on the first test when they enter that class. So they are getting the skills that they need for English and for math as they move forward. One of the next. [Vance Gray] 13:14:48 I would say reports that I'm asking our IT team to pull is the next course in sequence. I really want to see over time how students that have gone through since fall of 2024 to fall of 2026. [Vance Gray] 13:15:04 What their next course of sequence has been, and whether or not they have passed that class. That will help us better understand what we're doing, what may need to be tweaked, or wholly thrown out. [Vance Gray] 13:15:20 in both our math and our English class. So I wanted to really hit hard on that with these graphs so that individuals would know that depending on what they're doing in those remedial classes will help them, um. [Vance Gray] 13:15:36 set the parameters for those placement tests. Next slide. [Vance Gray] 13:15:41 Again, here, what we're doing at our institution to help us understand equity, bias, and differential impact. [Vance Gray] 13:15:52 PIT is a school that deals primarily with adult learners, which many of you are seeing at your institution. But our average age is 25 to 32. [Vance Gray] 13:16:08 and also we're primarily an African American female institution. [Vance Gray] 13:16:13 We have primarily healthcare-related fields or majors. We have them all, but primarily we have individuals that could end our lives. So it's really important that we do things right in both English. [Vance Gray] 13:16:30 For a communication perspective and math, because rounding. errors can really challenge a patient. So that's really important to us. And many of you will know that better than I do. But this is a term by term. Look at what we're doing with EdReady to make sure that we see the same mean score gain. [Vance Gray] 13:16:51 over a period of time, semester to semester, and then year over year. [Vance Gray] 13:16:57 Next slide. Oh, okay, yeah. [William Engfer - NROC] 13:16:58 Yeah, just to interrupt quickly here, um, Dr. Gray, if you could speak a little bit more about how you actually integrate EdReady into the process that they're going through in these courses, you know, when they're coming to take the placement exam, then when they're… when the students are actually working through their study path, when does that take place? [Vance Gray] 13:17:09 Yeah. [William Engfer - NROC] 13:17:16 In relation to the semester. [Vance Gray] 13:17:16 That? Yeah. Great. So we have terms, so we run them every 11 to 12 weeks. So right at the beginning, in the first week, these students are taking that initial diagnostic, and they're working weekly on the pathway. So that initial diagnostic that's offered in a ready that pathway that these students are taking. [Vance Gray] 13:17:42 We take a snapshot of what their initial score is, and then individually, each of these students are working on that path. And then at the end, they all take that same diagnostic to see. [Vance Gray] 13:17:57 where they are. So that gives us the ability to look at the central tendency of these students in each of these remedial classes, both English and math. [Vance Gray] 13:18:09 I know our numbers at our institution are changing because we need to do that, but… and we looked at. I look at that term by term. So that's so right at the beginning, all the way to the end. [William Engfer - NROC] 13:18:25 Gotcha. [Vance Gray] 13:18:25 And then it's traditional remedial course. So we're working on all those skills that they need to be successful or college ready. [Vance Gray] 13:18:36 all along and individually, students are working on their pathway. So Ed Ready is a part of that course. It's not the only part of that course. It is a central part of the course. Does that help, William? [William Engfer - NROC] 13:18:51 Absolutely, yeah. Also, in that coursework, our, um, the instructor's sort of coordinating their instruction to, uh, lesson topics or unit topics in EdReady, or are students sort of set on their own path. [William Engfer - NROC] 13:19:05 At the start of the semester in EdReady, and it runs parallel sort of as they're going. Could you dive a little more into that [Vance Gray] 13:19:13 Yeah, I think that's a good question. Our EdReady is dynamic, right? So it's it is intentional for every student. So their pathway to success, we look at a 90% target rate. And so. [Vance Gray] 13:19:30 Some students surpass that, and some students get right up to that point in order to pass this class. You need a 77% right? And so you saw some of our pass rates at right at the beginning. [Vance Gray] 13:19:46 But this course, the Ed Ready portion of this course is dynamic to the student, but along the way, the professor is looking primarily at what are groups of students doing. [Vance Gray] 13:20:01 But also, here's what the all of these students need overall to be successful, both in math and in English. So they're not tailoring their instruction because there's only one instructor in the classroom. It's not, um… Like we have in Washington state or in some places in California, where they have two an adult-based learner and the actual instructor in that course to be able to go over and add context. So we don't… we're not combining, um. [Vance Gray] 13:20:36 what I would call supplemental instruction with instruction right then and there, so it's not on demand. This is a real live class, and the Ed Ready portion is both in the class and at home. [William Engfer - NROC] 13:20:49 That's great. [Vance Gray] 13:20:52 You're welcome. Next slide. Yeah, and we'll we'll deal with any questions that folks have along the way. When we think about placement, we also want to make sure that we are thinking about the high schools around us, the feeder schools around us. [Vance Gray] 13:21:08 What our landscape has been all along, because… at your school. [Vance Gray] 13:21:14 Transcripts and coursework may carry more weight than actual placement test or putting individuals in a remedial class. And so that's really important to know prior to making a change. [Vance Gray] 13:21:30 To your placement test, or to how you're doing in that remedial course. It's really important that also to align yourself with the schools in the area. That's really important because every school. [Vance Gray] 13:21:45 This is going to be different at every turn. So there isn't one school or one school system, maybe the United or Los Angeles Unified may be all went away, but that's not the case everywhere. [Vance Gray] 13:22:01 Oakland Unified School District, we have the same rules. But again, there are some schools that are more resourced. [Vance Gray] 13:22:10 than those that are resource constrained. And I say that in large populations like Los Angeles versus Oakland. There may be some disparate areas that folks need to pay attention to. So when we think about the theme, this number four. [Vance Gray] 13:22:26 We need to be thinking about how are we aligned with our high school? And are we doing what our local high school dual enrollment, or what have you has us do to make sure that we are admitting folks based on their requirements, as well as our own? [Vance Gray] 13:22:44 And our regional creditors, like middle states or SACS, or the Western Association. That's really important. Like, how do we really account for all of those things? I think that's part of what makes, um, placement. [Vance Gray] 13:23:00 really seen as a part of a larger ecosystem within our campus, within national norms and local norms. That's really important. So, next slide really gets us to slide number 5, which talks about. [Vance Gray] 13:23:15 For theme five, which talks about really all of those traditional testing methods, whether we're using EdReady, which I really think is the way we're going here at our institution and what we're doing, but also whether or not we're using. [Vance Gray] 13:23:31 those multiple measures, SAT, ACT, COMPAS, Accuplacer, all of those things are important. But this is where you can really be innovative at your institution. Like, besides allowing individuals to choose. [Vance Gray] 13:23:49 Whether or not they're going to submit their ACT or their SAT scores really allow them to choose what it is that they're going to do. What are they going to submit in order to be accepted. [Vance Gray] 13:24:04 In Georgia, we did away with singular support courses, right? There was no English 199 or 99 by itself. There was English 097 along with English 101. So in Georgia, they were really using that co-requisite method. [Vance Gray] 13:24:26 For students that didn't opt to use SAT or ACT. They went with more of the co-requisite method. And that worked really well for our institution that I was at at that time. But always, when we think about being innovative. [Vance Gray] 13:24:42 We need to be thinking about what works for us as a private or a independent college versus what they're doing at the national level, the size of our institution, who we have to work with and all. But it really is important. [Vance Gray] 13:24:59 Not just to pick one model, but to recognize that there are a lot of options out there that we can use or that can be used, and that's really important for all of our institutions that here. [Vance Gray] 13:25:13 I will say, being transparent, again. Our institution chose to do away with, and this was like 30 years ago, the cost of placement tests. [Vance Gray] 13:25:29 Um, versus switching to another provider, and we decided to make our own tests, a homegrown test, and that has worked. Although now, when we look at that, we're seeing gains. [Vance Gray] 13:25:44 Using EdReady, but we're also seeing a heightened level of cheating, of academic dishonesty, and so we're trying to figure out ways to incorporate that into our decision-making. And we think that using a ready. [Vance Gray] 13:26:01 Honor lock or respondents will really help us in that instance. So it's really important to think about the institution overall. [Vance Gray] 13:26:11 More than, um, just picking one test. So we had to think about costs. And so, using EdReady in our classrooms, we already have a way to institute it ready for our placement as well, and that gives us that. [Vance Gray] 13:26:27 option that we're gonna go with that really is helping us here at PIT, which may or may not help at other institutions. That was really important. One of the seven that we were really looking to make sure that we did right. [Vance Gray] 13:26:44 And, um, I'll go into a little bit more detail toward the end. Next slide. [Vance Gray] 13:26:52 I've touched on this, but theme number six, which we really need to be thinking about if you're serving on any of those committees are the institutional as well as operational realities. That's going to be really important. [Vance Gray] 13:27:08 Because, again, if you're making this change, it's not just pedagogical, it is institutional. [Vance Gray] 13:27:16 Can you handle the costs? Are we going to increase the advising workloads? Are we going to have more students, less students? How does that impact course scheduling, seat availability, faculty training or data systems. [Vance Gray] 13:27:31 Like, what is this one singular, seems really simple, but is really complex change at our institution. What are we doing? Like it affects students time to degree, like whether or not this student is going to be. [Vance Gray] 13:27:48 stuck in that pattern of remediation, getting that student ready? Did we design and pick the right solution to move that student from remediation to college level to graduation? [Vance Gray] 13:28:03 Did we do that? Will we retain this student? Were they left into the system that they could never get out of? And then what is that doing to our resource allocation on our campus? That's really important. So any form. [Vance Gray] 13:28:20 Like, of reform or change at your institution needs to make sure that we take in consideration all of the operational realities that you find on your campus. That is probably one of the most important aspects. [Vance Gray] 13:28:37 Everything that I'm saying here today, regardless of the data, regardless of all the other issues of fairness, of bias, of institutional impact. [Vance Gray] 13:28:48 What are the real operational realities that we are all up against? That may be cost, it may be it's too infrequent. We forgot about that. We don't understand the purpose of why we do this in the first place. [Vance Gray] 13:29:03 Aren't all college students college ready? Those are things we really have to think about. And so I would ask that you, instead of walking away from this presentation, what I want you to with the notion that I don't know what to do, I want you to walk away. [Vance Gray] 13:29:21 from this presentation with a list of questions, with data, so that you can ask those other members of your group or of your committee. What do I do in this instance? We're faced with all of these challenges, and really. [Vance Gray] 13:29:37 Placement really isn't on a lot of our minds. If we think about it, it's we have a placement test, or we allow students to choose to bring in what they want to choose, um, or what they want in terms of what they've done, or we, um… have automatic admissions, and so that's really important. This really is theme number six, but it really should be theme number two. [Vance Gray] 13:30:04 Right? Next slide. And really, when we're thinking about. [Vance Gray] 13:30:11 Theme number seven here. Have we involved our faculty? Did they vet this? Have they taken the time to look at both the math and the English placement test or other areas, other metrics? [Vance Gray] 13:30:27 that you can pull in, because what I found at both unionized, non-unionized workforces is that they want. [Vance Gray] 13:30:38 More control or less control, or no control, right? Depending on what their workload is, faculty involvement at all of these campuses is also paramount. [Vance Gray] 13:30:53 Like, that governance structure needs to really include faculty in that process. And many of you on this call know it. [Vance Gray] 13:31:03 It slows down the process, right? Because faculty have a lot of things on their plate, but it really is part of what we should do. Like, who gets to decide how placement works? Is that just the admissions team? [Vance Gray] 13:31:18 Or are they working with others? How do we balance consistency across the institution with disciplinary autonomy? That's really important. How they do things in nursing. [Vance Gray] 13:31:31 really is different than how we do things in health science, or in business, or in general education. So, that's really an issue for us, how we buy in for that change that we're gonna make, and something that is as simple. [Vance Gray] 13:31:48 As a placement test, or how we place students in either math or English. Like, and the hardest part is really keeping faculty engaged, right? That is a process that really needs to happen more often on our campus. [Vance Gray] 13:32:06 On the next slide, one of the things that I wanted to share with you was what faculty actually say, like, when they're involved in the process. So we had faculty at our institution who are taking the tests. [Vance Gray] 13:32:21 I know who these faculty are. And I'm sharing with you in full transparency, without their names, the questions that faculty have, so math faculty. [Vance Gray] 13:32:33 Do we have a planned cost score? Is it going to be that 77, that 90? Where are we going to land with our placement tests? Like, will the tests be proctored? [Vance Gray] 13:32:45 Are they gonna do this on their own? Will we have a proctoring service, or will we, um, do that ourselves? We currently send that test out to students, and they send it back. [Vance Gray] 13:33:00 Like, is that enough? Like, will students be able to complete the test or the lessons to improve their scores? Or can they retake the entire test? So we think about EdReady as the tool that we plan to use. [Vance Gray] 13:33:17 Students can submit that score that very first initial score, or will we have them submit their score at the end of the week? [Vance Gray] 13:33:28 Like, is it a one and done? And that's the question that the math faculty here, one of… at least one of the math faculty here is asking. [Vance Gray] 13:33:40 English faculty, one of the things that I did here was lift exactly what they said, and they said that they reviewed the placement test and believe that it reflects the changes and the suggested material as a team. [Vance Gray] 13:33:57 I… they agree with the other feedback that has been shared. Like, they were able to complete the test in 30 minutes, but they think it may take a student longer. Like, they think it may take longer. Is this going to be a set test? [Vance Gray] 13:34:10 They said they noticed a strong emphasis on reading and fewer grammar-specific items. We had the opportunity to choose those items to sort of look at. We want students to have these skills in English. [Vance Gray] 13:34:26 And these skills in math, and then that was a reiteration of what EdReady provided, and so we provided feedback, they provided feedback. We provided more feedback. So this has been a back-and-forth issue for us. [Vance Gray] 13:34:43 for 6 months. So far, on how we go about making that particular change. But overall, I think at least the English faculty member said they think that this is a solid indication for us to move forward. So we're really involving faculty. [Vance Gray] 13:35:01 Which was number seven of the things, but it's really something that we need to do throughout the process, making sure we're doing that, but also include admissions, also include enrollment, also include student affairs. [Vance Gray] 13:35:17 So they understand what we're doing and why. And that… I think that's really important. [Vance Gray] 13:35:23 I think my next slide is really just a opportunity for us to look at, again, as a reminder, what those seven themes are. And let me go over those for you as well. When you're thinking about placement. [Vance Gray] 13:35:40 Whether you use EdReady or not, whether you use another form. [Vance Gray] 13:35:45 We really ought to be mindful of the purpose behind the placement test. Like, are we using evidence and measures to figure out what we're doing here? We have more students or fewer students? [Vance Gray] 13:36:01 What's the equity, the bias, and the differential impact that we see currently and what we plan to see, and whether or not you are a Dei institution or an institution that believes in reflecting. [Vance Gray] 13:36:17 Your, um, students with the complexity of what we see around the US, that's up to every institution, but we really ought to be getting out of the way of bias. [Vance Gray] 13:36:32 differential impact that students have along the way. Is this test, or are the things that we're doing aligned with our local population? Are we doing what the high schools in the area or the theater schools want us to be able to do? [Vance Gray] 13:36:53 The college level. What alternatives, you know, have you thought about? Do you need an entry exam? Will students be able to provide you with their direct or indirect assessments? Are they going to use act, sat, GMAT scores, LSAT scores. [Vance Gray] 13:37:14 at the graduate level. Accuplacer or compass, like, thinking about all those alternatives to traditional testing, what works for your institution? [Vance Gray] 13:37:27 Institutional guidelines, institutional regulations, really the reality of what's happening at your institution needs to be part of the process. [Vance Gray] 13:37:38 And again, always faculty involvement with the process. And then just overall big picture. [Vance Gray] 13:37:48 None of this can be done in isolation. We have to work in teams, in groups, moving items from one area to another. Math, English, faculty, of course, administration, department chairs, program directors. [Vance Gray] 13:38:04 and the like, all of that needs to be done, and so that's really important here. It's really important everywhere that we work, whether that's LA, Washington State, you know, big shout out to Washington State, where I went to school. [Vance Gray] 13:38:19 Georgia, where I worked, and the Pennsylvania area where I am now. It's really important that we rethink these placement tests in a way that doesn't isolate individuals or departments or their autonomy. [Vance Gray] 13:38:35 We have to talk to each other. Next slide. [Vance Gray] 13:38:40 Here, I did also want to just show that what we were doing in our institution also works with either DevEd reports or with assessment reports. [Vance Gray] 13:38:53 for our regional accreditors, that's really important, so we can't leave our DevEd approaches separate and apart from what we're doing, those also need to be assessed. And one of the things that you see here is our path rates, our paths rates. [Vance Gray] 13:39:11 for developmental English, as well as developmental math, right? And do placement tests provide that sort of predictor that you thought they would provide. So again, when I first came to this institution. [Vance Gray] 13:39:28 I looked at our Gen Ed, our whole entire gen ed, and was thinking, why are there so many individuals passing. [Vance Gray] 13:39:38 and college level English. and college-level math. [Vance Gray] 13:39:43 Where are these courses taking place? Many of them were asynchronous online. And so they are now being proctored. Every test is being proctored, and those scores are balancing themselves out so that they're more traditional. [Vance Gray] 13:40:01 in the way that I would, as part of academic affairs, expect them to be. [Vance Gray] 13:40:07 I also have come to the conclusion that sometimes placement tests don't always show the aptitude of an individual. It might just show their circumstance, and that's really important. [Vance Gray] 13:40:23 So wherever our developmental roles are, there ought to be an opportunity for students to submit their tests at the end of the week. Deadlines are important. I always believe in that. But I also believe that there are far too many individuals. [Vance Gray] 13:40:43 stuck in the cycle of remediation when we have individuals in those courses that may not, may not be using either the Washington State method of instruction or a California method of instruction. [Vance Gray] 13:41:00 which has two individuals in those classes to make sure that more individuals are meeting or attempting to meet the learning outcomes. That's really important to me. [Vance Gray] 13:41:13 So assessment is important. I should have had this slide up to the back, and I sort of moved it into the, um… I would say appendix, just in case that question came up. So, thanks. Next slide. [Vance Gray] 13:41:30 So I want to thank you for giving me the opportunity to share and be rather transparent about what we're doing here at PIT to really look at how we're going to use EdReady in our systems, and then what systems. [Vance Gray] 13:41:47 of it ready we have in place that we're already using and that are measuring and looking at to see do they work? Does everybody work for not only my institution, but for the students in that class? [Vance Gray] 13:42:02 That was more important for me, and so now I know I can take it ready and move it to the outer part of what we're doing, as opposed to just in those remedial classes. So to me, I want to thank you, and thank you all for allowing me to spend a little bit of time with you. [William Engfer - NROC] 13:42:21 Thank you very much. Yeah, that was super insightful, and I really appreciate the, uh, the holistic look, really, at these themes of placement, how placement impacts students. Um, I certainly know that, you know, when we're working with partners and and. [William Engfer - NROC] 13:42:36 working EdReady into a placement model, we obviously want it to replace the high-stakes, one-and-done barrier placement tests that have existed at many institutions for a very long time, right? I think people have been. [Vance Gray] 13:42:43 Yeah. [Vance Gray] 13:42:51 Yeah. [William Engfer - NROC] 13:42:52 Having the conversation, rethinking placement, sort of across the country, um, to a large degree for a number of years now, but we really pride ourselves on having EdReady be a part of that. I would definitely love to dive in a little deeper into some of your experience with that. [Vance Gray] 13:43:07 Okay. Yeah. [William Engfer - NROC] 13:43:09 Particularly, you know, talking about the results that have… that you've seen here, you've seen these score gains with EdReady, you see that students are making progress as they're going through those remedial courses. Um, the impact out from that, what do you see sort of moving forward from. [William Engfer - NROC] 13:43:27 from those results, do you think that it's going to increase your retention rates going forward? Do you foresee it, sort of. [William Engfer - NROC] 13:43:36 increasing student engagement, um, what do you think would sort of be the outcome from there? [Vance Gray] 13:43:41 Yeah, I think in terms of what we're using and ready for right now, I do think that Ed Ready is going to help us increase our retention rates, as well as our graduation rates, which are really. [Vance Gray] 13:43:56 The things, the levers that allow us to report to our accrediting bodies what we're doing, how we're letting, how we're helping students learn. Like, that's really important in a field where workforce. [Vance Gray] 13:44:14 the focus has been on workforce. Well, I want to make sure that I'm receiving bedside treatment really well right from the best prepared individuals. But I also want to receive an email that's written properly. [Vance Gray] 13:44:30 that has all of the individual skills there. So it's helping us increase retention rates. That is a measure over time. So I think that that helps institutions grow. [Vance Gray] 13:44:44 Um, you… and I'm sure you've heard the expression, it's cheaper to keep the student. [Vance Gray] 13:44:50 It's cheaper to keep the student apply the Ed Ready skills of pathway placement along a path, get ready, right? So to me, that's important. Graduation rates. We know that in the US. [Vance Gray] 13:45:07 Only 50% of all of our students graduate in their field of choice over a 4 or 6, or even an 8-year period. 50% of all of our students. So automatically, the more students we can graduate. [Vance Gray] 13:45:25 the better we have in increasing the number of students in the U.S. That have either associate's degree or a bachelor's degree. So that's really part of my effort, part of my role. I do think that. [Vance Gray] 13:45:40 at ready the engagement is also important. Far too many professors have to rely on, like, the number of students that they're teaching. You know, I was at an institution where you weren't doing your job if you didn't teach 150 students. [Vance Gray] 13:45:59 A semester. That's five. That's a 5 5 for many of the faculty on this call. That's important. So Ed Ready also does some of that engagement for that faculty member. [Vance Gray] 13:46:15 that can't get to every student right away, but they'll get to that student advisement. If you have a faculty advising model or a midterm with feedback or final grades. Ed Ready is there with that student all the way, all along. [Vance Gray] 13:46:30 So it's really like having a second instructor in the class for that particular assignment. That really helps along the way. So we hope to do not only student engagement, but retention and graduation rates. [William Engfer - NROC] 13:46:46 That's great, and I'm very happy to hear that, you know, you feel like it's really building that success, and especially building the experience for the student, right? That's first and foremost our mission, our goal as an organization with EdReady is to help. [Vance Gray] 13:46:55 Yeah. [William Engfer - NROC] 13:47:02 Students learn. Um, you know, we break down our mission to meeting students where they are, preparing them for what's next, um, so that's really great to hear. We do have a. [Vance Gray] 13:47:11 And I will add, I will add, William, that I am a proponent of people, students of color being able to get through these remedial classes, be able to demonstrate those skills. [William Engfer - NROC] 13:47:12 Yeah, please. [Vance Gray] 13:47:27 in a way that the student walks away from that class knowing that they belong in college. So belonging is really a critical part of what I've done as a student myself. [William Engfer - NROC] 13:47:42 That's great. Um, I actually… definitely, the student confidence aspect of EdReady is a huge part of what we pride ourselves on, right? A placement test can be something that completely discourages a student going into a higher education experience, and. [Vance Gray] 13:47:51 Yeah. Yeah. [William Engfer - NROC] 13:48:00 We want that to be as open as possible. You know, we build that into our system too, right? When a student is taking a diagnostic test, they are encouraging, you know, pop-ups that tell them to keep going and tell them, you know, reassure them that this is not, you know, one of those high-stakes tests. [Vance Gray] 13:48:06 Yeah. [William Engfer - NROC] 13:48:17 that they have the option to really work through it as they need to. So, I know we certainly pride ourselves on that, and it's great to hear that you have that priority as well, about opening those doors, breaking down those barriers for students. We do have a question. [William Engfer - NROC] 13:48:33 In here, um, about where you sort of established this mindset, and how it sort of evolved at PIT. Um, you know, to what degree did you bring the mindset in about placement, uh, when you started working there versus. [William Engfer - NROC] 13:48:50 What you were already encountering there, and I would actually even expand that out, maybe, to talk about what sort of, you know, to dive a little bit deeper into the administrative barriers, maybe, or the administrative support, the… I know you talked about interacting with the faculty as well. What sort of environment did you sort of cultivate this. [William Engfer - NROC] 13:49:10 this idea about placement [Vance Gray] 13:49:13 So it has really been a long and hard struggle. You know, I was born and raised in Oakland, California. I was the first generation college student at the University of Washington in Seattle. [Vance Gray] 13:49:29 Um, I, um, went from Seattle to experience what an HBCU was like in getting my MBA and PhD from Clark Atlanta in Georgia. I had the opportunity to work at Spelman College. [Vance Gray] 13:49:47 for 10 years, which is an all-female HBCU. I also have the opportunity to work at Agnes Scott College for four years, which is a well-prepared. [Vance Gray] 13:50:01 female student school, a traditionally white but very minoritized institution, both private in Georgia, and work with Atlanta Metropolitan State College in Georgia. So I really think that that mindset. [Vance Gray] 13:50:17 that sense of belonging came from what I experienced, not only in Oakland, but in Georgia. So really a dichotomy of experiences, and as a first-generation college student, it was really important for me. [Vance Gray] 13:50:33 to break down those walls, those barriers within my own family, as well as at the institutions that I worked at. That was really important to me. And taking my cue from, you know, being a Black political scientist. [Vance Gray] 13:50:50 really was important to me. I learned that in Georgia. In Chicago, I had the opportunity to chair an equity committee and be in that environment where equity was really a part of what we were doing, a really fantastic. [Vance Gray] 13:51:09 female president, a male vice president, and a female dean of student services. We really got together to say in Chicago that if we could do it, they could do it, right? And that's all of our students. [Vance Gray] 13:51:27 And Pennsylvania, in the city or Pennsylvania City College, right? Or Philadelphia Community College have the opportunity to work with a now a female president. [Vance Gray] 13:51:42 Dr. Marshall, which really… you know, honed me in to really think about what the mindset would be there, as well as the chief of staff. Um, I think those things were really important to me, and understanding the mindset of students. [Vance Gray] 13:52:00 They don't go to college to fail. They go to college to succeed, to be something for their family. So I really appreciate that question, but at PIC, where I am now, over the course of the last year and three quarters last year, and. [Vance Gray] 13:52:17 8 months, really having that opportunity to work with our gen ed faculty in both math and English and across all of our institution, really, um, they have been open because they have been starving for this. [Vance Gray] 13:52:34 For a long time, someone that would help fight the fight, because our administration and many of our of you are used to this, administration does not always reflect the tenor. [Vance Gray] 13:52:49 The demographic of the students that we serve. And that's really important. And so I think bringing a very perspective into the group has helped here at PIT, but everywhere I've been. [Vance Gray] 13:53:04 So thanks for that question. [William Engfer - NROC] 13:53:06 That's great, yeah, it's amazing to hear your experience and your through line through those different institutions, because every institution does it differently, right? Some institutions are so primed to innovate and administrators really support that, and. [Vance Gray] 13:53:16 Yeah. [William Engfer - NROC] 13:53:23 Other institutions? Maybe not so much. So, um, that's really great to hear. [Vance Gray] 13:53:27 But also administration has to hear, and they don't always get the opportunity to hear from all of the faculty, and many of us have a lot of different experiences. I like to think of myself as young. [William Engfer - NROC] 13:53:31 Hmm. [William Engfer - NROC] 13:53:36 Great. [Vance Gray] 13:53:43 and spry, but I know I've been at these institutions and made a significant impact on not only the students, but administration. And you don't always get that opportunity. [William Engfer - NROC] 13:53:56 Yeah, absolutely. I mean, along those lines also, I'd love to hear if you've had any feedback, um, you know, other than what you mentioned, specifically about student experience here, what the students have experienced in these… in this coursework. Maybe their experience using EdReady. [William Engfer - NROC] 13:54:12 How maybe that confidence has been boosted. Have you… have you seen or heard any of that with your student body? [Vance Gray] 13:54:19 I do. Um, I'm starting to look at that more and more. I haven't always had the opportunity in many institutions do not always convey what those comments are from students to administration. [Vance Gray] 13:54:34 For various reasons, right? But I get the opportunity to set up and to read and review all of our end-of-course… student evaluation of instruction. And to me, that's really important. So, I read those. [Vance Gray] 13:54:49 And when I saw in one of the feedback surveys that students do, that please did somebody here that wants to teach this course, I was touched. [Vance Gray] 13:55:03 To me, that said this faculty member needs a little bit more training, maybe a little bit more of a touch on how to help students learn how to help this students learn. So I think most institutions are getting. [Vance Gray] 13:55:35 To be able to review that so that I can share. Not all faculty at institutionalized institutions or unionized versus non-unionized, get that opportunity, and that's important, too. [Vance Gray] 13:55:35 More familiar with being able to take that comments section of end-to-student instruction and share that without penalty, with administration. It's really difficult. It's hard, but I make the case for myself. [Vance Gray] 13:55:52 So that's important to note. Not many administrators get the opportunity to hear from faculty or read reports on what students are actually demanding. And so that mindset shift and change. [William Engfer - NROC] 13:56:00 Right. [Vance Gray] 13:56:07 It's happening slowly at our institutions, but we can't be… it's easier for me to say than for a faculty member to say, you know what? This is what students are saying, either about me or about my colleague. [William Engfer - NROC] 13:56:22 Right. That's great. It's great to hear that, and I mean, I'm sure everyone listening in has experienced that process too, right? We're all trying to make progress in having that feedback loop really work functionally, and also that, you know, people's experience continues to improve and, and, um. [Vance Gray] 13:56:23 So it's it's a process. [William Engfer - NROC] 13:56:44 and grow, obviously. So, um, well, if there are any last-minute questions, please feel free to put them in the chat and Q&A box. We'll wrap up here in just a minute. I wanted to. [William Engfer - NROC] 13:56:55 promote our next webinar that we'll be having, which actually aligns a little bit with some of the stuff that we've talked about today. It's about dismantling math barriers for future STEM leaders. This actually talks about somebody who's using EdReady, um, for much younger learners. [Vance Gray] 13:57:01 Hmm. [William Engfer - NROC] 13:57:11 So that will be on March 18th. Please feel free to join us for that. You can sign up via our website. I want to really thank Dr. Gray for all your insight today. It's been fantastic to hear about how you're integrating a really dynamic, innovative perspective on placement to help students, to lead them into. [William Engfer - NROC] 13:57:32 you know, a higher education experience that they'll hopefully feel empowered and really engaged in. So, I want to thank you very much for joining us. Thank you all for joining us today, um, for this webinar. If there's any other questions that you have, please feel free to get in touch with either of us. [William Engfer - NROC] 13:57:49 Um, as a reminder, the recording for this webinar will be available via our website as well. So, thank you all for joining us today, and we look forward to seeing you on, hopefully, another of our webinars in the future. [Vance Gray] 13:58:02 Thank you so much. [William Engfer - NROC] 13:58:03 Thank you.