Mar 5, 2024
Did Dr. Deming forbid setting goals? Dive into this discussion about healthy goal setting, learn why your process matters, and the four things you need to understand before you start on goals. This episode is the first in a 4-part series about goal setting.
TRANSCRIPT
0:00:02.2 Andrew Stotz: My name is Andrew Stotz and I'll be your host as we continue our journey into the teachings of Dr. W. Edwards Deming. Today I'm continuing my discussion with John Dues, who is part of the new generation of educators striving to apply Dr. Deming's principles to unleash student joy in learning. This is episode 21 and we're talking about goal setting through a Deming lens. John, take it away.
0:00:26.9 John Dues: Yeah, it's good to be back, Andrew. Yeah, 'tis the season for resolutions, I suppose, so I thought we could talk about organizational goal setting and sort of doing that through a Deming lens. And I was thinking about, at a recent district leadership team meeting, I put the following quote up on a slide. I said, "Goal setting is often an act of desperation." You got to watch people's faces when they see that. And to give some context, we're sort of updating our strategic plan at United Schools Network and my point in putting that on the slide as a part of strategic planning was to start a discussion on sort of what I think is healthy goal setting and how that's not typical to what I've seen across my career in schools, education organizations.
0:01:22.4 JD: And I wanted to provide a framework for the team so that anyone that's setting a goal as a part of the strategic planning process sort of had this sort of mindset as we're going through the goal setting process. I think that the typical reaction to that quote, at least in my experience, has been something like, "But I thought that goal setting was something that highly effective people or highly effective organizations do." And my basic argument is that I think that that's the intention, but it's rarely the case, whether that's individuals or organizations. And there's these, what I've come to sort of frame as four conditions that have to be met during the goal setting process. And without those, you kind of get fluff for a goal setting, probably more likely just completely disconnected from reality. I think... Yeah, go ahead.
0:02:22.5 AS: I just wanted to talk to everybody out there that's listening and viewing. I mean, I'm sure you're going through goal setting all the time and as we talk about, it's the beginning of the year right now, this is actually, we're recording this in mid-January of 2024. So it's like I've been working on what's our vision? What's our mission? What's our values? Where are we going? What is our goal? What is our long term goal? What is our short term goal? And I don't know about you guys, but for me, it gets a little confusing and round in circles sometimes and overwhelming, and then this whole idea about, that goal setting is often an act of desperation. It's like I've been working on this stuff for recently over the last week or so and then I just heard you say that and I was like, "Oh, I'm really interested to learn more." So let's go through those four conditions.
0:03:18.5 JD: Yeah, I'll get to those in a second. But I... So I'm not saying don't set goals necessarily. And people have that same reaction typically to that statement, but it's goal setting is often an act of desperation. So it's not the goal in and of themselves, but generally it's the process that you go about and the lack of sort of logic behind the goals that I'm talking about. And I know on these podcasts, many of my examples have sort of I've been banging on like State Department accountability systems and stuff like that. I'm going to continue to do that today but I think the same sort of errors happen at the school system level, at the individual school building level, at the individual teacher or principal level, it's just the stakes are higher when you're talking about states and countries, systems of education.
0:04:11.8 JD: But what I've seen is over the last two decades, certainly post-No Child Left Behind, what it seems like is that there's often these goals set and they're sort of, the targets are sort of chosen out of thin air. And then there's this whole accountability system built around those goals and then in the case of states, we then rate and rank school systems or schools based on how they perform in relation to those goals. And again, the same type of thing is happening at the school system level, at the school level but probably the state accountability systems is what most educators are familiar with when I'm talking about goals. And in Ohio, like a lot of states, we give state tests, we give them third through eighth grade.
0:04:58.5 JD: They take reading and math every year, third through eighth grade in Ohio and you have to hit this 80% benchmark in terms of the percent of kids that are proficient in your school to meet the state standard. So the first question is, why not 60%? Why not 95%? Why not 85%? Why not 82.5%? Just random, you know? And my hunch is, the first problem is that that benchmark for passage rates, if you asked 100 people at the State Department or 100 people working in public school systems in Ohio, I'm not sure if anybody could give you that answer, why 80%? So the first problem is that that target itself is chosen arbitrarily and without sort of a deep consideration. And so that's sort of where the fork initially comes...
0:05:58.3 AS: And I would say that if I look at that 80%, it's like below that and you would seem like you're really underachieving, and above that, it's like, let's be realistic here of what the system can produce.
0:06:11.6 JD: Well, it's a B, it's a B minus. You know, that's familiar, a C, you're not allowed to bring home a C, but a B is okay. So, I mean, my guess is, I don't know where that particular target came from, but my guess is it's something maybe not too far off from, "well, it's sort of a B minus" in the typical grading scale in the United States.
0:06:32.4 AS: Probably came just the way we just discussed it.
0:06:35.1 JD: I would not be entirely surprised. So a lot of the problem with goal setting and when I'm saying act of desperation, it has to do with that arbitrary nature of the goal in and of itself. And so what I've sort of told the team here is that let's put forth some conditions that came up, I mentioned four, that we should understand prior to ever setting a goal. So the first thing we want to understand is what I call the "capability of the system" under study. So in this case, we've talked about third grade reading because that's such an important time period in a student's life.
0:07:13.4 JD: The states, lots of states put a lot of emphasis on it. In Ohio, there's a third grade reading guarantee that exists in other states as well. So we'll kind of look at data in that realm. So the first one, what's the capability of that third grade reading system? The second condition is we have to understand the variation that that system produces. So what are the ups and downs in the data? What are the patterns in the data? So capability, variation is the second condition. The third condition is, is the system that we're studying, is the data stable? When we look at the patterns of the data over time, is there predictability to it?
0:08:01.8 JD: Is there stability to that data or is it all over the place? And then the third thing or sorry, the fourth thing we want is a logical answer to the question, "by what method?" So let's take sort of a deeper look at each of those four conditions, kind of unpack those a little bit. We'll use third grade reading state testing data. I have some data on a chart, but I'll share my screen in a second for those that are viewing the data. And then for those that are only listening, I'll sort of narrate what we're looking at so you'll still get some value out of the description. So you see my screen now?
0:08:42.5 AS: Yep.
0:08:43.3 JD: Okay, cool. So we've looked at these charts before in previous episodes. It's been a while. So this is what some people call a control chart. I call it a process behavior chart because it's literally a description, a visual description of a process unfolding over time.
0:09:01.7 AS: And maybe I'll just describe it. At the title it says, Ohio Third Grade Reading State Testing Proficiency Levels. On the y-axis is the percent proficiency ranging from, of course, zero to 100. And on the x-axis, we have seven school years going from the 2015 to 2016 school year all the way to 2022 to 2023 school year. And then most importantly, we have points, that's a blue line here, but the points that are showing the movement of third grade reading state proficiency levels year by year or school year by school year. Continue.
0:09:51.1 JD: Yep, that's right. That's a good description. So those blue dots are the percent of third graders that are proficient each testing year. And to give you some context, in Ohio about 125,000 third graders take that state reading test each year. One thing you'll notice is that there is no data for 2019-'20. That's because we give the test in the spring of a school year. So in the spring of 2020, schools were shut down due to the pandemic so there was no state test. So we missed one year of testing, but that's really not, that's not really pertinent to this discussion. So the other thing you'll see on here is the green line is the average of the data running through there.
0:10:38.4 JD: And those red lines that are on either side of the data are, some people call them control limits, I call them the lower and upper natural process limit. And they're based on a statistical calculation. They're not where I want the lines to be, they're where they are based on the data. And for those watching, the data points are 54.9% proficient in '15 -'16. The next year in '16 -'17, 63.8% of the third graders were proficient. In '17 -'18, 61.2% were proficient. In '18 -'19, 66.7% were proficient. In 2021, er, 2020 -'21, kind of dipped down to 51.9%. Then in '21 -'22, 59.8% of the third graders were proficient. And then in our most recent year, 62.3% of kids were proficient.
0:11:35.3 AS: So out of all those points, let's just say a high of about roughly 70% and a low of a little bit higher than 50%.
0:11:45.5 JD: Yeah. Yep. Yep. So the low was like, I think 59%. I can look back. Low was 54.9%, the high was 66.7%.
0:11:57.5 AS: Okay.
0:12:00.3 JD: And that works out to about an average of 60% across that seven-year time period.
0:12:08.3 AS: And when we talked about the 80%, is that 80% related to these test results?
0:12:13.7 JD: Sure. Yep.
0:12:13.8 AS: This is what the state is saying it should be?
0:12:18.1 JD: So the state says that in any individual school building, in any individual school system, and so as a result, in the state as a whole, 80% of third graders should meet the proficiency benchmark, basically. So in the state, on average, across the state, when you look at all the third graders, 80% of the kids are not at proficiency. It's lower than that year in and year out across the last seven years. And I should say I picked the starting point as 2015-'16, that was the first year of a brand new test. So it's really a new testing system as of that year. And then it stayed pretty consistent in terms of what the kids are being asked to do. Prior to that, the test was a different format. So it was sort of like a different system.
0:13:04.6 AS: And this is from all schools, so it's Ohio, it's not your school?
0:13:09.6 JD: Right. So this is all Ohio public schools.
0:13:12.9 AS: Okay.
0:13:13.9 JD: Yep. Which are required to give the state test once a year. So, like I said, beginning with this spring 2016 testing season, Ohio began administering this new state test, which is why I started with 2015- '16. And that's where the data starts. So again, schools need to have at least 80% of their students score proficient or higher in each tested area, including reading. So what we're doing here is sort of looking at that first condition. We're trying to figure out what's the capability of this third grade reading testing system. And when I say system, I'm literally talking about everything that could impact third grade reading test scores.
0:14:00.4 JD: Now, I mean, you could almost make an infinite list, but I'm talking about the actual students in Ohio public schools, the third graders themselves, their teachers, the various reading curricula that's being used in schools, technology related to reading programming, supplemental materials, the schools themselves, how the schools themselves are organized. And you can go on and on about any number of in-school and out-of-school variables that might impact a third graders performance on a state test.
0:14:37.2 AS: And I think about resources like between schools and parents and teachers and administrators, everybody's putting forward... Putting forth resources to try to get to this.
0:14:46.9 JD: Yep. The reading standards themselves, the reading test, that's all a part of the third grade reading system. And basically, for those that are viewing the video or heard the description, the capability is outlined in the process behavior chart. I mean, that's literally what the process behavior chart doing. It's, it's, it's visualizing the capability of that third grade reading system. So one thing that's pretty clear when you look at this seven years worth of data is that it's very unlikely that the state of Ohio is incapable of hitting that 80% mark. Now, seven years of data is not 20 years of data, but we, in none of the seven years that have occurred have we gotten anywhere close to that 80% mark. So that's one thing we can see.
0:15:39.9 AS: Sorry, what was the conclusion that you just said?
0:15:42.8 JD: Well, we're, we can see from the data here, even though it's only seven data points, which is something to work with, but it's not 20 data points, it's not 25 data points but it's pretty likely that the third grade reading system, that we're incapable as a state of hitting 80%...
0:16:00.9 AS: Okay, so the capability of the system, the goal of the, of the state representatives that set the 80% seems to be slightly outside of the capability of the system.
0:16:14.4 JD: I'd say more than slightly.
0:16:15.8 AS: More than slightly. 0kay.
0:16:16.9 JD: Yeah, I'd say it likely... I would go as far to say, I try to talk scientifically so it's, we are likely incapable of hitting that 80% mark as a state.
0:16:26.5 AS: Okay. Got it.
0:16:27.7 JD: Not impossible.
0:16:29.0 AS: That's point number one.
0:16:30.5 JD: Yeah, well, and these red natural process limits actually tell us what we could expect from this particular system based on what we've seen so far. So those process limits, kind of way to think about them is as you get more data points, especially as you get 20, 24 data points, they sort of start to solidify. So an individual data point has less of an impact on the limits. So I would call them a little bit soft right now, an individual data point kind of could have an outsized impact because we don't have tons of data but what these red lines are telling us is that our reading system, this third grade reading system is capable of hitting rates somewhere between 41%, where that lower line is, and 79%, where that upper line is.
0:17:19.8 JD: That's why I say that the 80% is unlikely, rather than impossible. It's technically within the capabilities of this system as illustrated by this process behavior chart. But based on the way the limits are constructed, the limits come from the data itself, how the data, not only the magnitude of the individual data points, but it's also taking into account the point to point variation. So time is an important factor in that formula that's used to calculate the limits. And so based on how the limit is constructed, there's about a 3 in 1000 chance that we would hit that 80% mark. So that's why I say...
0:18:03.4 AS: So you're saying there's a chance?
[laughter]
0:18:06.5 JD: Very unlikely, very unlikely, right? So that's capability, that, this, that's sort of looking at the chart and talking about how capable is our system. The next thing we want to look at... Well, the last thing you could say is that that 60% average across those seven years is a pretty good descriptor, especially as you look at where the dots fall, some above, some below, that's a pretty good descriptor of the overall capability of the system, that's 60% proficiency.
0:18:37.6 AS: Right.
0:18:39.3 JD: So the second thing we'll take a look at is using the chart to understand the variation in our system. So again, we have seven data points. We just mentioned that they're bouncing around this average of 60%. And actually with seven data points, you have three that are below the line and four that are above. So about as even as you could be between how many points are below the line, how many points are above the line. So if you describe the year to year test results starting back in '15-'16, they increase and then decrease and then increase and then decrease and then increase and then increase again, a little bit in that last of the seven years.
0:19:27.8 JD: So when you look at the data, there's no sort of signals in those patterns that indicate that the increases or decreases are of significance. So in the Deming language, probably most people aren't familiar with the "common cause" language, but basically it's just saying that the thousands of variables that impact these test results are part of a common cause system. Just like, they're bouncing around, but the bouncing around is not meaningful. But what actually happens is, you know, inevitably when people describe these results, they'll pick two years. Let's say they look at, well, let's even say they look at the last three years and people will say, "Oh, we've increased the third grade reading test scores 'cause they went up a little bit from 2021 to 2022 to 2023."
0:20:19.5 JD: But again, the increases are meaningless when you're viewing this through the sort of understanding variation, knowledge about variation Deming lens. So, but again, even though seven data points isn't a lot of data, it's pretty clear from what we see so far that that, setting that 80% goal, holding schools and school systems accountable from a state perspective, it's not having any impact on the outcome of this third grade reading system. So that's what I mean to connect back to this goal setting is often an act of desperation. It's a hope and a dream that 80% of kids in this system are gonna meet this proficiency standard. It's just not happening by setting a goal.
0:21:10.2 AS: Right.
0:21:15.4 JD: The third thing is looking at stability. So we want to know if the results are predictable in this particular system. So the thing to think about here is if the system is in fact predictable, it means that the results are sort of performing as consistently as the system is capable of making it. And this Ohio third grade reading system is in fact a stable system. So based on these results so far, we can reasonably expect that future results will continue to bounce around this current average. That's just what's gonna happen. So the results might be a little bit below the average, maybe they'll remain a little bit above the average, but in all likelihood, unless something else of significance changes, this is what we can expect from this system. Now...
0:22:13.1 AS: And for some people that may not totally understand the Deming lens, point number two and point number three may be a bit confusing because you're thinking, what's the variation of the system? Well, doesn't the variation of the system also tell you if the data is stable? How would you describe the difference in those two points?
0:22:39.9 JD: Well, it's stable because there's no patterns in the data that signify instability. So there are different sets of patterns that different organizations like Western Electric had a set of patterns that they sort of established because that's sort of where these charts were invented. The Institute for Healthcare Improvement has a set of rules that they use. They are big in training and using control charts. I sort of, my basic approach is to try to keep things as simple as possible. So I default to Donald Wheeler who wrote a book literally called Understanding Variation among many others on using these charts and interpreting these charts. But he basically says, and I sort of have adopted this idea of just three simple rules that I look for.
0:23:29.0 JD: So I look for a single point outside of the red lines, either above or below that's so unexpected, that's a significant pattern, just one data point. I'll also look for three out of four that are closer to one of the red lines than they are to that average green line. And the other big thing I look for is eight successive data points that are on either side of that average line. So if a single point is outside either the upper or the lower limit, that's a pretty high magnitude chance that there is something very different going on now in your system. The eight points in a row is sort of like a moderate but sustained indication that something has changed in either direction. But in this case, we have a stable system. There's no patterns in the data that suggest instability.
0:24:33.0 JD: So it's good in the sense that the system is predictable. And so, let's say I sort of would then try something new, an intervention of some kind, and then look to see what happens. I know at the outset of the experiment that it was a stable system and I can be relatively assured that it's the thing that I introduced that brought about the change. But stability should not be an indication of good, necessarily. You can have a stable but unsatisfactory system, which is how I would describe this particular system. It's stable, meaning it's predictable based on what we see so far, but it's also unacceptable that 40%, two out of five kids are not proficient on the third grade reading test.
0:25:30.1 AS: The good news is your cancer is stable, the bad news is you have cancer.
0:25:33.4 JD: Your cancer is stable, right. It's the cancer is stable, but maybe not growing. How about that analogy?
0:25:39.5 AS: Yeah.
0:25:41.2 JD: Yeah. So we have a stable system but producing less than desirable outcomes. So at that point, the only thing that I can do is work on the design of the system itself. Something about the inputs, something about the throughputs. Maybe I... One big push here in Ohio is to sort of adopt the principles called the science of reading. So teaching reading in a scientific way, like a research-backed way. And so perhaps that's an intervention that could be attempted and Ohio's sort of attempting it. But that sort of everybody getting behind an approach that's been shown to work, that's very different than just setting a goal and then holding educators accountable to the goal.
0:26:26.3 JD: And that's typically what's happening. And when you do that, then you cause frustration. Because if people knew what to do to make things better, then they'd probably do it. So they're being held accountable for something that they maybe don't know how to improve, or maybe they don't have the resources to improve. And so that's why Deming would say "substitute leadership." And that's what he was talking about, leadership towards improvement. And that's a good segue to the last condition, we've mentioned this idea that the 80% goal is beyond the capability of system, so we have to think about methods. By what method then can we improve because this setting a target isn't gonna work. Nothing's changing just because we have this target.
0:27:21.3 JD: And so instead, what happens, and I've seen this my entire career, is that some schools in Ohio regularly surpass this benchmark. Many other schools are nowhere near it. But my sort of a priori hunch, so my pre-testing hunch would be sort of like the overall system, the individual school's third grade reading test results are fairly stable. So what I mean is that low scoring schools stay low scoring, and the high scoring schools stay high scoring. And we sort of admonish the low scoring ones and celebrate the high scoring ones but there are people doing great things in all of those different types of schools.
0:28:14.3 JD: But the fact is, if you took the staff at one high scoring school and put them into a low scoring school, I think you'd be hard pressed to get the same results because so many of the other things that are in place at that high scoring school would not travel just because the staff travels. You know? And so that's, again, where frustration comes in. Then this 80% target really just becomes this sorting mechanism. It's not a roadmap towards improvement and it's literally sorting the schools, the ones that don't hit this benchmark and the ones that do but then you have these other things that happen. What teacher wants to consistently work at a low scoring school when they don't feel like they can do anything else? They can't affect change, what do they...
[chuckle]
0:29:05.3 AS: Have you seen the chart of that school?
0:29:08.4 JD: What did you say?
0:29:09.4 AS: Have you seen the chart of that school you're gonna go work at?
0:29:11.5 JD: Have you seen the chart, and so I'm gonna go work somewhere else that gets all the awards. And so you have this, if anybody studies systems, you have this sort of self-fulfilling thing that the rich get richer, sort of, right? The resources tend to pile up. And so instead what we need to do is think about this last sort of condition, by what method, by what method. Okay, if you're gonna say we're gonna set this 80% goal, by what method can we work together and achieve that? So I brought up one possibility is to sort of implement the science of reading. Now, doing anything as an initiative statewide is very challenging for any number of reasons because the obstacles are gonna be different in different locations that are low scoring.
0:30:02.1 JD: So I don't want to paint the picture that you can just sort of, when people say use evidence-based stuff, well, the evidence-based stuff often doesn't take into account many, many different contextual factors that are important. So I don't want to say there's some silver bullet because there's not. But what I do know is that I think you could argue that having these targets set like this that just sort are not good for anybody. And so maybe they're doing more harm than they are good. And I just want to at least take that into account, because this could work, not only for people working in schools, but also policymakers to think about these things, to at least understand. So if you told me, I've looked at the data for 15 years, I understand the capability of whatever system that is being studied.
0:31:05.8 JD: I understand how the results have shifted up and down over those 15 years, I understand the stability level of those results and I'm still moving forward with the target, I mean, I could accept that a little bit more than just completely arbitrary, but it still sort of begs the question, by what method? Who can do this? So I just think that's... That's really what I'm talking about when I'm saying goal setting is often an act of desperation, that the targets are arbitrary and that this thinking that should underlie this substitution of leadership for just picking targets is really the sort of the approach that we should be looking as, especially systems leaders, school systems or state education system leaders, that type of thing.
0:31:56.4 AS: And for the technical listeners or viewers who want to understand how you calculated the upper and lower natural process limits, maybe you can describe using standard deviation or tell us how you're doing that.
0:32:12.0 JD: Yeah, well, so it's, in this particular type of chart, you can see up here it says X chart, which there's, typically with an X chart, there's another chart below that charts the moving ranges between each successive point. So usually it's two charts together and it's called an XMR chart. Just to simplify things, I just included the X chart, but the XMR chart is sort of like the Swiss army knife of charts, meaning that it basically works with any type of data. It doesn't need to be normalized, as long as it's data that occurs over time. Now, people have strong opinions that that's not the case, but again, I sort of follow the teachings of Donald Wheeler and that's sort of his take on things and I you know, I've subscribed to that.
0:33:00.2 JD: But basically what the chart is doing is it's looking at each data point and it's using the moving range along with some scaling factors that were sort of invented by Walter Shewhart 100 years ago and then refined over time by statisticians like Deming to develop the formula. So it's not standard deviation. Your standard deviation doesn't take into account time. Standard deviation is the distance from the mean, but it's a sort of a static measurement. Whereas this is taking into account not only the variability, but also the time that variability occurred. So that's the key...
0:33:46.6 AS: In other words, if you had a process where you had 20 years and you've made a significant shift in the way you're doing things, if you were calculating a standard deviation based upon the whole data set, you would be using a data set that's really not reflecting the behavior of the system now...
0:34:08.5 JD: That's right.
0:34:10.5 AS: As opposed to sort of a rolling style or using the most recent periods as what you should be using to set the control limits.
0:34:20.1 JD: Yeah, that's right. So I think, yeah, so the big factor is the process behavior chart, the XMR chart, takes into account the point-to-point differences and standard deviation doesn't take time and how the changes occurred over time into account in terms of that calculation.
0:34:40.3 AS: Okay, so let's just wrap up.
0:34:41.4 JD: And I should say someone smarter than me on these should definitely fact check me on that, but I think I have the basics right.
0:34:49.5 AS: I have to admit that you got me thinking about one of the goals I've been setting for admissions into my Valuation Masterclass bootcamp and is what I'm pushing for something beyond what the system's capable of? And so while you were speaking, I was gathering my data and playing around and thinking about it in relation to what you're thinking because I definitely understand point number four, by what method, that we have to think about new methods or else we're gonna get the same result. But I also can say that I didn't understand the number one capability of the system 'cause I didn't have a control chart on it. Now I do as a result of this conversation. And so I challenge anybody out there that's listening or viewing, it's time to make your control chart.
0:35:38.6 AS: The second thing is I had an intuitive feel for what was the variation of the system but when I look at the chart now, it's much bigger than what I had thought. So I can see, in fact, yeah. And then number three is, is the data stable? And I just kept it simple, for my data points I just used standard deviation. And what I found from my upper and lower control limits is that I have one data point that broke through the upper 1 standard deviation line and also the upper 2 standard deviation line. And there was something very unique that I did at that time that we stopped doing for good or bad, but at least I can attribute that to a specific action.
0:36:31.5 AS: And then the fourth point that you've made, so capability of system number one, number two, what is the variation of the system? Number three, is the data stable? And number four, by what method? Of course, that to me is the whole key, once we've got, I think most people don't understand points one, two, and three about their system that they're trying to get a goal out of. But then by what method is really hard. I mean, we've been doing it this way, now... And it's not producing the result that we want, so what's the method to get us to the goal that we want? And I think to me, that's a huge challenge.
0:37:08.5 JD: Yeah. And a key to that last point, and maybe a good point to wrap up on, from a Deming lens and thinking about the system of profound knowledge and let's say the understanding of psychology is that in the state accountability system, the by what method goes like this, "By what method are you going to improve?" Right? But in the Deming methodology, it's, "All right guys, by what method are we going to improve these third grade state reading results?" Right?
0:37:37.5 JD: And in that first case, the finger wagging, what do people do? They try to protect their corner. "No, it's not that bad. We improved a little bit." "No, no, no, it's not us, it's them." So all the energy gets put towards trying to sort of write fiction about our results, which we talked about before, versus actually trying to improve things. And that's part of that, why you need all parts of the System of Profound Knowledge, including psychology, to actually bring about improvement with a group of people.
0:38:10.4 AS: So a great place to wrap up, as you're thinking about improving things, instead of saying "by what method" as a command, why not say "by what method" as a question? John, on behalf of everyone at the Deming Institute, I wanna thank you again for the discussion. And for listeners, remember to go to deming.org to continue your journey. And you can find John's book, Win-Win: W. Edwards Deming, the System of Profound Knowledge and the Science of Improving Schools on Amazon.com. This is your host, Andrew Stotz, and I'll leave you with one of my favorite quotes from Dr. Deming, people are entitled to joy in work.