Metal detectors were the door to the school, the first weapon screens I had seen outside of an airport. It was 1975, inner city Chicago. I was delivering a pilot test for my employer, a testing firm under contract with the Chicago Board of Education. We were creating the Chicago Life Skills Test, a multiple choice, machine scored test given to juniors in high school to find those who needed help to graduate. The Board, the teachers, and my employer all wanted to end the scandal of illiterate high school graduates.

But the metal detectors told a different story. I believe, after my experience, that the Chicago Life Skills Test was meant to cover political errors, shifting the blame to the students and individual schools. It appeared to me that the elected and unelected officials and lawyers of the school system gave the contract to my employer, not to improve the minds of students, but to protect the Board from law suits by illiterate graduates. You will remember that in those years a few such suits were successful. While teachers and parents aimed their efforts toward the students and the classroom, the political appointees, without realizing it, aimed their efforts toward the voters and the courtroom. The unspoken, best strategy to avoid losing in court was:

1. Don't graduate kids that can't read.
2. Don't fail them directly. Use a test to force them into remedial classes which make them drop out voluntarily.
3. Use a test to judge graduation. It will protect teachers from pressure to pass the unlearned.
4. Believe the test is there to find the kids who need help, hiding the real, legal purpose, even from ourselves.
5. Hire a consulting firm to write the test so we can blame them if it doesn't work.

To save money, the consulting firm hired me, not an education professional, to edit the test. I had no idea how the system protected itself and I made a big mistake. I read the contract and took it seriously. August Ed.D.'s, heads of university departments of education, with vitae the size of a small library's card catalog, would write the items (professional lingo for questions). Actually, they would get their graduate students to write the items. I would fit them into the matrix (professional lingo for a list of stuff they want to test for), typeset them with graphics, print them, and run the pilot tests. My employer would then use his excellent software to analyze each item pilot tested and build the final version. The deadline came and a dribble of items came in, far too late for me to ask the professors for a second try. Were they any good? You be the judge with this invented item similar to what I received:

Here are the two guide words at the top of a dictionary page: fit and flag.

Which of the following words would appear on this page?
a) five
b) fat
c) flame
d) fist
e) float

Sounds good, sounds professional. But imagine a scene six months later, a street beside a city construction project. The reporter from channel 7 has found a copy of the test and reads this item to a construction worker whose son has just failed the test. Imagine the soundbite on the six o'clock news: "What do you mean guide words in the dictionary? I haven't opened a dictionary in thirty years, and I've got a city job!"

The fix was easy. I rewrote the item to use the yellow pages. Same skill, same literacy, same alphabet, same kid, same father, different quote on the six o'clock news. "Well, maybe my kid ain't ready to graduate. I have to use the Yellow Pages. I guess he better learn how too."

The contract called for us to deliver four pilot tests for review and approval by the Board's internal expert. I saved what I could from the academic efforts, rewrote many to refer to this planet, and made up most of the items myself.

To give him credit, the expert liked the tests. He saw the practical orientation of the items and encouraged it. And he pushed for a lot more visuals: photocopies of bus schedules, menus, store sale signs, and real help wanted ads. Great stuff for parents and voters.

The el (pronounced like the letter L) is Chicago's system of elevated trains, a system used every day by the people of the inner city. I put a photocopy of an actual el schedule in the test and then asked questions like "CJ has a job interview at 9:00 am. The company is a block away from the State Street station. What is the last train CJ can take to get to the interview on time?" Then I listed five different train times from the schedule, only one of them right.

With these practical and visual items, I went through the metal detectors and we went through the pilot tests to see which items would be useful.

The results were surprising but not nearly as surprising as how the system responded to them. The suburban schools, the kids who did well on the other items, blew the item about the el big time. Don't blame them. Have you ever seen a schedule for the el before? Believe me, it is high art to know what when goes with which where. But the kids in the inner city, behind the metal detectors, did well. They got the right answer, even when (from the rest of the test) it was obvious that they could barely read.

But that was not the kind of result the customer seemed to want. I got fired before the final version of the test was printed so I can't tell you how it came out, but I can tell you about the argument.

My employer's software produced statistics about the distractors (professional lingo for answers in a multiple choice test, even the correct response). A good item was not one that discriminated between ignorance and knowledge, but one that discriminated between good students and poor students. Therefore, if everybody selected the same distractor, good and poor students alike, the item was discarded, even if it was important stuff to know, even if it was right. And no matter how important, if the good students didn't get the right answer, the item was thrown out. The only items that survived were those that got different responses from good students and poor students.

The items about the el were not regarded favorably. They must be bad discriminators, it was hinted, because they contradicted every article of faith held by the educational kingdom. But even the other items, the ones that tested well in pilot, were not used to help students, but to protect the system.

All of that language and all of that "science" hid the fundamental error, a circular reasoning, a pattern of bureaucratic self-protection. The Chicago Board of Education defined a good student, not by success in life, not by measures of character and knowledge, not by tests of skills or accomplishments, but only by grades and graduation from a good school. And further, good schools were not defined by the ability to get around on the el, but by the money spent on them and by the degrees and professional status of the teachers in that school. It should not surprise anyone that the test made up from these results proves that we need more money for schools. Nor should we be shocked by the low scores of students in poor schools. In short, political clout decided what was "excellent" in Chicago. I was used to hide this fact from parent, students, and teachers.

Accidentally dropped into an opportunity to reform a bit of our educational establishment, I managed to actually make a difference. But it made no difference to the great machine we call public education, which absorbed my work and turned it to its own purposes, using my efforts for its own self-preservation. Money, power, politics, and an insular mentality make small reforms and little victories meaningless. Contributing to what should have been a deep reform of methods and goals in tax funded schools, I became convinced we need to end any role for the state in teaching our children.

Charles Johnsen is a Lutheran preacher and computer chip designer in Aurora, Colorado.

This article is copyrighted by the Alliance for the Separation of School & State. Permission is granted to freely distribute this article as long as this copyright notice is included in its entirety.

[top]