Through a weird set of circumstances, it seems that i have the summer to focus on writing. I’ve spent the past three weeks working my way through the piles of writing I’ve done over the last 14 years and one of the key themes that I’ve found is the disconnect between our goals for education and our practices. The choice of assessment as a place to start this journey is an incidental one, but it’s been really interesting. I’ve dived into the history of assessment in our field and thought it might be fun (for me at least) to track some of the things I’ve found.

Inevitably I ran into some trouble around what I actually meant by assessment or, as the conversation developed, what i meant by grading. There is a confusing history to this conversation, and I’m not sure I’ve been able to track all of it, but I’ll do my best to lay out what I’ve found and trust that someone will fill in the blanks that I’ve missed.

When i hear about AI improving the ‘success’ of students I’m left with the question “improving what exactly?” Are we making them better at compliance? Is game based education making students more creative? It’s just a question of what our goals are. Here are my notes.

Grading for what exactly?

For the purposes of this discussion, let me suggest that one way of seeing ‘grading’ is as a form of assessment that makes a scaled judgement of the performance of a student against an arbitrary standard. I might give you a pass, or an A or a 72% or call you an ‘Inferiores boni‘ or whatever else you can come up with that has as scale of winners and losers. I say ‘arbitrary standard’ because, as every teacher secretly knows, you have to make up a grading rubric. You can call it valid or verified or rigorous but one way or the other someone is still making it up.

Another basic premise that I would posit is that grading is an extrinsic motivator. It is the way that we as arbiters of the education system motivate students about what they should learn, when they should learn and, inevitably, what it means to have learned. I got an A. I learned. As an extension of this I agree with Grant and Green when they say

[extrinsic motivators] improve performance in “algorithmic,” or repetitious, tasks but are less effective or even counterproductive at “heuristic” tasks that require creativity, concentration, or intuition.

Grading is good at ‘encouraging people’ to do complicated tasks that are often represented by memorization, obedience and linear thinking. If those are our actual goals. If our goals are complex and include things like creativity… we’re looking to support intrinsic motivation. Grades don’t support intrinsic motivation.

Assessment as gate keeping (pass/fail)

We have a long history in education of thinking about assessment as a method of quality control or gate keeping people from a particular field. We see it now in things like the MCAT & LMCC (for medicine in CAD) and Red Seal examinations for The Trades. They are also a good mechanism for maintaining the status quo. You might argue that having a group of people in a field maintaining the status quo is a good thing, and maybe it is, but it tends to slide its way towards keeping out people with new ideas or who come from different backgrounds.

You can step all the way back to the first universities at Paris to see (you passed/you didn’t pass). A student was nominated by his Master to do the examination, to be able to prove, in a public discourse that they were prepared ‘to lecture’. They were judged by a committee of Masters which included a representative of the Pope and a representative of the city of Paris and if they succeeded, they were granted the ‘licence to teach’. Wilbrink

According to Mary Lovette Smallwood, there are records of this being done at Harvard in the 17th century leading to the development of the four tier system at Yale in 1785 – optimi, secondi optimi, inferiores boni, and pejores. We’ve moved, in a sense, from pass/fail to awesome pass, pass, kinda pass and not really passed.

A word on the Smallwood thesis. If you think no one will ever read your PhD thesis… take heart, that one is cited EVERYWHERE.

So, if your goal is to make sure that people who become ‘certified’ are the same as the people already certified, i can see how this works. I can also see the problems… onward.

Catechetical assessment

Another thread of assessment I found in the archives is the call and repeat model. It dominated medieval classrooms and, in some cases, still does today. I say a thing, you repeat that thing, I judge whether or not you said the thing I did. There are any number of reasons for taking a catechetical approach to learning. I’m going to use Charlemagne’s 789 edict (Admonitio generalis) as my example. It set forth some goals for the training of priests and regular folks about how their religion actually worked.

Charlemagne was a bit of a literalist. He was desperately concerned that Priests were mispronouncing their benedictions. He was, in effect, worried that people were going to hell because God couldn’t understand bad Latin. The Correctio was a series of quizzes designed to train priests in the basics of what they needed to know to keep people out of hell. (Rhinj)

Basically… there were verifiable things that needed to be figured out. I, as maybe the bishop, would ask you the questions… you would answer. Then you would return to your monastary/parish and setup a school where you transferred these lessons to other people. We all know what’s true. You just have to remember it. I have no way to prove it, but it stands to reason that our idea of school is heavily impacted by this… hence all the catechetical approaches still existing in our school system.

The thing I like about this example is that the goal that Charlemagne has was very clear. His practices were perfectly lined up to them. Believe this. Now remember it. Now tell other people the exact same thing.

William Farish and the birth of individual grading

There is certainly a point at which we moved from ‘yeah, you got the general idea’ to ‘you got 72%’. There are a number of people who would like us to believe that William Farish is the person who is responsible for the innovation. Some of them would even go so far as to say that he did it because he thought he could process more students and make more money. It took me a while to track down how this story developed… but here goes

2000/2005 Hartmann writes about William Farish founding grading so he could make more money from his students. This is oft cited, but it took me forever to find what he was citing. It describes Farish as the evil founder of grading. It seems that Hartmann was quoting Postman.
1992 Neil Postman “In point of fact, the first instance of grading students’ papers occurred at Cambridge University in 1792 at the suggestion of a tutor named William Farish.” Please note that all the extra ‘Farish hated his students’ stuff is not present and seems to have been… colourized… by Hartmann.
1967 Hoskin Postman is citing Hoskin and for the rest of the story I’ll turn it over to Christopher Stray’s excellent article

Hilken (1967, p. 40) stated that as moderator in 1792 Farish had introduced the practice of assigning marks to individual questions. The source Hoskin himself relied on (Hilken, 1967) was a short history of engineering at Cambridge written by the then secretary to the faculty. Of the sources Hilken gives for his account of William Farish only one makes any reference to marks. This is Farish’s obituary in the Christian Observer: ‘He was the means of introducing into the University of Cambridge the system of classifying the candidates for a degree according to the number of marks obtained at their examination’
(Anon, 1837, p. 675; copy, with other sources on Farish in Magdalene College Old Library, M5 29).

So. Farish kind of introduced grading to Cambridge. But the story that is all over the net about being the grandfather of grading is mostly nonsense.

Grading and assessment as individual process

Whether our friend Farish is the actual founder, 1792 is close enough to the time where this started to happen. We have other instances,

• Joan Cele as the initiator of the grade system of education (grades 1-8) and exams created to judge when you’ve passed to the next grade (Wilbrink)
• ‘sub omni canone’ (outside the canon) from the Jesuits, and the idea of grading of this kind being imported from the chinese bureaucratic testing. (Schubert)
• Class point systems and the ‘nota asini’ (ass’s mark) for students who didn’t get enough points. (Wilbrink)
• Prizes awarded like with the Mathematical Tripos competition at Cambridge, where the winner got a life-time annuity. (late 18th century)

There have been lots of innovation and encouragements. They are, for the most part, directed at trying to get lots of people to ‘work’. They intend to measure the compliance of our students. Is our goal about compliance? Or, as it says in basically every strategic plan in education in the world, are we trying to support independent, creative citizens?

My last thoughts are with Hoskins and systems of control and a mathematised model of reality.

Hoskin (1979) emphasised the importance of such a change, as a significant moment in the development of the fine tuned marking system. In Hoskin’s neo-Foucauldian narrative this even becomes a crucial one in the emergence of a modern system of control, of ‘normatising individuation’. It was ‘a most momentous step, perhaps the major step towards a mathematised model of reality. … The science of the individual was now feasible. … The blunt weapon of banding yielded to the precision tool of the mark’ (Hoskin, 1979, p. 144). (from Stray)

Are we happy with the ‘mathematised model of reality’ that lives at the root of what we call Artificial intelligence? Does it serve our goals?

Works referenced