Virtually testing times

What do tests measure?

One reader of last week’s post made a comment that echoes that of many teachers we know in these days of teaching by video link.  He pointed out that the grade he gives in his Calculus course is supposed to measure how well the student has learned the material; however, he has no control over the student’s environment, and cannot see what help might be used.  In particular, there are programs that can perform symbolic integration, differentiation and algebra, the things he’s supposed to be teaching and the students learning.  So instead of grading the student, he’s grading a sort of cyborg, the combination of student and any computer resources the student may have at hand.

Rather than lamenting this as a failure of the system, however, our commentator observed that this may be a fair measure of his students’ ability.  Assuming that they will in the future have access to computers at least as powerful as those they have now, his test will measure how well they’ll be able to do Calculus.  The lone engineer forced to do esoteric calculations in his head to save the day was a staple of mid-twentieth-century science fiction, but is hardly realistic.

Here is another example.  Our astronomer has on his bookshelf a two-volume work entitled Applied Optics and Optical Design.  It was the standard reference for lens (and mirror) designers from the time it appeared, during the First World War, until capable optical design software appeared over a half-century later.  A large part of the first chapter is advice on how to set up and work through a tedious calculation using tables of logarithms of trig functions, to fine accuracy, without making a mistake.  Nowadays, not only is a full page of this work taken care of by pushing the button of a calculator; the whole process of trying out variations of a design is done automatically.  Manual skill in many areas of mathematics has become rather a John Henry phenomenon.

(This doesn’t mean that the process of learning math has become uniformly less tedious.  Our tutoring consultant reports that textbooks now include quite a few problems of numerical approximation, requiring many punches of the hand calculator’s buttons.  That would have required days of work in the era of log tables, and so they were not found in the textbooks of the time.)

So should we simply give in, and leave it all to the machines?  Of course not.  We’ve already commented on undesirable consequences of abandoning maps for computerized directions, and of relying on translation programs instead of learning a foreign language.  And if we test the combination of students and machines, we widen the inequality that already exists between those who can afford good ones and those who can’t.  Most important, the the students need some idea of what their computers are doing, and how.

At the most basic level, they need a way to check the results on their calculator screen.  Just this past week our tutor had a student who decided that 5 out of 400 was 12.5%; key-punching errors of this sort are common, and students who notice that the answer should be just over one per cent are rare.

As computer functions become more complex, the problem of supervising them grows enormously.  In this sense being a cyborg is harder than wielding pencil and paper.