The fallacy of pass rates as a measure of employee performance

South Africa’s Former Chief Justice Mogoeng Mogoeng once decried the unfair performance assessment of public prosecutors by the National Prosecuting Authority (NPA). Speaking during the release of the 2018/19 Annual Report of the South African Judiciary, former CJ Mogoeng criticized the NPA’s approach, which vilified prosecutors for not getting enough convictions in the courts of law, or having too many acquittals. A prosecutor whose portfolio had too many accused persons walking away, without a guilty finding was regarded as having dropped the ball.

Listening to the former CJ’s animated speech evoked similar feelings in me, quite often induced by the expectations universities have of academics. Whilst the performance of prosecutors is unfairly assessed using their conviction rates, the assessment of lecturers in the academic sector is, inter alia, pegged on the pass rates in the modules they are responsible for. This assessment becomes central for promotion purposes and performance-based bonuses etc. If the number of students that fail a particular module is regarded as alarmingly high (and at times the mere fact that some students failed), reflects on the lecturer concerned. In other words, university management understands this to mean that because students are failing then there must be something wrong with the lecturer concerned, or their mode of delivery of the learning materials. The use of these rates which are totally outside the control of the officer being assessed, forms of the basis of the analogy I wish to draw between the case of the prosecutor and that of the lecturer.

What stands out in both cases is that it is not the person being assessed (prosecutor/lecturer) that sits for the assessment that determines the success rate; or that has the final say in the trial process. In the case of prosecutors, as the CJ alluded to, they simply present the case for the state as best as they can. Many factors will affect this task of the prosecutor. The quality of evidence that investigators are able to supply for the prosecutor to work with, the ability of the prosecutor to utilise that evidence and to persuade the court to favour its version of events, and the disposition of the judicial officer determining the matter. All these factors do not necessarily give the prosecutor the final say in the matter. The conviction or otherwise of the accused remains the exclusive preserve of the only impartial arbiter in this matter, the judge.  In the case of the academy, the lecturer designs and presents the learning materials, guided by the subject’s NQF level, the graduate attributes of that institution, and the desired learning outcomes of each module. No matter how beautifully crafted learning materials are, the lecturer does not study for that module’s exam, neither does he/she sit for that assessment. The lecturer simply assesses the written work of the students, who have been furnished with all the materials they require to successfully pass that particular module. It is quite astonishing that the performance of the student (in the case of the lecturer) and of the criminal justice system (in the case of the prosecutor) boils down to these two offices.

The academic sector today simply does not take into account that not all students who enrol for a particular module have a desire to pass; or if they do, have the motivation to work towards it. Similarly, the prosecutor’s case is complicated by the fact that a majority of accused persons do not set out to get a conviction, but to be absolved from their crimes. Their defence team will therefore do all it can to ensure their client does not get convicted. (A few individuals will desire a conviction for one reason or another – such as earning stripes within a gang or being returned to prison because that life provides free lodging and maintenance). That is why it is quite mind boggling how these performance assessments would then blame the lecturer and the prosecutor for low success rates.

The demands placed on these two offices to achieve high success rates have the potential to create a situation where, for the purposes of being seen to be meeting one’s key performance indicators, the prosecutor or lecturer would end up manipulating system to achieve those rates. For the prosecutor, this could include fabrication of evidence, or as the CJ indicated, disregarding exculpating evidence. Prosecutors would end up withholding information which they know would advance the case of the accused in order to meet the desirable conviction rate. This could also extend to prosecutors forging ahead with prosecution in the hope of obtaining a conviction, even where it is not in the interests of justice to do so.

For the university lecturer, this could include setting ridiculously easy assessments, being overly lenient when marking scripts and mechanically processing students’ assessment with the sole goal of meeting the ‘acceptable passing threshold’.

The bottom line is that the use of pass rates in the academy as an indicator of employee performance does not give a holistic picture. At best it scapegoats the lecturer without getting down to the root causes of students’ failures. Similarly, high pass rates falsely give universities confidence in their performance without interrogating the possibility of manipulation of the outcomes. Unlike research outputs, which are measurable, pass rates should not be used by institutions of higher learning to assess the performance of staff due to their unreliability. What compounds the problem is that in most instances the performance planning phase does not involve the parameter of ‘high pass rates’, yet it forms part of the performance assessment and review. The performance planning phase only focuses on ‘teaching and learning’, which cannot and should not be equated with a high pass rate. The biggest risk in insisting on high pass rates is that universities end up being accused of being ‘qualification mills’ or giving out free degrees, thereby negatively impacting the integrity of the qualification. This does not only harm the university’s integrity, it also affect its important stakeholder, the student.

Leave a Reply

Your email address will not be published. Required fields are marked *