‘Theatre of fairness’: making retrospective adjustments to dropout warning systems purchased from publishers merely shifts the errors without reducing them

Six equity-based adjustments were tested using data from a public secondary school in Ontario. The indicators show a consistent trend, but small marginalised groups remain underserved.

Higher education institutions are increasingly buying early warning systems that are supposed to identify students at risk of dropping out. Because these tools are proprietary, colleges can't inspect or modify the model—they just need to tweak its outputs to mitigate inequality. McConvey and her co-authors wanted to measure the value of this leeway.

Using student records from a public college in Ontario, they applied six "post-hoc" equity interventions to a search alert system, simulating the constraints of a purchase contract. By tracking precisely who receives false positives and false negatives, they find that corrections redistribute gaps between groups without addressing them on a regular basis.

When the size of the group is used as a criterion

Two of the implementations even benefited already advantaged groups, because they defined disadvantage based on group size; small, marginalized populations lost out. The authors speak of "equity theater": dashboards display metrics that are getting closer, while the burden of errors persists, or even increases, for some students.