Effectiveness of the Tutor Learning Initiative

The TLI provides small-group tutoring (2–5 students) to support those whose learning has been disrupted or fallen behind in literacy and numeracy. Members of my team collated student-level assessment data (specifically the Progressive Achievement Test in reading and mathematics), matched participation data in TLI and non-participation cohorts, and constructed a statistical analysis framework to test whether participating students gained more than comparable non-tutored peers.

They then used regression-based modelling and baseline-matched comparisons to control for prior achievement, thereby mitigating bias from selecting weaker students into tutoring. Specifically, by using more sophisticated methods than previous analysis, the team addressed regression to the mean and Simpson’s paradox effects that can otherwise distort simple pre-post comparisons.

A highlight of this work was the engagement with the auditees to explain the statistical methods and findings. The results were contentious and our data analysts’ ability to respectfully defend the methodology and findings was critical to the audit’s success.

For full details see the full report.