Verification of the 2014 Annual National Assessment
DATE: 4 December 2014
MEDIA RELEASE 04 December 2014
An analysis of the data collected to verify the 2014 Annual National Assessment (ANA), showed that with a few exceptions, the ANA results are consistent with the verification results, which is good news for confirming the quality of the assessments of learners in grades 3, 6 and 9. A Human Sciences Research Council (HSRC) team reported on the verification findings at the national and provincial level and for the subject areas of mathematics, home language and first additional language.
The 2014 ANA assessment results http://www.education.gov.za/LinkClick.aspx?fileticket=/pJ4g17UXZk=&tabid=36 was released today [4 December 2014] by Basic Education Minister Angie Motshekga at the Muzomuhle Primary School, Diepsloot, in Gauteng.
Over the past four years, the ANA has played a critical role in informing the provision of quality education for all learners in South Africa. This assessment has two major components – universal and verification ANA.
In universal ANA all tests are marked by class teachers who are expected to use the results to diagnose performance in literacy and numeracy and develop strategies for improvement. The verification component is designed to track the ANA processes for the purpose of ensuring that we have valid and reliable data supporting the findings from Universal ANA.
The verification ANA design also includes collecting scripts from a sample of learners from sample schools. These scripts are marked by an independent agent. The data are then captured and then the HSRC analysed the data to assess the findings from universal ANA.
Given that the verification ANA findings are based on a sample of the data we expect errors due to sampling”, explained Dr Vijay Reddy of the HSRC, who led the team that carried out the analysis of the 2014 verification ANA data. These errors are used to develop what is referred to as confidence interval (CI) so that for each finding there is an interval within which the true finding from the population should fall.
“In a perfect scenario where educators assigned marks are true reflection of learners and the sample of learners and schools are accurate representation of the population of schools and learners in South Africa, we would expect the findings from universal ANA to be within the range of findings from the verification ANA. And, we would say, the universal ANA results are consistent with the verification ANA results,” Reddy said.
The findings indicate that in general, the Universal ANA results are consistent with the Verification ANA results - about 80 results were analysed with 45 indicating consistency between universal and verification ANA, 18 with universal ANA results falling outside but less than 2% from the upper boundary of the verification results, and 17 with substantial anomaly between the universal and verification results.
This would mean that while imperfections in the ANA processes did not have a major impact on the universal ANA findings, the inconsistencies in the universal and verification results require investigations into which ANA processes that might have created this problem.
“We have asked DBE to investigate the anomaly, for which there could be a number of reasons, including sampling within schools that was not random; inflation of marks by teachers; and measurement errors associated with the estimates from the universal data. Verification ANA is and would continue to play a critical role supporting accuracy and reliability of universal ANA", Reddy said.
For further information, please contact Ina van der Linde
E-mail: ivdlinde@hsrc.ac.za; Cell 0823310614