Supporting teachers and learners of programming by understanding feedback on syntax, semantics and style

Funding year: 
2 years
University of Auckland - Te Whare Wānanga o Tāmaki Makaurau
School sector
Project start date: 
January 2018
Project end date: 
March 2020
Principal investigator(s): 
Andrew Luxton and Ewan Tempero
Research team members: 
Paul Denny, Nasser Giacaman, and Tyne Crow (University of Auckland)
Research partners: 
Margot Phillipps, ACG Sunderland College Charlie Smith, Pakuranga College


Project description

Effective teaching requires providing high-quality feedback. The feedback needed for computer programming can be classified as being: for syntax (how to write valid code), for semantics (how the code behaves), or for style (how the code looks). Each category has different challenges in how to provide high-quality feedback. Each requires different skill sets to evaluate the students’ work and to provide the appropriate feedback.


Our overall goal is to support teachers in providing high-quality feedback on the syntax, semantic, and style problems in students' computer programs, and to improve the learning experience for students. Programming problems encountered by students often take an inordinate amount of time to solve, and often requires assistance from an expert. This impedes learning, not only due to the time required, but also due to the frustration it creates. While the computer will highlight syntax problems, the messages are often confusing. While semantic problems are visible from incorrect behaviour, determining what is wrong requires an ability to diagnose the problem. Problems of code style are as difficult to evaluate and explain as problems in writing prose.
For teachers to help students resolve these problems, they need a great deal of experience, and even then, the time it takes makes it difficult to provide feedback to all students in a timely manner. Our aims are to provide support for teachers to help them gain the necessary experience quickly and improve their efficiency when helping students, and to provide support to students to be able to resolve their own problems more quickly.

Why is this research important?

Understanding how to program computers is now seen as an important skill, one that New Zealand has in short supply. This importance is reflected in the recent introduction of Digital Technologies at all levels of primary and secondary schools. However, there is also a shortage of experienced teachers in Digital Technologies already, and the demand for them will only grow. This research will provide relevant knowledge and support for new and existing teachers. This will improve the effectiveness of existing teachers, and upskill new teachers more quickly than is possible currently. It will improve the quality and timeliness of the feedback provided to students learning to program computers. It will also provide support to students, improving their learning experience.

What we plan to do

We will collect baseline data from teachers identifying the major areas where improved feedback will be useful through focus groups and surveys and work product (computer code) from students. The data collected will be both quantitative (e.g. measurements of code, answers to survey questions) and qualitative (from focus group data and free-text responses in the survey).
We will develop interventions based on this data targeted at both teachers and students. Interventions for teachers might include resources to develop their knowledge in how to provide effective feedback on syntax, semantics, and style. Interventions for students might include learning resources for the three categories. Interventions for both students and teachers may include tools to support interpretation of syntax error messages, or evaluation of code style.
We will conduct a formal evaluation of the effectiveness of the interventions. Differences in the baseline post-intervention survey data will be analysed to determine if a quantitative difference in confidence or attitude is measurable, or if there are thematic differences in responses to qualitative questions.  Student programs collected as baseline data will be compared with programs collected post-intervention to determine the impact of the interventions on students.

Our partners

Margot Phillipps is an expert teacher of programming from ACG Sunderland College. She will provide assistance and advice on recruitment of digital technology teachers, working with the teaching community, and proposed interventions.
Charlie Smith is an expert teacher of programming from Pakuranga College. He will provide advice and assistance with proposed interventions and potential development of resource material.

Contact details

Andrew Luxton-Reilly ​
Science Centre – Building 303
Level 5, Room 523
38 Princes Street

Ewan Tempero
Science Centre – Building 303
Level 5, Room 533
38 Princes Street