PD in development of an automated assessment and feedback system for education

The high-level learning goal of the Machine Learning for Signal Processing (ML4SP) course is to educate students towards a fundamental understanding of the theoretical background of deep learning methods and demonstrate their similarities to many other, since long-established, signal processing methods. The course focuses specifically on those methods that embed strong priors in the associated neural network architecture to enable data and memory-efficient deep learning, characteristics that often prove vital in healthcare applications.

When designing a deep learning method, a user typically makes many implicit assumptions. These assumptions to a large extent determine the success of the neural network. In the ML4SP course, these assumptions are made explicit, providing the students the means to better design their neural networks and to also enable them to design neural networks that combine physics- and data-driven models. Many neural network designs disregard knowledge that the user already has about underlying structures in the data or about characteristics in the data that are affected by the sensors with which the data is collected. These ignorant neural networks will need to learn all these structures, requiring lots of data. Unfortunately, in many applications
(e.g. healthcare) data is scarce and costly. By embedding knowledge that the user already has, the neural network will only need to learn those aspects that the user cannot describe with physical models yet. This leads to high data efficiency, smaller memory footprint, and faster learning.

The course design is such that theoretical lectures are combined with practical assignments. During these assignments, students will implement various neural networks, such as variational autoencoders, to bring the theory into practice and deepen their knowledge and understanding of the course content. From an educational perspective, these assignments have proven relevant for achieving the learning goals of the course and they are appreciated by the students, as evidenced by their high ratings. Unfortunately, at the same time, these assignments pose a challenge to the design and execution of the course. Ideally, students would get the freedom to experiment with their neural network designs in the way they think is best and subsequently receive detailed feedback about the rights and wrongs of their ideas, closing the learning cycle and making the course design effective from a didactic point of view.

The primary objective of the project is to develop a method for assessing programming codes of students for assignments of the course Machine learning for signal processing to:
provide quantitative feedback by means of a grade;
provide qualitative feedback by means of detailed feedback to students about mistakes and possible improvements.

As a secondary objective, the method should generalize to other types of assignments to enable its use in other courses as well.

The project is expected to yield a computer-based tool that can be tuned to the specifics of a given assignment and that can import and assess student submissions, providing both quantitative and qualitative feedback. During the project, the performance of this tool will be evaluated by applying it to a database of submissions and their assessment that has been collected over the years of the course.

This project will be embedded within the Signal Processing Systems group at the Department of Electrical Engineering, Eindhoven University of Technology and funding has been obtained from the BOOST! program that aims to innovate teaching methods.