JENUI 2023: best paper

On November 30, 2022, OpenAI introduced ChatGPT. Like most of us, we were surprised 🤯 with its incredible capabilities. From the first moment, we wondered what impact it would have on the different facets of our day-to-day life: in Computer Engineering, research, and teaching. Based on an idea from our colleague Roberto, we decided to focus on the latter.

The result of several months of effort is our work “Impact of ChatGPT in the evaluation methods of a Computer Engineering degree “, which we presented to the JENUI of 2023 and for which we have been awarded the prize for best paper of the conference. In this edition, dominated by ChatGPT (something that has kept some speakers awake at night), the paper “To rival or not: analysis of Wooclap’s competition mode based on performance and audio processing “, by our colleagues from the University of Murcia, was also awarded.

Here you can see Roberto presenting our experience to the rest of the attendees (about 10 minutes plus question time):

We will share the proceedings as soon as they are published so you can read the work and give us your feedback. In the meantime, here is the abstract:

Generative Artificial Intelligence has undergone an essential evolution during the last years and especially in 2022. One of the intelligence that has caused the most buzz in academia is ChatGPT, which provides an interface that significantly simplifies the productive use of a large language model. These language models can parse and generate text with great speed and quality. These capabilities could have a relevant impact on teaching-learning methodologies and assessment methods. In order to analyze the possible impact of ChatGPT on assessment methods, in this work, ChatGPT’s ability to solve the exams of 15 Software Engineering subjects of a Computer Engineering degree has been tested. The results show that ChatGPT could noticeably impact the assessment methods; it can overcome many questions and problems of different natures in multiple subjects. As a fundamental contribution, a detailed study of the results by type of questions and problems is provided, which allows us to establish recommendations to be taken into account in the design of assessment methods.

Are you interested in the subject? Do you want to collaborate with us? Contact us, and we’ll talk.

Update: You can read the cited work right here.