ChatGPT in the resolution of a math exam: Results obtained in Portuguese and in English language
- DOI
- 10.2991/978-94-6463-380-1_5How to use a DOI?
- Abstract
Artificial Intelligence has had a remarkable development in recent years, being brought to public attention due to the emergence of advanced language models such as ChatGPT. These models have been reported to be capable of achieving passing scores in examinations required for accessing professional orders in fields such as Law and Medicine. The purpose of this study was to examine the capabilities of ChatGPT to answer correctly the questions of the Portuguese Mathematics 2022 12th grade exam (Matemática A). The questions of this exam were given to ChatGPT both in the original language (Portuguese) and in an English translation that was produced also with ChatGPT. Some questions had accompanying figures, described textually in the exam, that were not given to ChatGPT. The results of the research showed that, in both languages, ChatGPT did not achieve the minimum passing score of 95 points (out of 200). The performance in English was slightly better, with a score of 77 points, compared to 63 points in Portuguese. The study also showed that, when the solution of a question must be decomposed in several steps, ChatGPT makes errors in those steps more frequently than when asked to solve those steps separately. Therefore, ChatGPT performed better when given simple, direct, questions compared to complex problems that require combining multiple pieces of information. The study also analysed the consistency of ChatGPT’s answers, concluding that ChatGPT may give both correct and incorrect answers to a given question, with similar assertiveness. In conclusion, these results show that while ChatGPT shows promise in answering some questions, there is room for improvement in the domain of mathematics.
- Copyright
- © 2024 The Author(s)
- Open Access
- Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
Cite this article
TY - CONF AU - Cândida Barros PY - 2024 DA - 2024/02/29 TI - ChatGPT in the resolution of a math exam: Results obtained in Portuguese and in English language BT - Proceedings of the International Conference on Lifelong Education and Leadership for All (ICLEL 2023) PB - Atlantis Press SP - 37 EP - 47 SN - 2667-128X UR - https://doi.org/10.2991/978-94-6463-380-1_5 DO - 10.2991/978-94-6463-380-1_5 ID - Barros2024 ER -