Tuesday, January 3, 2023

Is Chat GPT "smart" enough to be accepted to the university?


Let's see how well it scores on the Swedish university entrance exam ("högskoleprovet").

 

For fun, I put Chat GPT to the test and it got a score of 39 of 70 at the May 2022 exam. If excluding the diagram-based questions, the maximum score is 58, giving Chat GPT a relative score of 67%*. This translates to a normed score of 1.0 (2.0 is the maximum), which beats 70% of the participants and qualifies ChatGPT, as one of the 28 students, for the Economy Program at Örebro University. Congrats!

 
Looking a bit more into details, ChatGPT scored exceptionally well on the Swedish verbal part with a score of 25/30. I was especially impressed with the reading comprehension. It scored worse on the mathematics part, 14/28, which makes sense considering how the model was trained. It is still better than random guessing, and some mathematical "reasoning" was awesome and correct.
 
What does it all mean?
We already see amazing companies emerging using these technologies, e.g., Sana labs and Hypertype. For 2023 even more advanced language models will be released, also a model trained specifically for Swedish.
 
How will this influence my job as project leader?
 
This is a fascinating topic, and we are just at the beginning of what is likely to be the most significant technological advancements of our lifetime. I will devote as much time in 2023 as possible to explore how this can be used for (project) leaders and share my findings.
 
Let's go!
 
*Depending what assumptions are being made, the relative score is between 56% and 67%.



No comments:

Post a Comment