ChatGPT3 has been all over the news recently. In the past few weeks, researchers have announced that it performed within passing range on the three components of the U.S. Medical Licensing Exam, and a recently published paper claimed that it received a B/B- grade on a Wharton MBA-level operations exam.
The fact that the bot passed the exam is not overly surprising, but some of the findings in Wharton Professor Christian Terwiesch’s paper titled, “Would Chat GPT Get a Wharton MBA? A Prediction Based on its Performance in the Operations Management Course,” are. While the bot handled basic operations management and process analysis questions extremely well, Terwiesch emphasized that the bot made some fairly simple calculation errors within the exam.
“Chat GPT at times makes surprising mistakes in relatively simple calculations at the level of 6th grade Math,” he wrote. He continued on to note that these errors can become, “massive in magnitude.” Additionally, he noted that the current version of the ChatGPT3 could not handle more advanced process analysis questions, such as those that include multiple products or demand variability, and that having “a human in the loop” proved meaningful. When a human expert provided a hint, not only was the bot adept at correcting itself, but it was able to learn so that the hint was not necessary in later iterations.
Professor Terwiesch also provided his view of the implications that the technology may have on the MBA degree itself. Rather than expressing concern, he thinks the bot could afford students and professors an additional learning resource, specifically in developing necessary leadership skills in critical evaluation and the assessment of alternatives.
“MBA students are preparing for leadership positions, in which they will have to make managerial decisions based upon actions brought to them by ‘consultants, co-workers, and direct reports.’ Using the bot to generate plausible, well-presented (although possibly incorrect) alternatives in response to a scenario can provide students with meaningful experience in evaluating alternatives,” he said.
Read the whole paper, including the additional implications, here.
Related: