Developing the competition

Blum enlisted the help of two computer science graduate students, Devang Jayachandran and Pranit Maldikar, to develop the problem set for the competition. They had already worked with Blum on a conference paper where they examined ChatGPT and what it could do in the context of programming contests.

“We had a pretty good sense of its capabilities — what kinds of problems it could solve, what kinds of problems it couldn’t completely solve,” Blum said. “We worked to come up with a problem set that would require the students to interact with ChatGPT. So, ChatGPT would certainly help get them started but wouldn’t solve completely for most of the problems.”

Shaub promoted the competition around campus and visiting classes to make a quick pitch. She’d first start by asking how many students had used ChatGPT — and she found most students raised their hands.

The organizers hoped for 50 competitors, but around 135 students participated.

“While the turnout for the event was beyond anything we had hoped for, the real measure of success for me was to see how much students’ confidence grew by the end of the competition, which can be attributed to Dr. Blum and his vision to use ChatGPT to create an unconventional competition,” said Kelsey Kirk, faculty adviser for the Women in Tech organization.

Blum said the novelty of ChatGPT increased interest, both among students and faculty. He noted that some political science students participated because their professor thought it would be good to expose them to it.

“These tools are going to have a variety of society-wide impacts,” Blum said.

Jayachandran noted that programming competitions typically have a high bar for entry — one has to know a lot of programming to participate.

“Our contest allowed even amateurs to be able to compete because the base template for the code itself would most often be generated by ChatGPT,” he said, noting that this approach made the most sense as AI-tools evolved. “You can’t really take ChatGPT away in a lot of cases. It makes more sense to embrace the existence of it and the kind of work along with it.”

Maldikar said ChatGPT still doesn’t eliminate the need to understand the underlying question being asked.

“If you are solving a question, you need to understand what the question is,” he said. “If you simply copy and paste, in my experience, that won’t work. ChatGPT can get you started and be a really helpful tool to start the problem — but you need to understand the problem.”

He added that students could even use ChatGPT to learn about the concepts used in a problem.

Blum said all the teams earned some points, most of them solved multiple problems and they were engaged throughout.

“Given that we have a wide range of students participating — most brand new to competitive programming — that says a lot about the scaffolding these tools can provide and, ultimately, how they can broaden participation [in programming competitions],” Blum said.

Shaub expressed gratitude for support from Gretta Kellogg and Soundar Kumara from Penn State’s Center for Applications of Artificial Intelligence and Machine Learning to Industry, as well as from Harrisburg LaunchBox for a financial contribution.