Two university professors explore the similarities between cheating during online classes and cheating in online chess communities during episode 004 of The Score Podcast on academic integrity
On episode four of The Score, host Kathryn Baron (@TchersPet) spoke with Dr. Alexander Matros, a professor in the Darla Moore School of Business at the University of South Carolina, and Eren Bilen (@Ernbilen), Assistant Professor of Data Analytics at Dickinson College. Both Matros and Bilen are chess players, about their September 2020 study on online cheating amid COVID-19 explains how the International Chess Federation and the Internet Chess Club deal with cheating. Their suggestions offer insight about how universities might address cheating.
The full episode is available to listen on Apple, Spotify or at The Score. Excerpts from the episode follow.
Note: Removal of filler words and minor edits have been made for clarity.
Kathryn Baron (01:38): Why did you select online chess as a barometer for online cheating in colleges and universities?
Dr. Alexander Matros (01:44): If you look back, chess had this problem for over 20 years and chess was played online on different platforms, including this Internet Chess Club. And it was interesting to see how they deal with the problem of cheating. And we tried to use this experience for what we can see now in education.
Kathryn Baron (02:04): Now, what are the differences though? Because clearly a private chess club can do different things than a college or university. If you look at a private chess club, they have this problem that if somebody’s trying to cheat, then if this information is public knowledge, then nobody wants to join the club because it’s very difficult to compete with this guy who might win everything. And cheating might take different forms, for example, somebody might help you, then you can use computer support. There are many different ways.
Dr. Alexander Matros (02:39): What we saw last year, for example, during the pandemic, this was a very similar situation because if you think about cheating on exams, for example, at colleges, university schools, you can also get outside help from somebody in your room. You can also try to use your computer and search for answers. From this point of view, these are very similar problems. And the difference is only that in chess, we saw that more than 20 years ago we can try to look at their previous experience. And in academia we just saw that last year.
Kathryn Baron (03:13): Why has it been going on in chess for so long? That just kind of surprised me.
Dr. Alexander Matros (03:19): Just again, this is the nature of the beast. In humans, they would like to win. They would like to get some money and in chess, if you are not the top player, you would like to prove that you can beat everybody else. And if you see that you have some problems, one possibility is to use some outside help. Plus a lot of money is at stake. For example, if you qualify for this event, then even if you’re eliminated immediately in the first round, you can win like $5,000. And now we are talking about like more than 20 years ago. It was a lot of cheating. People tried to help, people tried to use computers, and so on. This was just one kind of extreme. But if you look at the research side, you can see that in different competitions, people try to show that they’re the best. Sometimes money is involved, but even if you remove money, [and use] pride, you can try to show that I’m better than my opponent. And people did some experimental studies when they found that in a tournament without prices, people would still be ready to put some effort in order to win the competition.
Kathryn Baron (04:29): So it’s not just that the stakes are high,or there’s money involved or winning a competition, but there’s also something about human nature that’s involved.
Dr. Alexander Matros (04:35): Yes. So even if the price is equal to zero, then people are ready to put in effort when you have to spend resources and then at the end you win nothing, but you win your pride. Okay. So I managed to beat these guys. Yeah.
Kathryn Baron (04:51): In academia, you specifically looked at advanced placement exams, which can earn high school students college credit. And what did you find there? And what was different? Was it online for the first time? Were there hybrids? What changed during COVID?
Eren Bilen (05:08): Yeah, the 2020 AP exams were the first time that these AP exams were given online basically because of COVID. And what happened was if you look at Google searches, and this is public information, you can just access this information, easily. What you see is the 2020 AP exam for the math subject was given on May 12. This was in the afternoon Eastern time. We had 2:00 PM on May 12. If you look at some of the keywords related to math concepts, such as derivative, integral, critical points, inflection point, things like that, you’ll see a spike, exactly at 2:00 PM, and then following 3:00 PM, and so on, the spike basically disappears.
The next day, on May 13, it was the English literature subject. If you do a similar study, you check, this time instead of checking math-related keywords, you check literature-related keywords. You can [search] imagery, literary techniques, diction, things like that. You get the spike, exactly at 2:00 PM on May 13. This is again the time of the test.
And then last, you can even check physics. For example, this was the next day on May 14, but this time was not 2:00 PM, it was 4:00 PM in the afternoon. And you get this spike on physics related keywords at exactly 4:00 PM on May 14. It looks like students basically do some Google searching in order to find the answers, was this helpful [to the student]? Yes. No. We’re not sure, but at least students tried.
Kathryn Baron (06:57): At least they tried to cheat. Was this an unproctored online exam?
Eren Bilen (07:06): That is correct. It was unproctored.
Kathryn Baron (09:12): And you had posed several questions in your report including whether colleges or universities can expect a surge in cheating to continue. And you write that unlike the face to face examination, “Cheating should be expected in online testing.” And you add that, “Cheating is a part of the student equilibrium strategy in the online examination.” So what does this say about us? It just seems to be a sad commentary on who we are in our ethics.
Dr. Alexander Matros (09:42): In our paper, we looked at this problem from at least two directions. First was the theoretical approach and second, we looked at some data from a real life exam and had a simple model. In this simple model, we just assumed that a student can either cheat or not to cheat. So if nobody cheats, then professors would never monitor. And then it would be so simple to cheat. And then we also looked at data. Eren, maybe you can just talk a little bit about the data, what we found.
Eren Bilen (10:21): In the data, we were quote-on-quote, “lucky” in the sense that we had one special tool that enabled us to basically pinpoint what’s going on. The issue was we looked at the time the students took to answer their questions. We gave them basically a test with 20 questions. And these questions were not multiple choice. The students had to enter numbers using their keyboards. And what we saw was that some of the students had very strange timings. For example, on a question that you expect a student to take on average, let’s say five minutes, the student gave an answer in seven seconds. You can say, “Okay, this is one occasion. The student just input a random number or something.” That was not the case. [What they gave] was the correct answer. For example, the correct answer was, let’s say, 347. A student was able to pick that number 347 in less than 10 seconds. And this kept going and going. Next question. Similar. Third question, again, similar. It kept on going for 20 questions. The overall time the student took to complete the exam was about 10 minutes.
Kathryn Baron (12:30): But Eren, in seven seconds, how did they cheat, could they actually look something up online that quickly?
Eren Bilen (12:36): You cannot do this in seven seconds. What we believe that students had was the answers from other students who volunteered to take the test before they did, and they gave them the correct answers. And then you basically had a list in front of you with question names and then the correct answers. They basically looked at this test, the answer sheet, and it probably took them on average, 10 seconds to be able to figure out that was the question that they were seeing on the screen. And basically, they inputted the correct number using their keyboards. It looks like this on average takes 10 seconds.
Kathryn Baron (15:51): You earlier had mentioned fairness. And it does seem that this issue raises some huge ethical issues around fairness, because a student who works very hard to get good grades could very likely do worse in a class because that student didn’t cheat. And even though teachers and professors know from say homework assignments and classroom participation, which students are studying, what can they do when the test results don’t reflect that because of cheating?
Dr. Alexander Matros (16:21): Yeah. I think in a sense, you ask very, very important questions. In a sense, during this pandemic during the whole year, we had some expectations, we had some, you can call this social norm, so what we expect. Let’s say people would come to a class and they would take a test and then you can rank them based on these results. And everything is from this point of view, more or less fair. Now, if you take a test at home, especially if it’s not proctored, and nobody knows who took this test, then the situation now is such that we have another social norm. When you have these expectations or if you have these beliefs that everybody else is cheating, this immediately puts you in a situation where you might be the best student, but you feel that you have no chances to compete with this, as a student, unless you cheat as well.
When you have these expectations, these are self-fulfilling expectations. And now if everybody cheats, everybody expects that. And then they play according to these morals.
Dr. Alexander Matros (20:06): If you put a little bit of effort trying to check them, then maybe they would just abstain from this kind of behavior. And then even simple monitoring can remove a lot, a lot of cheating. It would definitely not remove all cheating, but it would remove simple ones. So, for students like you describe, who would actually prepare in their rooms, you cannot eliminate that, but they put in so much effort. If they would study instead, they would do so much better.
Dr. Alexander Matros (22:38): But online, you have some clues, it’s never direct evidence. It’s only like indirect evidence. You can say, “okay, so the student took a test and finished this test in 5 minutes with 20 questions. It was multiple choice. And their answer is perfect.” But then is it possible? Yes, it’s possible. Because again, you can also win a lottery, you just put the number and then you just like won. So, the student had a good day, and answered everything correctly. Then it’s possible. You cannot say this was impossible. Students guess correctly, so perfect.
Kathryn Baron (24:39): But do your colleagues feel that there is a lot of cheating going on in their classes or do they feel that their students, I’m just wondering is there a consensus that, “Yeah it’s going on,” or are they sort of in the dark about it?
Dr. Alexander Matros (24:54): No, I think this is clearly a consensus that there was cheating and what people will do. They would try to find some ways on how to deal with that.
Dr. Alexander Matros (26:14): In my first 10 years, I had zero cases. And during the pandemic yeah, I did report several cases.
Kathryn Baron (31:25): So let’s wrap up with a question that’s in some ways a summary, but maybe some very more specific recommendations that you have for teachers, professors, and colleges, maybe each of you give me two, like here are the top two things we need to do.
Eren Bilen (32:33): Yeah. We have to move from a bad equilibrium to a better one, absolutely. I absolutely agree. In order to do that, we need to use some sort of proctoring. It could be in-person proctoring, it could be live proctoring, but with the use of proctoring, we can basically move from those bad equilibria to the better ones. Because in a bad equilibrium basically, you give an option to a student to cheat, but if you’re using proctoring, then hopefully 99% of the time, a student won’t be able to cheat. That’s the key takeaway that I want to point out.
Listen to the entire episode 004 of The Score with guests, Dr. Alexander Matros and Eren Bilen here.
