The Latest Cheating Scandal

[Jonathan Edwards]

So often in life we are easily deceived by false statistics - often because they are incorrect. If the odds PRIOR to an event are 1 in 10,000 that a specific player scores as this player did then if you are looking with hindsight you are really asking the question what are the odds of ANY 2200+ player putting in such a performance. If there are 1000 such players playing 10 events a year then the odds are 1 i.e. a certainty.

Just to be annoying... With the numbers you've given, the odds of at least one 2200 player would be a little over 63%. But I get the point you're making =).

[Mike Scott]

No problem - I made up the numbers! Perhaps someone can work out the actual probability? That said any probability simply based on the grading system are probably unreliable for the reasons outlined in the last paragraph of the article given below (taken from http://innocent.org.uk/cases/sallyclark/#watkinsbmj)

Anyway my point is best made by people remembering Sally Clark, the mother wrongly convicted and jailed for killing her two children largely due to an appalling misunderstanding of probabilities:-

On 9 November at Chester Crown Court Sally Clark, a Cheshire solicitor, was convicted, by 10-2 majority, of smothering her two infant children. With conflicting forensic evidence, the Crown's case was bolstered by an eminent paediatrician testifying that the chances of two cot deaths happening in this family was vanishingly small - 1 in 73 million. This seriously misunderstands probability theory. It is speculation whether Sally Clark would have been acquitted without this evidence. But with this mathematical error prominent the conviction is unsafe.

Imagine an archery target with two arrows sticking in the very centre of it. This provides greater evidence of the skill of the archer if the target was in place before the arrows were fired than if it was drawn around them afterwards. Probability theory requires calculation of the probability not only of the event in question but also of all events that are as extreme or more extreme. When the target is drawn first you calculate the chance of both arrows hitting the centre of the target. But when the target is drawn round the arrows afterwards you calculate the chance of both arrows hitting the same point, whatever that point. With two independent arrows one probability is the square of the other.

Suspicion was drawn to Sally Clark by the occurrence of two deaths so the probabilities should not have been squared. The odds of 1 in 73 million shrink to 1 in 8500. But this figure is itself meaningless. There is in fact a wall full of arrows with the target drawn around the two that are close together and the others ignored. Mathematical formulas for this situation often surprise people. For example, with only 23 people in a room the odds are better than 50% that two of them have the same birthday.

From whole population data Reese calculates the square of the population risk of cot death as 1 in 2.75 million. There are 378 000 second or subsequent births each year in England. So if cot deaths are random events two cot deaths will occur in the same family somewhere in England once every seven years. But cot deaths are not random events. There have been several studies of recurrence. At least one study did show no increase in recurrence rates. But several others showed recurrence rates about five times the general rate, implying recurrence somewhere in England about once every year and a half. Two studies showed even higher rates.

The fact that studies of recurrence have been done means this event is not vanishingly rare. In a case series of recurrent infant death Emery classified two cases as recurrent cot death out of 12 cases occurring in Sheffield in 20 years. Wolkind et al found five cases in their unsystematic English case series of 57 recurrent infant deaths. Both these studies distinguished cot death from accident, illness, murder, and neglect.

The prosecution used the figure of 1 in 73 million rather than 1 in 2.75 million because of the family's affluence. Yet taking data from an epidemiological group and applying it stereotypically to all members is an example of the ecological fallacy. Social class is a complex reality of interassociated circumstances - education, work, income, lifestyle, culture, contacts, residence, opportunities, social class of origin, etc - statistically summarised for use in population studies by selecting the one variable which performs best as an indicator. This does not mean that individuals have the attributes of the statistical group.

[Mike Scott]

the odds of at least one 2200 player would be a little over 63%

Done a little research (consulted a different Jonny) and agree with the 63%. Something about accumulative binomial i think.

[D-Oswald]

There just seems to be so many factors to point to the fact that he was aided in some way. The game that really convinced me that he was aided was his round 3 destruction of a GM:

Still a newbie so not exactly sure how to get one of those game positions (but will try!).

1. d4 Nf6 2. c4 e6 3. g3 Bb4+ 4. Bd2 c5 5. Bxb4 cxb4 6. Bg2 O-O 7. Nf3 d6 8.
O-O Qe7 9. a3 bxa3 10. Rxa3 b6 11. Nc3 Bb7 12. d5 e5 13. Nh4 g6 14. Qd2 Nh5 15.
Qh6 f5 16. e4 Ng7 17. exf5 gxf5 18. Nb5 Rf6 19. Qg5 Qf7 20. Rxa7 Rxa7 21. Nxa7
f4 22. Ra1 Na6 23. Nc6 Bc8 24. Nf5 Bxf5 25. Rxa6 h6 26. Qh4 Bd3 27. Rxb6 e4 28.
Rb7 Qxb7 29. Qxf6 e3 30. fxe3 fxe3 31. Ne7+ Kh7 32. Qf8 h5 33. Qg8+ Kh6 34.
Qh8+ Bh7 35. Be4 1-0

[pgn]1. d4 Nf6 2. c4 e6 3. g3 Bb4+ 4. Bd2 c5 5. Bxb4 cxb4 6. Bg2 O-O 7. Nf3 d6 8.
O-O Qe7 9. a3 bxa3 10. Rxa3 b6 11. Nc3 Bb7 12. d5 e5 13. Nh4 g6 14. Qd2 Nh5 15.
Qh6 f5 16. e4 Ng7 17. exf5 gxf5 18. Nb5 Rf6 19. Qg5 Qf7 20. Rxa7 Rxa7 21. Nxa7
f4 22. Ra1 Na6 23. Nc6 Bc8 24. Nf5 Bxf5 25. Rxa6 h6 26. Qh4 Bd3 27. Rxb6 e4 28.
Rb7 Qxb7 29. Qxf6 e3 30. fxe3 fxe3 31. Ne7+ Kh7 32. Qf8 h5 33. Qg8+ Kh6 34.
Qh8+ Bh7 35. Be4 1-0[/pgn]

The position looks pretty normal after the opening on move 11. And then d5!? which seems strange to me. It appears totally unnecessary and just loosens up the c5 and e5 squares. After playing 9. a3, and going for a plan of attacking down the a-file, I would have expected 12. Qa1!? or something. But he now starts playing in the centre with 12. d5.
Then 13. Nh4!? which seems totally articificial. So from 9. a3 and playing on the queenside, he goes 12. d5 attacking in the centre and then 13. Nh4 attacking the kingside. In my opinion there just seems to be no logical plan or play there. Then comes the manouevre Qd2-h6... but what does that achieve? what is the plan there? white isn't going to be able to mate with a knight on h4 and a queen on h6? Meanwhile, it appears to me that Black has benefited with g6, Nh5 and f5 gaining space and a potential attack on the kingside. Of course, White's play is not bad and I don't understand it cause I'm not a super computer and can calculate moves to 3300. But Whites seems to be playing without a plan or any logic. Then after kingside operations, White returns his attention to the centre with 16. e4, before then attacking again on the queenside with 18. Nb5!?. How can someone plan this seemingly artificial play all over the board. Who attacks the queenside, centre, kingside, centre then back to queenside with strange moves and without seeming to achieve anything. And this plan seems to work against a strong grandmaster... Black goes slightly wrong and White slaughters him perfectly. I just completely do not understand the logic behind Whites plans. Plans Lilov thought were based purely on calculation (i.e a computer).

Of course this is by no means proof, just my opinion. Not a conclusion I wanted to come to, but one I have.

So regarding andy's comment of how much proof is required? Should it be "beyond reasonable doubt" or "in the balance of probability"? With evidence being difficult to gather at the moment, it certainly is a tricky one.

[andyburnett]

Having watched Lilov's video in its entirety (1 hour 10 minutes) I have almost no doubt that Ivanov was cheating. The game David gives above is a perfect example of engine chess, and his engine match-up is so ridiculously consistent in almost all of his other games that I can't think of any other explanation.

Rather than working out the probability of a 2200 player achieving a 2600+ result, which we know can happen though it's very unusual, it would be revealing to know the probability that a 2200 player could mirror Houdini 2's (rated about 3100) first choice over 95% of the time? I'd guess we're in the realms of lottery numbers (10 million to 1).

When you look at his results and games prior to his last 2 tournaments, it's beyond reasonable doubt in my opinion that this is not the same player.

[Andrew McHarg]

it would be revealing to know the probability that a 2200 player could mirror Houdini 2's (rated about 3100) first choice over 95% of the time? I'd guess we're in the realms of lottery numbers (10 million to 1).

Much, much more unlikely than that. :U

[Andy Howie]

Unless the evidence is actually found then all mathematical probability theory will be just that. As long as there is a possibility that it can happen then concrete proof is required.

One thing to be very careful when checking v a computer is to make sure that there are no "forcing" moves. Often you find that one move will jump out v the others so it would be wrong to suggest the move matches. You also need to base on more than one game as top players will match computers more than you think!

I used to be involved with detecting cheating in online games and know most of the detection methods. Interestingly none of the people I also know to be involved with this have come out and said anything which makes me wonder.

I am in the yet to be convinced camp. If I get some time I might take a look at the games and apply some of the methodology we used to use.

[Andrew McHarg]

Unless the evidence is actually found then all mathematical probability theory will be just that. As long as there is a possibility that it can happen then concrete proof is required.

DNA evidence is used in court to convict people of murder, despite there being a chance (albeit ridiculously unlikely) of it being wrong. It's not used in isolation, but it's one of the many crucial pieces of evidence these days, and will more often than not be seen by the jury as absolute proof. The odds of someone matching Houdini 95% of the time across a whole tournament make it similarly unlikely that it's happening by chance. Coupled with the alleged dip in performance when the live feed stopped, it is strongly indicative of foul play. Keeping in mind as well that the 5% where there wasn't a match was probably opening theory.

So I'm inclined to side with Andy (Burnett); if the evidence above is true then he was certainly cheating. =)

[Andy Howie]

I did a quick and "dirty" analysis of the game above. The missing data is how long the player took between moves. This really is essential as there is quite a swing between some of the moves as you move through the ply. I took these to 20 ply. First column is the move, second is where the choice was in the computer eval. Third is difference from the second move if it was the first, or the difference from the first move if not the first move.

move Diff from next Diff from 1
11 1 0.12
12 1 0.11
13 2 0.01
14 2 0.01
15 1 0.4
16 2 0.37
17 1 0.21
18 2 0.01
19 1 1.34
20 1 0.31
21 1 0.86
22 1 0.65
23 1 0.51
24 1 0.61
25 1 2.59
26 1 1.36
27 1 0.91
28 1 0.47
29 1 10.05
30 1 0.79
31 1 8.23
32 1 11.6


Here is a counter proposal for you based on this game alone. Is it not within the realms of possibility that his opponent did not play well. There are 5 moves here that appear to be computational blunders.

I am not for one second saying he did not cheat, however most people's idea of checking the computer to see if the computer selects the move is to run the game and see if it appears as the first move at any point. Anyone with the computer experience he has would know that you have to throw in some lower level moves where the count is a lot lower (quite a few of the earlier moves had 6 or 7 close choices) to throw off suspicion if you are actually wanting to cheat. Catching that type of behavior is hard but it is possible

[Alan Jelfs]

it would be revealing to know the probability that a 2200 player could mirror Houdini 2's (rated about 3100) first choice over 95% of the time? I'd guess we're in the realms of lottery numbers (10 million to 1).

Much, much more unlikely than that. :U

But wouldn't you have to measure that against the probability of a 2600 player mirroring Houdini's moves?