I often find with the tasks that there are test-cases which really should have been included in the problem for completeness. This allows code which doesn't correctly solve the stated problem to be submitted.
I am trying to do a good job of reporting problems as "Wrong Solution" when I notice that code has been submitted which doesn't really solve the stated problem, but this process seems very tedious and slow.
The problem is especially bad when a task has been solved by many different people all of whom forgot the same test case. Reporting all of the "Wrong Solutions" can get really boring. It also ins't clear that the administrators are actually removing the solutions which aren't correct.
I think more time needs to be put into making a more complete set of test cases before letting solutions be submitted for the problem. For example, the fact that the Bullet and Wall task doesn't checkio([[2, 0], [2, 4], [0,2], [2, 3]]) == True means that lots of people made overly simplistic solutions.
Created at: 2012/11/16 01:20; Updated at: 2013/09/30 13:46
