Python wcon pip possible performance issues

[cheelee]

It took my machine 3 and a half minutes to load a 48Mb (after unzip) file - https://zenodo.org/record/1034927/files/unc-80%20(e1069)%20nRHO-1%20QT309%20on%20food%20L_2011_11_11__13_12___3___4.wcon.zip

It takes only a few seconds on the partial javascript implementation at http://movement.openworm.org/wconviewer to load the same file.

I think we need to figure out if our common use-cases demand such a heavy-weight implementation, and if there is a way to allow the common case to run a lot quicker.

[Ichoran]

Isn't this mostly due to schema checking? IIRC that was most of the overhead. (The Scala was still way faster--maybe half a second for 48MB if that, but they weren't absurdly far apart any longer.)

[MichaelCurrie]

Yes it's because the library implementing the schema checking that we are relying on is painfully slow for some reason.

I even raised this as an issue on their repository:

python-jsonschema/jsonschema#277

So this issue was sort of already closed as a #wontfix, since the workaround is just to use the option not to check the schema if you want speed.

#55

[cheelee]

Ah! Thanks for the info! I'll see what I can do about it, and perhaps offer an option to enable/disable schema-checking depending on what we think ought to be the default for WCON file loads.