add ThreadedEvaluator; fix docstring of ParallelEvaluator#95
add ThreadedEvaluator; fix docstring of ParallelEvaluator#95bennr01 wants to merge 10 commits intoCodeReclaimers:masterCodeReclaimers/neat-python:masterfrom
Conversation
I have added a 'neat.threaded.ThreadedEvaluator' for evaluating genomes in threads. This is useful when using a python implementation without GIL. The ThreadedEvaluator is based on the ParallelEvaluator.
neat.threaded.ThreadEvolver will now start its worker automatically
the documentation of 'neat.parallel.ParallelEvolver' said that 'eval_function' should take one argument (the genome), but it is called with two arguments(the genome and the config).
|
To get the test coverage back up, take a look at the tests subdirectory and do a search for "parallel", then add corresponding functions. (I'd also noticed the documentation glitch; it's noted in my docs_update branch - see #92 ) -Allen |
I have added a test for neat.threaded.ThreadedEvaluator based on the test for neat.parallel.ParallelEvaluator.
1 similar comment
|
@drallensmith thanks, i have added a test (and actually found a bug, i always forget that there is no xrange in python3). |
1 similar comment
i removed the test for checki if 'neat.threaded.ThreadedEvaluator.__del__' stops the threads. This is because __del__ is not always called and thus may result in false test results.
I have changed travis.yml to use `pypy3.5-5.8.0` instead of `pypy3`. Travis uses an outdated version of `pypy3`. `pypy3.5-5.8.0` contains some fixes for multithreaded scripts, which *may* fix the bug in the travic-ci build for `neat.threaded.ThreadedEvaluator`.
travis-ci: explicitly set pypy3 version to `pypy3.5-5.8.0`
|
Putting in travis_wait also works, BTW - see my current pull request. (Sorry, didn't realize that you were having that problem...) Quite welcome re testing - I just had that happen over on LARG/HFO with me; since it's mostly C++, they were using "did it build" as their test, along with running a few examples, so I didn't find a bug until after I actually added some testing scripts. |
travis-ci uses an outdated `pypy3`, which contains a bug when using multiple threads. This causes some test to fail on `pypy3`. Replacing `pypy3` with `pypy3.5-5.8.0` in `.travis.yml` fixes the tests. I also added another test for general functionality of `neat.threaded.ThreadedEvaluator`.
|
Oh - it actually does only take 1 argument, it's just that the argument in question is a tuple of genome, config. (The Pool function won't do more than one argument.) I have to say I wish python tuples weren't just defined by commas - they're easy to miss. |
|
@drallensmith thanks for the continued help. |
|
Quite welcome. Of course, fixing the tested code requires knowing about the bugs first... |
|
I am closing this PR and will combine it with my next PR. |
|
Hey just wanted to say thanks for all the hard work on this--if I don't get some free time soon I'll just commit your PRs so they don't languish. |
I have added the
neat.threaded.ThreadedEvaluatorand fixed a mistake in the docstring ofneat.parallel.ParallelEvaluator.__init__.the
ThreadedEvaluator-class is like theParallelEvaluator-class, but uses threads instead of processes.This is useful when using python implementations without an GIL like jython.
I also created an example of the
ThreadedEvaluatorbased onexamples/xor/evolve-feedforward-parallel.py.The docstring of
neat.parallel.ParallelEvaluator.__init__stated thateval_functionshould take one argument (the genome object)[...]. However,eval_functionis called with two arguments (the genome object and the config). This is now corrected.Edit: This PR now also modifies
.travis.ymlto enforce the usagepypy3.5-5.8.0instead ofpypy3. Travis uses an old pypy3, which causes some bugs in multithreaded script.