pip install batch_processorfrom threading import Thread
from batch_processor import BatchProcessor
def batch_func(batch):
return [v + v for v in batch]
def create_bulk_request(n, processor):
print(n, processor.process(n))
processor = BatchProcessor(batch_func, worker_num=2, batch_size=32)
threads = []
for i in range(200):
t = Thread(target=create_bulk_request, args=(i, self.processor))
t.start()
threads.append(t)
for t in threads:
t.join()This piece of code generates discrete 200 incomming requests that can be processed in batches whose size is 32. There are 2 workers processing these requests. They take batches and double each integers in the batch, then return results in batches.
- run test cases
python tests/test_runner.py
- generate coverage
coverage run tests/test_runner.py
- generate coverage html
coverage html
build the distribution
python setup.py sdist bdist_wheelupload to registry
python -m twine upload dist/*