IronArray for Python
This package has iron-array C library (develop branch) integrated as a submodule. In order to clone the repo with all the submodules, do:
git clone --recurse-submodules https://github.com/inaos/iron-array-python
git submodule update --init --recursiveIn general, you can update your local repo with:
git pull
git submodule update --recursiveAlso, for developers it is good to konw that having the iron-array C library integrated as a submodule allows to quickly update to its latest commits in develop branch easily (even if the iron-array-python repo has not been updated to use the latest version yet):
git submodule update --remote --recursiveIn case you have Intel IPP libraries installed (for a turbo-enabled LZ4 codec within C-Blosc2), make sure that you run:
source ~/intel/bin/compilervars.sh intel64so as to allow the iarray library to find the IPP libraries.
We rely on scikit-build, numpy and others to build and test the package, so please be sure to install the requisites in your environment:
python -m pip install -r requirements-build.txtIn addition, we need LLVM development and SVML packages that can be easily installed from conda:
conda install -c intel mkl-static
conda install -c numba llvmdev
conda install -c intel icc_rt # SVMLWhile you are at this, it is also wise to install numba and vmtouch util for running tutorials and benchmarks:
conda install -c numba numba
sudo apt install vmtouchWe can proceed now with the compilation of the actual Python wrapper for iarray:
rm -rf _skbuild iarray/iarray-c-develop/build/* iarray/*.so* # *.pyd* if on windows (total cleanup and optional)
python setup.py build_ext -j 4 --build-type=RelWithDebInfo # choose Debug if you likeThis will compile the iron-array C library and the Python extension in one go and will put both libraries in the iarray/ directory, so the is wrapper is ready to be used right away. As the whole process is driven with cmake, making small changes in either the C library or the Python extension will just trigger the re-compilation of the affected modules.
Also note the -j 4 flag; this is a way to specify the number of processes in parallel that you want to use during the build process.
Thanks to the nice integration of scikit-build with cmake, you can even pass cmake configure options directly from commandline. For example:
python setup.py build_ext -j 4 --build-type=RelWithDebInfo -- -DDISABLE_LLVM_CONFIG=False
Sometimes the llvm_config command does not work as intended (as in Clear Linux), so you will need to disable it.
python setup.py build_ext -j 4 --build-type=RelWithDebInfo -- -DDISABLE_LLVM_CONFIG=True
pytest (base)
=================================================================== test session starts ====================================================================
platform darwin -- Python 3.7.4, pytest-4.3.0, py-1.8.1, pluggy-0.13.1
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/Users/faltet/inaos/iron-array-python/.hypothesis/examples')
rootdir: /Users/faltet/inaos/iron-array-python, inifile:
plugins: hypothesis-4.18.3
collected 164 items
iarray/tests/test_constructor.py ............................ [ 17%]
iarray/tests/test_copy.py ....... [ 21%]
iarray/tests/test_expression.py ......................................................... [ 56%]
iarray/tests/test_iterator.py ........ [ 60%]
iarray/tests/test_load_save.py .... [ 63%]
iarray/tests/test_matmul.py ................ [ 73%]
iarray/tests/test_random.py ........................................ [ 97%]
iarray/tests/test_slice.py .... [100%]
========================================================== 164 passed, 0 warnings in 7.13 seconds ==========================================================```One can build wheels for the current platform with
python setup.py bdist_wheelThe wheels will appear in dist/ directory.
Note: see https://github.com/pypa/auditwheel package on how to audit and amend wheels for compatibility with a wide variety of Linux distributions.
You may want to install this package in the system. For doing this, use:
python setup.py install