Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 379d281

Browse filesBrowse files
asenyaevasenyaev
authored
Build zlib from the source on Linux with the latest version (opencv#715)
* Added zlib build from the source on Linux, handling auditwheel issues * Removed unused variable * Removed bzip2 from dependencies for OpenCV Python packages
1 parent 7bb3e77 commit 379d281
Copy full SHA for 379d281

File tree

Expand file treeCollapse file tree

7 files changed

+75
-59
lines changed
Filter options
Expand file treeCollapse file tree

7 files changed

+75
-59
lines changed

‎.github/workflows/build_wheels_linux.yml

Copy file name to clipboardExpand all lines: .github/workflows/build_wheels_linux.yml
+1-9Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -35,14 +35,13 @@ jobs:
3535
env:
3636
ACTIONS_ALLOW_UNSECURE_COMMANDS: true
3737
REPO_DIR: .
38-
BUILD_COMMIT: ${{ github.ref }}
3938
PROJECT_SPEC: opencv-python
4039
MB_PYTHON_VERSION: ${{ matrix.python-version }}
4140
TRAVIS_PYTHON_VERSION: ${{ matrix.python-version }}
4241
MB_ML_VER: 2014
4342
TRAVIS_BUILD_DIR: ${{ github.workspace }}
4443
CONFIG_PATH: travis_config.sh
45-
DOCKER_IMAGE: quay.io/opencv-ci/opencv-python-manylinux2014-x86-64:20220628
44+
DOCKER_IMAGE: quay.io/opencv-ci/opencv-python-manylinux2014-x86-64:20220824
4645
USE_CCACHE: 0
4746
UNICODE_WIDTH: 32
4847
PLAT: x86_64
@@ -55,9 +54,6 @@ jobs:
5554
working-directory: ${{ github.workspace }}
5655
- name: Setup environment
5756
run: |
58-
if [[ "${{ github.event.pull_request }}" == "true" ]]; then
59-
echo "BUILD_COMMIT=${{ github.base_ref }}" >> $GITHUB_ENV
60-
fi
6157
if [[ "${{ github.event_name }}" == "schedule" || "${{ github.event_name }}" == "workflow_dispatch" ]]; then
6258
echo "ENABLE_ROLLING=1" >> $GITHUB_ENV
6359
fi
@@ -135,7 +131,6 @@ jobs:
135131
env:
136132
ACTIONS_ALLOW_UNSECURE_COMMANDS: true
137133
REPO_DIR: .
138-
BUILD_COMMIT: ${{ github.ref }}
139134
PROJECT_SPEC: opencv-python
140135
PLAT: x86_64
141136
MB_PYTHON_VERSION: ${{ matrix.python-version }}
@@ -156,9 +151,6 @@ jobs:
156151
working-directory: ${{ github.workspace }}
157152
- name: Setup environment
158153
run: |
159-
if [[ "${{ github.event.pull_request }}" == "true" ]]; then
160-
echo "BUILD_COMMIT=${{ github.base_ref }}" >> $GITHUB_ENV
161-
fi
162154
if [[ "${{ github.event_name }}" == "schedule" || "${{ github.event_name }}" == "workflow_dispatch" ]]; then
163155
echo "ENABLE_ROLLING=1" >> $GITHUB_ENV
164156
fi

‎.github/workflows/build_wheels_linux_arm.yml

Copy file name to clipboardExpand all lines: .github/workflows/build_wheels_linux_arm.yml
+1-11Lines changed: 1 addition & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -35,15 +35,14 @@ jobs:
3535
env:
3636
ACTIONS_ALLOW_UNSECURE_COMMANDS: true
3737
REPO_DIR: .
38-
BUILD_COMMIT: ${{ github.ref }}
3938
PROJECT_SPEC: opencv-python
4039
MB_PYTHON_VERSION: ${{ matrix.python-version }}
4140
TRAVIS_PYTHON_VERSION: ${{ matrix.python-version }}
4241
PLAT: aarch64
4342
MB_ML_VER: 2014
4443
TRAVIS_BUILD_DIR: ${{ github.workspace }}
4544
CONFIG_PATH: travis_config.sh
46-
DOCKER_IMAGE: quay.io/opencv-ci/opencv-python-manylinux2014-aarch64:20220628
45+
DOCKER_IMAGE: quay.io/opencv-ci/opencv-python-manylinux2014-aarch64:20220824
4746
USE_CCACHE: 0
4847
UNICODE_WIDTH: 32
4948
SDIST: ${{ matrix.build_sdist || 0 }}
@@ -55,9 +54,6 @@ jobs:
5554
working-directory: ${{ github.workspace }}
5655
- name: Setup environment
5756
run: |
58-
if [[ "${{ github.event.pull_request }}" == "true" ]]; then
59-
echo "BUILD_COMMIT=${{ github.base_ref }}" >> $GITHUB_ENV
60-
fi
6157
if [[ "${{ github.event_name }}" == "schedule" || "${{ github.event_name }}" == "workflow_dispatch" ]]; then
6258
echo "ENABLE_ROLLING=1" >> $GITHUB_ENV
6359
fi
@@ -97,7 +93,6 @@ jobs:
9793
ACTIONS_ALLOW_UNSECURE_COMMANDS: true
9894
MB_PYTHON_VERSION: ${{ matrix.python-version }}
9995
PLAT: aarch64
100-
BUILD_COMMIT: ${{ github.ref }}
10196
NP_TEST_DEP: numpy==1.19.4
10297
NP_TEST_DEP_LATEST: numpy==1.21.4
10398
CONFIG_PATH: travis_config.sh
@@ -107,11 +102,6 @@ jobs:
107102
- name: Cleanup
108103
run: find . -mindepth 1 -delete
109104
working-directory: ${{ github.workspace }}
110-
- name: Setup environment
111-
run: |
112-
if [[ "${{ github.event.pull_request }}" == "true" ]]; then
113-
echo "BUILD_COMMIT=${{ github.base_ref }}" >> $GITHUB_ENV
114-
fi
115105
- name: Checkout
116106
uses: actions/checkout@v3
117107
with:

‎.github/workflows/build_wheels_macos.yml

Copy file name to clipboardExpand all lines: .github/workflows/build_wheels_macos.yml
-4Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,6 @@ jobs:
3535
env:
3636
ACTIONS_ALLOW_UNSECURE_COMMANDS: true
3737
REPO_DIR: .
38-
BUILD_COMMIT: ${{ github.ref }}
3938
PROJECT_SPEC: opencv-python
4039
MB_PYTHON_VERSION: ${{ matrix.python-version }}
4140
TRAVIS_PYTHON_VERSION: ${{ matrix.python-version }}
@@ -57,9 +56,6 @@ jobs:
5756
working-directory: ${{ github.workspace }}
5857
- name: Setup environment
5958
run: |
60-
if [[ "${{ github.event.pull_request }}" == "true" ]]; then
61-
echo "BUILD_COMMIT=${{ github.base_ref }}" >> $GITHUB_ENV
62-
fi
6359
if [[ "${{ github.event_name }}" == "schedule" || "${{ github.event_name }}" == "workflow_dispatch" ]]; then
6460
echo "ENABLE_ROLLING=1" >> $GITHUB_ENV
6561
fi

‎.github/workflows/build_wheels_macos_m1.yml

Copy file name to clipboardExpand all lines: .github/workflows/build_wheels_macos_m1.yml
-10Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,6 @@ jobs:
3030
without_gui: [0, 1]
3131
build_sdist: [0]
3232
env:
33-
BUILD_COMMIT: ${{ github.ref }}
3433
CI_BUILD: 1
3534
SDIST: ${{ matrix.build_sdist || 0 }}
3635
ENABLE_HEADLESS: ${{ matrix.without_gui }}
@@ -41,9 +40,6 @@ jobs:
4140
working-directory: ${{ github.workspace }}
4241
- name: Setup environment
4342
run: |
44-
if [[ "${{ github.event.pull_request }}" == "true" ]]; then
45-
echo "BUILD_COMMIT=${{ github.base_ref }}" >> $GITHUB_ENV
46-
fi
4743
if [[ "${{ github.event_name }}" == "schedule" || "${{ github.event_name }}" == "workflow_dispatch" ]]; then
4844
echo "ENABLE_ROLLING=1" >> $GITHUB_ENV
4945
fi
@@ -84,18 +80,12 @@ jobs:
8480
without_gui: [0, 1]
8581
build_sdist: [0]
8682
env:
87-
BUILD_COMMIT: ${{ github.ref }}
8883
OPENCV_TEST_DATA_PATH: ${{ github.workspace }}/opencv_extra/testdata
8984
PYLINT_TEST_FILE: ${{ github.workspace }}/opencv/samples/python/squares.py
9085
steps:
9186
- name: Cleanup
9287
run: find . -mindepth 1 -delete
9388
working-directory: ${{ github.workspace }}
94-
- name: Setup environment
95-
run: |
96-
if [[ "${{ github.event.pull_request }}" == "true" ]]; then
97-
echo "BUILD_COMMIT=${{ github.base_ref }}" >> $GITHUB_ENV
98-
fi
9989
- name: Checkout
10090
uses: actions/checkout@v3
10191
with:

‎docker/manylinux2014/Dockerfile_aarch64

Copy file name to clipboardExpand all lines: docker/manylinux2014/Dockerfile_aarch64
+20-9Lines changed: 20 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Version: 20220628
1+
# Version: 20220824
22
# Image name: quay.io/opencv-ci/opencv-python-manylinux2014-aarch64
33

44
FROM quay.io/pypa/manylinux2014_aarch64:latest
@@ -11,16 +11,28 @@ ARG NASM_VERSION=2.15.04
1111
ARG OPENSSL_VERSION=1_1_1o
1212
ARG QT_VERSION=5.15.0
1313
ARG YASM_VERSION=1.3.0
14+
ARG ZLIB_VERSION=1.2.12
1415

1516
ENV LD_LIBRARY_PATH /usr/local/lib:$LD_LIBRARY_PATH
1617

1718
# epel-release need for aarch64 to get openblas packages
18-
RUN yum install bzip2-devel curl-devel zlib-devel xcb-util-renderutil-devel xcb-util-devel xcb-util-image-devel xcb-util-keysyms-devel xcb-util-wm-devel mesa-libGL-devel libxkbcommon-devel libxkbcommon-x11-devel libXi-devel lapack-devel epel-release -y && \
19+
RUN yum install bzip2-devel curl-devel xcb-util-renderutil-devel xcb-util-devel xcb-util-image-devel xcb-util-keysyms-devel xcb-util-wm-devel mesa-libGL-devel libxkbcommon-devel libxkbcommon-x11-devel libXi-devel lapack-devel epel-release -y && \
1920
yum install openblas-devel -y && \
2021
cp /usr/include/lapacke/lapacke*.h /usr/include/ && \
2122
curl https://raw.githubusercontent.com/xianyi/OpenBLAS/v0.3.3/cblas.h -o /usr/include/cblas.h && \
2223
# libpng will be built from source
23-
yum remove libpng -y
24+
yum remove libpng zlib-devel bzip2-devel -y
25+
26+
RUN mkdir ~/zlib_sources && \
27+
cd ~/zlib_sources && \
28+
curl -O -L https://www.zlib.net/zlib-${ZLIB_VERSION}.tar.gz && \
29+
tar -xf zlib-${ZLIB_VERSION}.tar.gz && \
30+
cd zlib-${ZLIB_VERSION} && \
31+
./configure --prefix=/usr/local && \
32+
make && \
33+
make install && \
34+
cd .. && \
35+
rm -rf ~/zlib_sources
2436

2537
RUN mkdir ~/libpng_sources && \
2638
cd ~/libpng_sources && \
@@ -71,8 +83,8 @@ RUN mkdir ~/openssl_sources && \
7183

7284
RUN mkdir ~/nasm_sources && \
7385
cd ~/nasm_sources && \
74-
curl -O -L http://www.nasm.us/pub/nasm/releasebuilds/${NASM_VERSION}/nasm-${NASM_VERSION}.tar.bz2 && \
75-
tar -xf nasm-${NASM_VERSION}.tar.bz2 && cd nasm-${NASM_VERSION} && ./autogen.sh && \
86+
curl -O -L http://www.nasm.us/pub/nasm/releasebuilds/${NASM_VERSION}/nasm-${NASM_VERSION}.tar.gz && \
87+
tar -xf nasm-${NASM_VERSION}.tar.gz && cd nasm-${NASM_VERSION} && ./autogen.sh && \
7688
./configure --prefix="/ffmpeg_build" --bindir="$HOME/bin" && \
7789
make -j$(getconf _NPROCESSORS_ONLN) && \
7890
make install && \
@@ -102,17 +114,16 @@ RUN mkdir ~/libvpx_sources && \
102114

103115
RUN mkdir ~/ffmpeg_sources && \
104116
cd ~/ffmpeg_sources && \
105-
curl -O -L https://ffmpeg.org/releases/ffmpeg-${FFMPEG_VERSION}.tar.bz2 && \
106-
tar -xf ffmpeg-${FFMPEG_VERSION}.tar.bz2 && \
117+
curl -O -L https://ffmpeg.org/releases/ffmpeg-${FFMPEG_VERSION}.tar.gz && \
118+
tar -xf ffmpeg-${FFMPEG_VERSION}.tar.gz && \
107119
cd ffmpeg-${FFMPEG_VERSION} && \
108120
PATH=~/bin:$PATH && \
109121
PKG_CONFIG_PATH="/ffmpeg_build/lib/pkgconfig" ./configure --prefix="/ffmpeg_build" --extra-cflags="-I/ffmpeg_build/include" --extra-ldflags="-L/ffmpeg_build/lib" --enable-openssl --enable-libvpx --enable-shared --enable-pic --bindir="$HOME/bin" && \
110122
make -j$(getconf _NPROCESSORS_ONLN) && \
111123
make install && \
112124
echo "/ffmpeg_build/lib/" >> /etc/ld.so.conf && \
113125
ldconfig && \
114-
rm -rf ~/ffmpeg_sources && \
115-
yum remove bzip2-devel -y
126+
rm -rf ~/ffmpeg_sources
116127

117128
RUN curl -O -L https://github.com/ccache/ccache/releases/download/v${CCACHE_VERSION}/ccache-${CCACHE_VERSION}.tar.gz && \
118129
tar -xf ccache-${CCACHE_VERSION}.tar.gz && \

‎docker/manylinux2014/Dockerfile_x86_64

Copy file name to clipboardExpand all lines: docker/manylinux2014/Dockerfile_x86_64
+20-9Lines changed: 20 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Version: 20220628
1+
# Version: 20220824
22
# Image name: quay.io/opencv-ci/opencv-python-manylinux2014-x86-64
33

44
FROM quay.io/pypa/manylinux2014_x86_64:latest
@@ -11,16 +11,28 @@ ARG NASM_VERSION=2.15.04
1111
ARG OPENSSL_VERSION=1_1_1o
1212
ARG QT_VERSION=5.15.0
1313
ARG YASM_VERSION=1.3.0
14+
ARG ZLIB_VERSION=1.2.12
1415

1516
ENV LD_LIBRARY_PATH /usr/local/lib:$LD_LIBRARY_PATH
1617

1718
# epel-release need for aarch64 to get openblas packages
18-
RUN yum install bzip2-devel curl-devel zlib-devel xcb-util-renderutil-devel xcb-util-devel xcb-util-image-devel xcb-util-keysyms-devel xcb-util-wm-devel mesa-libGL-devel libxkbcommon-devel libxkbcommon-x11-devel libXi-devel lapack-devel epel-release -y && \
19+
RUN yum install bzip2-devel curl-devel xcb-util-renderutil-devel xcb-util-devel xcb-util-image-devel xcb-util-keysyms-devel xcb-util-wm-devel mesa-libGL-devel libxkbcommon-devel libxkbcommon-x11-devel libXi-devel lapack-devel epel-release -y && \
1920
yum install openblas-devel -y && \
2021
cp /usr/include/lapacke/lapacke*.h /usr/include/ && \
2122
curl https://raw.githubusercontent.com/xianyi/OpenBLAS/v0.3.3/cblas.h -o /usr/include/cblas.h && \
2223
# libpng will be built from source
23-
yum remove libpng -y
24+
yum remove libpng zlib-devel bzip2-devel -y
25+
26+
RUN mkdir ~/zlib_sources && \
27+
cd ~/zlib_sources && \
28+
curl -O -L https://www.zlib.net/zlib-${ZLIB_VERSION}.tar.gz && \
29+
tar -xf zlib-${ZLIB_VERSION}.tar.gz && \
30+
cd zlib-${ZLIB_VERSION} && \
31+
./configure --prefix=/usr/local && \
32+
make && \
33+
make install && \
34+
cd .. && \
35+
rm -rf ~/zlib_sources
2436

2537
RUN mkdir ~/libpng_sources && \
2638
cd ~/libpng_sources && \
@@ -71,8 +83,8 @@ RUN mkdir ~/openssl_sources && \
7183

7284
RUN mkdir ~/nasm_sources && \
7385
cd ~/nasm_sources && \
74-
curl -O -L http://www.nasm.us/pub/nasm/releasebuilds/${NASM_VERSION}/nasm-${NASM_VERSION}.tar.bz2 && \
75-
tar -xf nasm-${NASM_VERSION}.tar.bz2 && cd nasm-${NASM_VERSION} && ./autogen.sh && \
86+
curl -O -L http://www.nasm.us/pub/nasm/releasebuilds/${NASM_VERSION}/nasm-${NASM_VERSION}.tar.gz && \
87+
tar -xf nasm-${NASM_VERSION}.tar.gz && cd nasm-${NASM_VERSION} && ./autogen.sh && \
7688
./configure --prefix="/ffmpeg_build" --bindir="$HOME/bin" && \
7789
make -j$(getconf _NPROCESSORS_ONLN) && \
7890
make install && \
@@ -102,17 +114,16 @@ RUN mkdir ~/libvpx_sources && \
102114

103115
RUN mkdir ~/ffmpeg_sources && \
104116
cd ~/ffmpeg_sources && \
105-
curl -O -L https://ffmpeg.org/releases/ffmpeg-${FFMPEG_VERSION}.tar.bz2 && \
106-
tar -xf ffmpeg-${FFMPEG_VERSION}.tar.bz2 && \
117+
curl -O -L https://ffmpeg.org/releases/ffmpeg-${FFMPEG_VERSION}.tar.gz && \
118+
tar -xf ffmpeg-${FFMPEG_VERSION}.tar.gz && \
107119
cd ffmpeg-${FFMPEG_VERSION} && \
108120
PATH=~/bin:$PATH && \
109121
PKG_CONFIG_PATH="/ffmpeg_build/lib/pkgconfig" ./configure --prefix="/ffmpeg_build" --extra-cflags="-I/ffmpeg_build/include" --extra-ldflags="-L/ffmpeg_build/lib" --enable-openssl --enable-libvpx --enable-shared --enable-pic --bindir="$HOME/bin" && \
110122
make -j$(getconf _NPROCESSORS_ONLN) && \
111123
make install && \
112124
echo "/ffmpeg_build/lib/" >> /etc/ld.so.conf && \
113125
ldconfig && \
114-
rm -rf ~/ffmpeg_sources && \
115-
yum remove bzip2-devel -y
126+
rm -rf ~/ffmpeg_sources
116127

117128
RUN curl -O -L https://github.com/ccache/ccache/releases/download/v${CCACHE_VERSION}/ccache-${CCACHE_VERSION}.tar.gz && \
118129
tar -xf ccache-${CCACHE_VERSION}.tar.gz && \

‎patch_auditwheel_whitelist.py

Copy file name to clipboardExpand all lines: patch_auditwheel_whitelist.py
+33-7Lines changed: 33 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,39 @@
33

44
from auditwheel import policy
55

6-
policies = None
76

8-
with open(join(dirname(abspath(policy.__file__)), "manylinux-policy.json")) as f:
9-
policies = json.load(f)
7+
def add_zlib_versions():
8+
with open(join(dirname(abspath(policy.__file__)), "manylinux-policy.json"), 'r') as manylinux_policy:
9+
manylinux_policy_lines = manylinux_policy.readlines()
1010

11-
for p in policies:
12-
p["lib_whitelist"].append("libxcb.so.1")
11+
manylinux_policy_data = ""
12+
for line in manylinux_policy_lines:
13+
if "ZLIB" in line:
14+
if len(line) > 22:
15+
updated_line = line[:-2] + ', "1.2.9", "1.2.12"]'
16+
else:
17+
updated_line = line[:-2] + '"1.2.9", "1.2.12"]'
18+
manylinux_policy_replacement = line.replace(line, updated_line)
19+
else:
20+
manylinux_policy_replacement = line
21+
manylinux_policy_data = manylinux_policy_data + manylinux_policy_replacement
1322

14-
with open(join(dirname(abspath(policy.__file__)), "manylinux-policy.json"), "w") as f:
15-
f.write(json.dumps(policies))
23+
with open(join(dirname(abspath(policy.__file__)), "manylinux-policy.json"), 'w') as manylinux_final_policy:
24+
manylinux_final_policy.write(manylinux_policy_data)
25+
26+
def add_whitelisted_libs():
27+
policies = None
28+
29+
with open(join(dirname(abspath(policy.__file__)), "manylinux-policy.json")) as f:
30+
policies = json.load(f)
31+
32+
for p in policies:
33+
p["lib_whitelist"].append("libxcb.so.1")
34+
35+
with open(join(dirname(abspath(policy.__file__)), "manylinux-policy.json"), "w") as f:
36+
f.write(json.dumps(policies))
37+
38+
39+
if __name__ == '__main__':
40+
add_zlib_versions()
41+
add_whitelisted_libs()

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.