Closed
Description
lazy_loader loaded modules are not thread-safe: when concurrently accessed from several threads all but the first thread get partially loaded module missing most of functionality.
For example librosa uses lazy_loader to load resampy and gets the following error:
AttributeError: module 'resampy' has no attribute 'resample'
To Reproduce
Run the following snippet:
import librosa
import threading
import numpy as np
def resample():
librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best")
for _ in range(5):
threading.Thread(target=resample).start()
Expected behavior
No errors reported.
Actual output
Traceback (most recent call last):
Traceback (most recent call last):
File "C:\Build\envs\dec5\lib\threading.py", line 870, in run
Traceback (most recent call last):
File "C:\Build\envs\dec5\lib\threading.py", line 870, in run
File "C:\Build\envs\dec5\lib\threading.py", line 870, in run
Traceback (most recent call last):
File "C:\Build\envs\dec5\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample
self._target(*self._args, **self._kwargs)self._target(*self._args, **self._kwargs)
File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample
File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample
self._target(*self._args, **self._kwargs)
File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample
librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best")
File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample
librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best")
File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample
librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best")
File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample
librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best")
File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample
y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)
AttributeError: module 'resampy' has no attribute 'resample'
y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)
y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)
AttributeError: module 'resampy' has no attribute 'resample'AttributeError
: module 'resampy' has no attribute 'resample'
y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)
AttributeError: module 'resampy' has no attribute 'resample'
python-BaseException
python-BaseException
python-BaseException
python-BaseException
Exception in thread Exception in thread Thread-8:
Exception in thread Traceback (most recent call last):
Thread-5:
Traceback (most recent call last):
File "C:\Build\envs\dec5\lib\threading.py", line 932, in _bootstrap_inner
File "C:\Build\envs\dec5\lib\threading.py", line 932, in _bootstrap_inner
Thread-9:
Traceback (most recent call last):
File "C:\Build\envs\dec5\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "C:\Build\envs\dec5\lib\threading.py", line 870, in run
self.run()
File "C:\Build\envs\dec5\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
self.run()
File "C:\Build\envs\dec5\lib\threading.py", line 870, in run
File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample
self._target(*self._args, **self._kwargs)
File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample
librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best")
File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample
self._target(*self._args, **self._kwargs)
File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample
librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best")
File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample
y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)
y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)
AttributeError: AttributeError: module 'resampy' has no attribute 'resample'
module 'resampy' has no attribute 'resample' librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best")
File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample
python-BaseException
y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)
AttributeError: module 'resampy' has no attribute 'resample'
Exception in thread Thread-6:
Traceback (most recent call last):
File "C:\Build\envs\dec5\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "C:\Build\envs\dec5\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "D:\source\internal_ai_deepaudio\training\break_lazy_load.py", line 7, in resample
librosa.resample(np.zeros(1000), orig_sr=24_000, target_sr=16_000, res_type="kaiser_best")
File "C:\Build\envs\dec5\lib\site-packages\librosa\core\audio.py", line 677, in resample
y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)
AttributeError: module 'resampy' has no attribute 'resample'
Software versions
Windows-10-10.0.22621-SP0
Python 3.8.0 (default, Nov 6 2019, 16:00:02) [MSC v.1916 64 bit (AMD64)]
NumPy 1.23.5
SciPy 1.10.1
librosa 0.10.1
INSTALLED VERSIONS
------------------
python: 3.8.0 (default, Nov 6 2019, 16:00:02) [MSC v.1916 64 bit (AMD64)]
librosa: 0.10.1
audioread: 3.0.1
numpy: 1.23.5
scipy: 1.10.1
sklearn: 1.3.2
joblib: 1.3.2
decorator: 5.1.1
numba: 0.58.1
soundfile: 0.12.1
pooch: v1.8.0
soxr: 0.3.7
typing_extensions: installed, no version number available
lazy_loader: installed, no version number available
msgpack: 1.0.7
numpydoc: None
sphinx: None
sphinx_rtd_theme: None
matplotlib: None
sphinx_multiversion: None
sphinx_gallery: None
mir_eval: None
ipython: None
sphinxcontrib.rsvgconverter: None
pytest: None
pytest_mpl: None
pytest_cov: None
samplerate: None
resampy: 0.4.2
presets: None
packaging: 23.2
Metadata
Metadata
Assignees
Labels
No labels