Created by: loretoparisi

Hello, I have built a Docker image of DeepSpeech. I prefer to not install every time the swig wrappers since it should be enough to build once and then load at runtime. Therefore this is what I did

I have built my library in the lib/ folder within my swig folder, because I need the shared library (.so) to be there in the package without building it at install. Without any modifications to swig_import_helper I got the error

Traceback (most recent call last):
  File "/MyProject/swig_wrapper.py", line 6, in <module>
    import swig_decoders
  File "/MyProject/swig/swig_decoders.py", line 26, in <module>
    _swig_decoders = swig_import_helper()
  File "/MyProject/swig/swig_decoders.py", line 25, in swig_import_helper
    return importlib.import_module('_swig_decoders')
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_swig_decoders'

I have thereforre modified importlib in order load the local library:

def swig_import_helper():
        import importlib
        pkg = __name__.rpartition('.')[0]
        mname = '.'.join((pkg, '_swig_decoders')).lstrip('.')
        try:
            return importlib.import_module(mname)
        except ImportError:
            try:
                import os
                import importlib.util
                BASE_PATH = os.path.join(os.path.dirname(os.path.realpath(__file__)))
                spec = importlib.util.spec_from_file_location("module._swig_decoders", os.path.join(BASE_PATH, 'lib'))
                foo = importlib.util.module_from_spec(spec)
                spec.loader.exec_module(foo)
            except:
                return importlib.import_module('_swig_decoders')
    _swig_decoders = swig_import_helper()

the libraries are located in the local path swig/lib:

.
└── swig_decoders-1.1-py3.7-linux-x86_64.egg
    ├── EGG-INFO
    ├── __pycache__
    ├── _swig_decoders.cpython-37m-x86_64-linux-gnu.so
    ├── _swig_decoders.py
    └── swig_decoders.py

Also in the swig/build folder I have the static compiled libraries:

[17:24:43] ip-192-168-1-52:build loretoparisi$ tree -L 2
.
├── lib.linux-x86_64-3.7
│   ├── _swig_decoders.cpython-37m-x86_64-linux-gnu.so
│   └── swig_decoders.py
└── temp.linux-x86_64-3.7
    ├── ctc_beam_search_decoder.o
    ├── ctc_greedy_decoder.o
    ├── decoder_utils.o
    ├── decoders_wrap.o
    ├── kenlm
    ├── openfst-1.6.3
    ├── path_trie.o
    └── scorer.o

I'm still getting an import error when I build:

  File "MyProject/swig/swig_decoders.py", line 14, in swig_import_helper
    return importlib.import_module(mname)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named '_swig_decoders'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "MyProject/swig/swig_decoders.py", line 21, in swig_import_helper
    foo = importlib.util.module_from_spec(spec)
  File "<frozen importlib._bootstrap>", line 580, in module_from_spec
AttributeError: 'NoneType' object has no attribute 'loader'

Is this due to a library path error or to the importlib when using importlib.util.spec_from_file_location?

I have asked the same question here: https://stackoverflow.com/questions/60994100/swig-import-local-shared-library-in-python3