Python: Dynamically load all modules in a folder
In the __init__.py
of the directory you want to include recursively:
python
import pkgutil__path__ = pkgutil.extend_path(__path__, __name__)for importer, modname, ispkg in pkgutil.walk_packages(path=__path__, prefix=__name__+'.'):__import__(modname)
More Context
In my current project, I have a directory of data collectors that contain both a poller
and a parser
component. To keep things nice and clean, they're located in a folder called collectors
. The directory structure looks like this:
bash
collectors├── base│ ├── __init__.py│ ├── parser.py│ ├── poller.py├── graphite│ ├── __init__.py│ ├── parser.py│ ├── poller.py├── analytics│ ├── __init__.py│ ├── parser.py│ ├── poller.py└── __init__.py
base.poller
and base.parser
are factory classes, which are responsible for creating instances of either graphite.*
or analytics.*
. This means that these are the only classes I need to expose. To do this, I use __all__
in __init__.py
in the collectors
folder:
python
from base.poller import Pollerfrom base.parser import Parser__all__ = ["Poller", "Parser"]
This means that I can use the following in my code:
python
from collectors import Poller, Parser
The next thing to do, is import all of the graphite
and analytics
modules, as they register themselves with the factories on load. I don't need to access them directly in my code as I access things via the factories. To do this, I use the following code (modified based on code from nulledge).
python
import pkgutil__path__ = pkgutil.extend_path(__path__, __name__)for importer, modname, ispkg in pkgutil.walk_packages(path=__path__, prefix=__name__+'.'):__import__(modname)
Then in my code, I use the following to access the modules that registered themselves:
python
gpoller = Poller.create("graphite")