**Technical Contact:** Matteo Martincigh
== Pre-Implementation Review ==
== Post-Implementation Review ==
= Abstract =
This is a design note to add to ability to dynamically load a backend in the ARmNN's runtime
| Version | Date | Changes | Author
|---|---|---|---
| 0.1 | 10/06/2016 | Initial draft | Matteo Martincigh
= Overview =
Enable ArmNN to discover all the backends available on a system and dynamically load any it might find during the runtime's startup. It should be possible for the same compiled libarmnn.so deployed on different systems to load different backends. The same backend can be loaded simultaneously by different runtimes.
The current way of statically adding a backend at compile time must be retained.
= Detailed Design =
Main specifications:
*# A "backend object" or "dynamic backend object" is a shared object that conforms to ArmNN's IBackendInternal interface
# Load the backends as shared libraries from a "backends" subdirectory in the path where libarmnn.so is, or make the backends path an argument that can be passed to the runtime via CreationOptions (this second option being the easiest)
# Use dlopen, dlclose and dlsym to load, unload and obtain the address of a symbol in the shared object respectively (see code snippet below)
* Use dlopen,# Create a new "LoadDynamicBackends" method in the Runtime class. dlclose and dlsym to load,The method must be private so that it can't be called outside the normal intended use. unload and obtainCall the address of a symbol in the shared object respectivelynew method in the IRunTime::Create factory method
* All done in the Runtime constructor? Create a new "LoadBackends" method in LoadDynamicBackends must be called before the Runtime class.onstructor, Or add to the IRunTime::Create factory methodas the Runtime constructor need the BackendRegistry to be already populated
*# To be loaded properly, a shared object must implement all the methods defined by the IBackendInternal interface
* For this purpos# If any backend fails to load, the error must be reported but the runtime should ignore the failing backend and continue to the next one, the IBackendInternal interface should be renamed to IBackendif any, asand the suffix "Internal" will be now misleadingn ultimately start as usual
* The backends implementation must be thread-safe where necessary# Provide detailed documentation for the users who want to create their own (dynamic) backend
= Implementation Plan =
== Load the backends as shared libraries from a specific path ==
During the creation of the Runtime object, ArmNN should scan a given path searching for suitable backend objects. The (absolute) path can be specific through the CreationOptions class, as the same backend object can be loaded by multiple runtimes?that is passed to the Runtime for its construction.
* If any backend fails to loadA default path should be provided as a fallback, the error must be reported but the runtime should ignofor example a subdirectory named "backends" respect where the failing backend and continue to the next one, if anyArmNN shared object (libarmnn.so) lives on the system.
* Provide detailed documentation for the users who want to create their own (dynamic) backendGetting the path of a shared library from itself can be tricky on Linux system, as it requires to parse /proc/self/maps where the full path of every loaded shared object is listed.
== Use the dlfcn API to load the backend objects ==
Once the path to the backend objects has been located, proceed to load them using the standard dlopen, dlclose and dlsym functions.
== Create a new LoadDynamicBackends method in the Runtime class ==
Add a new private LoadDynamicBackends method to the Runtime class that makes use of the utilities developed in the previous point.
The LoadDynamicBackends should attempt to load any shared object that's considered suitable (i.e. complient to the IBackendInternal interface) regardless of its filename.
If a valid object is found, that backend should be registered in the BackendRegistry
== Add to the Backends documentation ==
Update the docs with information and examples about adding a custom dynamic backend object to an ArmNN's deployment.
Detail all the steps necessary to create a dynamic backend from scratch, and how to use make use of it in ArmNN.
Mention possible/common errors (where to get the error log, the object is not found, found but not loaded, loaded but not registered, registered but not used, etc.) and give basic troubleshooting advice on what the possible causes of the problems could be and suggest a valid fix.
Mention the possibility of having dynamic backends in the existing static backends documentation as well.