Dynamic Backend Loading

Updated 1,774 Days AgoPublic

Actions

Technical Contact: Matteo Martincigh

Pre-Implementation Review

Post-Implementation Review

Abstract

This is a design note to add to ability to dynamically load a backend in the ARmNN's runtime

Overview

Enable ArmNN to discover all the backends available on a system and dynamically load any it might find during the runtime's startup. It should be possible for the same compiled libarmnn.so deployed on different systems to load different backends. The same backend can be loaded simultaneously be different runtimes.

The current way of statically adding a backend at compile time must be retained.

Detailed Design

Main specifications:

• Load the backends as shared libraries from a "backends" subdirectory in the path where libarmnn.so is
• Use dlopen, dlclose and dlsym to load, unload and obtain the address of a symbol in the shared object respectively
• All done in the Runtime constructor? Create a new "LoadBackends" method in the Runtime class. Or add to the IRunTime::Create factory method
• To be loaded properly, a shared object must implement all the methods defined by the IBackendInternal interface (rename it to IBackend once and for all?)
• The backends implementation must be thread-safe where necessary, as the same backend object can be loaded by multiple runtimes?
• If any backend fails to load, the error must be reported but the runtime should ignore the failing backend and continue to the next one, if any
• Provide detailed documentation for the users who want to create their own (dynamic) backend