The Khronos Group, the industry consortium responsible for graphics APIs like OpenGL, WebGL, OpenCL, and Vulkan, has announced a new neural network exchange format 1.0 specification (NNEF 1.0).
NNEF 1.0 Standard
After gathering feedback from the provisional specification, NNEF 1.0 has been released as a stable, flexible, and extensible open standard for hardware manufacturers to reliably enable widespread deployment of neural networks onto edge devices. New tools are now available on Github including an NNEF parser and converters from Tensorflow and Caffe. There are importers too into popular inferencing environments including Android’s Neural Network API under development.
NNEF was created to reduce fragmentation by facilitating the exchange of neural networks among training frameworks and inference engines, increasing the freedom for users to mix and match the inference engines and training frameworks of their choice. The standard, the result of collaboration between industry-leading members of the Khronos Working Group, guarantees the stability that hardware and software companies can rely on for product deployment while maintaining the flexibility required to respond to the needs of rapidly-moving industries.
01 – NNEF is the latest new standard from the Khronos Group and deals with neural networks.
NNEF 1.0 will allow reliable export from the rapidly growing number of frameworks, including Torch, Chainer, Theano, PyTorch, and MXNet in addition to those for which tools already exist. A complete workflow from training, through optimization, to deployment is possible using NNEF as the standard transfer format between tools. Open source tools are available to allow easy creation of importers into custom and embedded edge-inferencing environments. For example, open source efforts are already underway to produce importers into more general-purpose environments such as NN API and CoreML.
NNEF was designed to accommodate a wide range of use-cases and network types. Its basic format is designed to be both human readable and easy to parse, whereas its extensions allow for human editing of large networks. It is designed to accommodate rapid evolution of frameworks without the need to change the base standard, and provides an extension mechanism to handle specific issues such as custom data formats for trained network weights.
At launch, the standard will be supported by two open source Tensorflow converters both for network descriptions based on protobuf and python code, and as a converter for Caffe. A Caffe2 open-source converter in development by Au-Zone Technologies will be available in Q3 2018. Various tools from member companies Almotive and AMD will be available, and an importer to the Android NN API is under development by a team from National Tsing-Hua University of Taiwan.
“Khronos recognized a growing format compatibility problem for companies wanting to deploy trained neural networks onto edge devices. We set out to build the first standard platform for engineers to easily deploy networks from training frameworks to inference engines. Today, we are proud to release NNEF 1.0 as a stable, open specification that will allow existing hardware to work but continue to evolve through flexible extension mechanisms to keep up with this fast changing field of machine learning,” said Khronos NNEF Working Group Chair Peter McGuinness. “In December 2017 we released the developer preview of NNEF and made an open call for industry feedback. Community response has been tremendous, confirming the demand for this standard and enabling us to achieve a responsive and complete NNEF 1.0 specification.”
The NNEF 1.0 specification and documentation are freely available on the Khronos website, and NNEF open source tools and projects are available in the Khronos NNEF Tools repository.