Android Neural Network API based Mobile Scoring APP
Introduction
With deep learning and AI technology rapidly growing, a lot of deep learning frameworks has been developed to support training a solution. After having a well-trained model, a device would be chosen to perform huge computations and parameter access of the model. However, that is a big challenge, like power consumption and performance issues, in power-constrained edge devices. Recently, mobile devices are enabled with AI capability by adopting advance deep learning model and powerful chips. And, the mobile operating system, Android, develops a unified neural network API infterface(NNAPI) for running computationally intensive operations for machine learning on mobile devices. It gives deep learning applications an opportunity to realize them on the mobile devices. In addition, for Android AI application developer, it provide them to dispatch their execution tasks to different processors easier by using tensorflow-lite to call NNAPI. To evaluate the abilities of deep learning mobile devices, AI System Benchmarking and Tuning Lab (人工智慧系統檢測中心), National Chiao Tung University, Taiwan has announced this benchmark for deep learning neural network on mobile devices, named NeuralScope benchmark. This is based on Android NNAPI to test computational capacities of mobile devices.
Android NNAPI
The NNAPI is an intermediate layer between the machine learning frameworks and the device’s hardware acceleration. Moreover, NNAPI is responsible for scheduling the each operation to the corresponding hardware component. However, only CPU is utilized in default NNAPI drivers. In order to execute on other processing units (APU,DLA,DSP etc.), the vendors of the smartphones or SoCs need to provide specific drivers. Therefore,it has no performance guarantee in Android 8.1+.
Test Summary
Our current test has a total of four categories, image classification, object detection, image segmentation and multi-task hybrid mode. And each of them have both floating & quantized test.
Image classification:
Object detection:
Image Segmentation:
Benchmark Results
The results in figures 1 and 2 show the score of each category on Nokia 5.1 plus and Huawei P20 respectively. On Nokia 5.1 plus, we can see that it performed well in classification and detection test. We speculate that this is related to the support of NNAPI on the chipset Helio P60 which Nokia 5.1 plus is powered by. Huawei P20, which is powered by Kirin 970, also have a great performance in classification, especially in the floating point operation, the NPU in Huawei P20 provides an excellent speed up.
Figure 3 show the score of Huawei Mate20. Mate20 is Chinese manufacturer’s latest flagship smartphone. It has specially enhanced the computing power in neural network operation. Therefore the results show that there is more than twice the acceleration than other devices.
Conclusion
Now, AI applications are common on mobile devices, so we should pay attention to whether the mobile phone provides acceleration in AI ? NeuralScope benchmark is a good tool for end users to evaluate their mobile devices.