Android Neural Network API based Mobile Scoring APP

Introduction

With deep learning and AI technology rapidly growing, a lot of deep learning frameworks has been developed to support training a solution. After having a well-trained model, a device would be chosen to perform huge computations and parameter access of the model. However, that is a big challenge, like power consumption and performance issues, in power-constrained edge devices. Recently, mobile devices are enabled with AI capability by adopting advance deep learning model and powerful chips. And, the mobile operating system, Android, develops a unified neural network API infterface(NNAPI) for running computationally intensive operations for machine learning on mobile devices. It gives deep learning applications an opportunity to realize them on the mobile devices. In addition, for Android AI application developer, it provide them to dispatch their execution tasks to different processors easier by using tensorflow-lite to call NNAPI. To evaluate the abilities of deep learning mobile devices, AI System Benchmarking and Tuning Lab (人工智慧系統檢測中心), National Chiao Tung University, Taiwan has announced this benchmark for deep learning neural network on mobile devices, named NeuralScope benchmark. This is based on Android NNAPI to test computational capacities of mobile devices.

Android NNAPI

The NNAPI is an intermediate layer between the machine learning frameworks and the device’s hardware acceleration. Moreover, NNAPI is responsible for scheduling the each operation to the corresponding hardware component. However, only CPU is utilized in default NNAPI drivers. In order to execute on other processing units (APU,DLA,DSP etc.), the vendors of the smartphones or SoCs need to provide specific drivers. Therefore,it has no performance guarantee in Android 8.1+.

Test Summary

Our current test has a total of four categories, image classification, object detection, image segmentation and multi-task hybrid mode. And each of them have both floating & quantized test.

Image classification:

The deep learning based neural networks are able to recognize object classes for one or more given input photos. They can recognize 1000 different object classes. There are four models, mobilenet-V1, mobilenet-V2, Resnet-50, and Inception-V3 in our benchmarking App. Individually, we provide one float model and one quantized model for each network.

Object detection:

Now, you can perform object counting on your phone. The model we use is a combination of mobilenet, a light-weight classification model, and a single shot multibox detector(SSD. An object detector does not re-sample pixels or feature maps for bounding box hypotheses and can detect 80 different object classes. It improves the speed for high-accuracy detection.

Image Segmentation:

The task evaluates whether you can perform auto background removing for exchange scene on your phone. It can recognize 20 different object classes and segment the recognized object using different colors. The model is resnet-50 with atrous convolution layers embedded.

Benchmark Results

The results in figures 1 and 2 show the score of each category on Nokia 5.1 plus and Huawei P20 respectively. On Nokia 5.1 plus, we can see that it performed well in classification and detection test. We speculate that this is related to the support of NNAPI on the chipset Helio P60 which Nokia 5.1 plus is powered by. Huawei P20, which is powered by Kirin 970, also have a great performance in classification, especially in the floating point operation, the NPU in Huawei P20 provides an excellent speed up.

Figure 3 show the score of Huawei Mate20. Mate20 is Chinese manufacturer’s latest flagship smartphone. It has specially enhanced the computing power in neural network operation. Therefore the results show that there is more than twice the acceleration than other devices.

Conclusion

Now, AI applications are common on mobile devices, so we should pay attention to whether the mobile phone provides acceleration in AI ? NeuralScope benchmark is a good tool for end users to evaluate their mobile devices.