前言

硬件：创通联达以高通QCS610作为SOC的开发板TurboX C610

SDK：Qualcomm Neural Processing SDK for AI v1.59.0

docs：Reference Guide

骁龙神经处理引擎（SnapDragon Neural Processing Engine）是高通用于加速神经网络的运行时推理框架。

Setup

必备条件

目前SNPE SDK开发环境仅支持Ubuntu-18.04；
支持Caffe，Caffe2，ONNX，PyTorch，TensorFlow，TFLite；
Python3.6
Android NDK（可选）：用于构建SDK附带的Cpp示例
Android SDK（可选）：用于构建 SDK附带的APK

安装docker-ubuntu18.04

docker pull ubuntu:18.04
docker container run -itd --name snpe_1.59.0 ubuntu:18.04
docker exec -it snpe_1.59.0 /bin/bash

安装python3.6

换apt-get源

cp /etc/apt/sources.list /etc/apt/sources.list.bak
vim /etc/apt/sources.list

删除原内容，添加以下内容，保存

#阿里云源
deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse

更新apt-get并安装python3.6

apt-get update
apt-get install python3.6
ln -s /usr/bin/python3.6 /usr/bin/python
python --version

安装SDK

apt-get install unzip
apt-get install sudo
unzip -X snpe-1.59.0.zip
source snpe-1.59.0.3230/bin/dependencies.sh

检查依赖python包安装情况，自行安装缺失的包：

source snpe-1.59.0.3230/bin/check_python_depends.sh
apt-get install python-pip
apt-get install python3-pip

这时发现python默认版本被指定为python2.7（可能是安装python-pip时安装了python2）：

python --verison
ls /usr/bin/python* -l

lrwxrwxrwx 1 root root 9 Apr 16 2018 /usr/bin/python -> python2.7
lrwxrwxrwx 1 root root 9 Apr 16 2018 /usr/bin/python2 -> python2.7
-rwxr-xr-x 1 root root 3633000 Feb 27 2021 /usr/bin/python2.7
lrwxrwxrwx 1 root root 9 Oct 25 2018 /usr/bin/python3 -> python3.6
-rwxr-xr-x 2 root root 4526456 Dec 8 21:08 /usr/bin/python3.6

可以看到/usr/bin/python软链接指向了python2.7文件夹，可以手动删掉软链接并重新指定。这里尝试使用update-alternatives来管理版本：

update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1
update-alternatives --install /usr/bin/python python /usr/bin/python3.6 2
update-alternatives --list python # 查看 python 对应的版本有哪些
update-alternatives --config python # 切换 python 版本

再次查看软链接：

ls /usr/bin/python -l

lrwxrwxrwx 1 root root 24 Feb 18 06:26 /usr/bin/python -> /etc/alternatives/python

发现软链接指向了/etc/alternatives/python，再看下这是个啥：

ls /etc/alternatives/python -l

lrwxrwxrwx 1 root root 18 Feb 18 06:30 /etc/alternatives/python -> /usr/bin/python3.6

哦，update-alternatives管理版本的机制就是增加了一个/etc/alternatives/python软链接（中间层）来实现的。那这时pip命令的版本怎么对应python版本呢？以pip -V打印版本信息为准，实测当切到python3.6时，无论pip，pip2，pip3都是对应python3.6的包管理，而切到python2.7时，pip和pip2都对应python2.7的包管理，pip3对应python3.6的包管理。

升级pip版本，并换源：

pip install -U pip
pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/ # 阿里源
# pip config unset global.index-url # 换回默认源

安装依赖包：

pip install numpy
pip install sphinx
pip install scipy
pip install matplotlib
pip install scikit-image
pip install protobuf
pip install pyyaml
pip install mako

docker内adb连接

首先确保主机可以通过usb-adb连接到板子，这时就可以用界面投影工具scrcpy.exe进入的Android系统UI界面，在UI内操作连接与主机同一局域网的WIFI，并记下板子的IP地址10.0.49.7。

在docker容器内尝试ping板子的IP：

ping 10.0.49.7

可以ping通的话，在主机端以TCP/IP方式重启ADB服务：

adb tcpip 5555

在docker内安装adb工具，并通过网络连接adb：

apt-get install android-tools-adb
adb connect 10.0.49.7
adb devices

* daemon not running; starting now at tcp:5037

* daemon started successfully
connected to 10.0.49.7:5555

···

List of devices attached
10.0.49.7:5555 device

安装Android NDK

拷贝下载好的android-ndk-r20b-linux-x86_64.zip到docker中，解压到想要存放的位置：

unzip android-ndk-r20b-linux-x86_64.zip

配置环境变量

配置SNPE SDK和Android NDK的环境变量：

vim ~/.bashrc

在文件最后添加如下内容：

# SNPE SDK
export SNPE_ROOT=/root/snpe-1.59.0.3230/

# Android NDK
export ANDROID_NDK_ROOT=/root/android-ndk-r20b/
export ANDROID_NDK=/root/android-ndk-r20b/
export NDKROOT=/root/android-ndk-r20b/
export PATH=$NDKROOT:$PATH

执行source ~/.bashrc使其生效

安装Caffe并配置环境

apt-get install caffe-cpu
source $SNPE_ROOT/bin/envsetup.sh -c /usr/bin/ # 添加到.bashrc

安装ONNX并配置环境

pip install onnx
pip install onnx-simplifier
source $SNPE_ROOT/bin/envsetup.sh -o /usr/local/lib/python3.6/dist-packages/onnx # 添加到.bashrc

编译示例程序

编译运行在ARM Android上的示例程序

cd $SNPE_ROOT/examples/NativeCpp/SampleCode
ndk-build NDK_TOOLCHAIN_VERSION=clang APP_STL=c++_shared

这会编译armeabi-v7a和arm64-v8a两个版本的可执行文件，分别为：

$SNPE_ROOT/examples/NativeCpp/SampleCode/obj/local/armeabi-v7a/snpe-sample
$SNPE_ROOT/examples/NativeCpp/SampleCode/obj/local/arm64-v8a/snpe-sample

执行示例程序

拷贝可执行程序及依赖库到板子上：

export SNPE_TARGET_ARCH=aarch64-android-clang6.0
export SNPE_TARGET_ARCH_OBJ_DIR=arm64-v8a
adb push $SNPE_ROOT/lib/$SNPE_TARGET_ARCH/ /data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/lib
adb push $SNPE_ROOT/lib/dsp/ /data/local/tmp/snpeexample/dsp/lib
adb push $SNPE_ROOT/examples/NativeCpp/SampleCode/obj/local/$SNPE_TARGET_ARCH_OBJ_DIR/snpe-sample /data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/bin/snpe-sample

准备alexnet模型和数据：

cd $SNPE_ROOT/models/alexnet/scripts
python setup_alexnet.py -a ../tmpdir -d

发现下载不动，只好手动下载：

下载完拷贝到$SNPE_ROOT/models/alexnet/tmpdir文件夹下，然后执行：

python setup_alexnet.py -a ../tmpdir

拷贝模型及数据到板子上：

cd $SNPE_ROOT/models/alexnet
mkdir data/rawfiles && cp data/cropped/*.raw data/rawfiles/
adb push data/rawfiles /data/local/tmp/alexnet/cropped
adb push data/target_raw_list.txt /data/local/tmp/alexnet
adb push dlc/bvlc_alexnet.dlc /data/local/tmp/alexnet
rm -rf data/rawfiles

配置板子上的环境变量：

adb shell
export SNPE_TARGET_ARCH=aarch64-android-clang6.0
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/lib
export PATH=$PATH:/data/local/tmp/snpeexample/$SNPE_TARGET_ARCH/bin

执行程序：

cd /data/local/tmp/alexnet
snpe-sample -b ITENSOR -d bvlc_alexnet.dlc -i target_raw_list.txt -o output_sample
exit

结果pull到本地（docker）：

cd $SNPE_ROOT/models/alexnet/
adb pull /data/local/tmp/alexnet/output_sample ./output_sample

调用python脚本分析结果：

python scripts/show_alexnet_classifications.py -i data/target_raw_list.txt \
											   -o output_sample/ \
											   -l data/ilsvrc_2012_labels.txt

会有以下结果输出：

Classification results
cropped/trash_bin.raw 0.950433 412 ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin
cropped/chairs.raw 0.363969 831 studio couch, day bed
cropped/notice_sign.raw 0.666179 458 brass, memorial tablet, plaque
cropped/plastic_cup.raw 0.720699 647 measuring cup

查看模型信息

snpe-dlc-info -i xxx.dlc

SNPE SDK跑ONNX模型

执行如下代码，判断环境是否配置正确：

snpe-onnx-to-dlc -h

下载VGG模型：

cd $SNPE_ROOT/models/VGG/onnx
wget https://s3.amazonaws.com/onnx-model-zoo/vgg/vgg16/vgg16.onnx

下载测试图片和标签文件：

cd $SNPE_ROOT/models/VGG/data
wget https://s3.amazonaws.com/model-server/inputs/kitten.jpg
wget https://s3.amazonaws.com/onnx-model-zoo/synset.txt

对图片进行预处理（1. resize到256x256；2. 中心裁切到224x224；3. 归一化；4. 保持为raw文件）：

cd $SNPE_ROOT/models/VGG/
mkdir data/cropped
python scripts/create_VGG_raws.py -i data/ -d data/cropped/
python scripts/create_file_list.py -i data/cropped/ -o data/cropped/raw_list.txt -e '*.raw'

模型转换：

cd $SNPE_ROOT/models/VGG
snpe-onnx-to-dlc -i onnx/vgg16.onnx -o dlc/vgg16.dlc

有如下打印：

WARNING - WARNING_GEMM: GEMM operation is not supported in the general case, attempting to interpret as FC
WARNING - WARNING_GEMM: GEMM operation is not supported in the general case, attempting to interpret as FC
WARNING - WARNING_GEMM: GEMM operation is not supported in the general case, attempting to interpret as FC
INFO - INFO_DLC_SAVE_LOCATION: Saving model at vgg16.dlc
INFO - INFO_CONVERSION_SUCCESS: Conversion completed successfully

与alexnet的例子类似，上述步骤2~5也可以用封装好的脚本$SNPE_ROOT/models/VGG/scripts/setup_VGG.py完成。

查看模型信息：

snpe-dlc-info -i vgg16.dlc

模型推理：

cd $SNPE_ROOT/models/VGG
snpe-net-run --input_list data/cropped/raw_list.txt --container dlc/vgg16.dlc --output_dir output

查看推理结果：

cd $SNPE_ROOT/models/VGG/
python scripts/show_vgg_classifications.py -i data/cropped/raw_list.txt -o output/ -l data/synset.txt

有如下打印：

Classification results
probability=0.351833 ; class=n02123045 tabby, tabby cat
probability=0.315166 ; class=n02123159 tiger cat
probability=0.313086 ; class=n02124075 Egyptian cat
probability=0.012995 ; class=n02127052 lynx, catamount
probability=0.003528 ; class=n02129604 tiger, Panthera tigris