# cann-halo

**Repository Path**: cann/cann-halo

## Basic Information

- **Project Name**: cann-halo
- **Description**: HALO/ODLA通过CANN的开放接口使能Ascend的Adapter Sample
- **Primary Language**: C++
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 8
- **Forks**: 8
- **Created**: 2021-04-27
- **Last Updated**: 2023-05-19

## Categories & Tags

**Categories**: chips, ai

**Tags**: None

## README

# ODLA集成Ascend操作指南

## 环境准备

### 硬件环境

Atlas300I(3000)服务器，要求内存不小于16G，根目录磁盘剩余空间大于64G。按照[《用户指南》](https://support.huawei.com/enterprise/zh/doc/EDOC1100115618?idPath=23710424%7C251366513%7C22892968%7C250702915)安装最新的驱动与固件

### 软件环境

gcc/g++:  服务器gcc/g++版本大于等于7.5.0

docker：在Atlas300服务器上安装好docker

### docker容器配置

1.在服务器上以root用户（注意：本readme所有命令都是在root用户下执行）下载ODLA镜像.参考[ODLA环境搭建readme](https://github.com/alibaba/heterogeneity-aware-lowering-and-optimization/blob/master/docs/how_to_build.md)拉取一个仅支持CPU的镜像：

```
docker pull registry-intl.us-west-1.aliyuncs.com/computation/halo:latest-devel-x86_64-ubuntu18.04
```

注意：不要使用该readme中的[start_docker_cpu.sh](https://github.com/alibaba/heterogeneity-aware-lowering-and-optimization/blob/master/utils/docker/start_docker_cpu.sh)脚本拉取和启动镜像

2.修改ODLA镜像启动脚本[start_docker_cpu.sh](https://github.com/alibaba/heterogeneity-aware-lowering-and-optimization/blob/master/utils/docker/start_docker_cpu.sh)添加了Atlas300I的驱动映射，即将：

```
ID=`docker run $docker_run_flag --privileged --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -t -d --name $CONTAINER_NAME -v $MOUNT_DIR:/host  --tmpfs /tmp:exec --rm $IMAGE `
```

修改为

```
ID=`docker run $docker_run_flag --privileged --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --net=host -t -d --name $CONTAINER_NAME --cap-add LINUX_IMMUTABLE --device=/dev/davinci0 --device=/dev/davinci_manager --device=/dev/devmm_svm --device=/dev/hisi_hdc -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver/:/usr/local/Ascend/driver -v $MOUNT_DIR:/host  --tmpfs /tmp:exec $IMAGE `
```

其他代码不做变更。

3.启动镜像。执行修改后的start_docker_cpu.sh，会自动进入docker容器内。

4.源码安装python3.7.5。请参考[《安装Python3.7.5（Ubuntu）》](https://github.com/alibaba/heterogeneity-aware-lowering-and-optimization/blob/master/docs/how_to_build.md)安装依赖、编译安装python3.7.5并设置环境变量。

5.在容器中创建HwHiAiUser用户：

（1）在宿主机的命令行下输入命令 

```
id HwHiAiUser
```

查看宿主机HwHiAiUser的gid 

（2）在容器内创建HwHiAiUser用户

```
groupadd -g <gid> HwHiAiUser && useradd -g HwHiAiUser -d /home/HwHiAiUser -m HwHiAiUser && echo ok
```

请将上述命令中的gid替换为（1）中查出的gid值。若返回“ok”，则说明创建成功。

6.将宿主机中的/etc/ascend_install.info拷贝到容器/etc/目录下

7.在容器中安装Atlas300I最新的CANN包。[下载](https://www.hiascend.com/software/cann)最新的cann社区版本或者商用版本，参考[《环境部署》](https://www.hiascend.com/document?tag=community-developer)安装cann ascend-toolkit包

## Halo集成Ascend

1.按照[ODLA环境搭建readme](https://github.com/alibaba/heterogeneity-aware-lowering-and-optimization/blob/master/docs/how_to_build.md)下载halo仓代码

git clone https://github.com/alibaba/heterogeneity-aware-lowering-and-optimization.git --recurse-submodules -j8

如果网络环境不好，不要添加-j8参数

2.下载ascend代码

 git clone https://gitee.com/cann/cann-halo.git

3.将cann-halo/ascend拷贝到heterogeneity-aware-lowering-and-optimization/ODLA/platforms/目录下

4.修改heterogeneity-aware-lowering-and-optimization/ODLA/platforms/CMakeLists.txt, 添加

```
option(ODLA_BUILD_ASCEND "Build ODLA Runtime Library for ASCEND" ON)
if (ODLA_BUILD_ASCEND)
  add_subdirectory(ascend)
endif()
```

5.在heterogeneity-aware-lowering-and-optimization下创建build目录并编译halo

```
source /usr/local/Ascend/ascend-toolkit/set_env.sh

mkdir -p build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DODLA_BUILD_TRT=NO -G Ninja ..
ninja 
```

其中HALO_ROOT是下载的halo代码路径，根据实际情况设置。在build/lib目录下生成了libodla_ascend.so，说明ascend集成成功

## Resnet50_v2样例集成Ascend

在halo odla平台的resnet50_v2中集成ascend，需要修改的有：

1.在[resnet50_v2样例入口脚本](https://github.com/alibaba/heterogeneity-aware-lowering-and-optimization/blob/master/models/vision/classification/resnet/run_resnet_v2_50.sh)中添加ascend入口

```
if [[ $TEST_WITH_ASCEND -eq 1 ]]; then
  echo "======== Testing with ODLA Ascend ========"
  python3 $curr_dir/../../invoke_halo.py --model $model_file --label-file \
          $curr_dir/../1000_labels.txt --image-dir $image_dir \
          --odla ascend | tee $2
fi
```

2.[invoke_halo.py](https://github.com/alibaba/heterogeneity-aware-lowering-and-optimization/blob/master/models/vision/invoke_halo.py)中使用halo工具生成模型代码文件时，权重文件要求是源码形式，所以要使用--emit-data-as-c选项，类似：

```
halo --emit-data-as-c -target cxx resnet50-v2-7.onnx -o resnet50_v2_model.cc
```

3.g++编译选项中的ascend相关头文件搜索路径、库搜索路径、链接库

（1）ascend头文件搜索路径

```
/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/built-in/inc 
/usr/local/Ascend/ascend-toolkit/latest/fwkacllib/include 
```

（2）ascend库搜索路径

```
/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/built-in/ai_core/tbe/op_tiling/ 
/usr/local/Ascend/ascend-toolkit/latest/fwkacllib/lib64/plugin/opskernel/ 
/usr/local/Ascend/ascend-toolkit/latest/fwkacllib/lib64/stub/
```

（3）链接的ascend库有libgraph.so, libge_runner.so, libascendcl.so, libfe.so和liboptiling.so

4.应用执行时依赖的ascend库搜索路径


```
/usr/local/Ascend/driver/lib64
/usr/local/Ascend/driver/lib64/common
/usr/local/Ascend/driver/lib64/driver
/usr/local/Ascend/ascend-toolkit/latest/fwkacllib/lib64
/usr/local/Ascend/ascend-toolkit/latest/atc/lib64
```

这些环境变量的设置也可以直接在docker容器内执行：

```
source /usr/local/Ascend/ascend-toolkit/set_env.sh
export PYTHONWARNINGS='ignore:semaphore_tracker:UserWarning'
```