From e31b21382f512eeb5fabd996fc9ed2b8443683e2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E6=9D=8E=E5=AF=85?= Date: Wed, 19 Dec 2018 15:22:18 +0800 Subject: [PATCH] Add python dependency requirements and how-to-debug doc --- docs/development/how_to_debug.rst | 85 +++++++++++++++++++++++++++ docs/index.rst | 1 + docs/installation/env_requirement.rst | 39 ++++++++---- setup/optionals.txt | 4 ++ setup/requirements.txt | 6 ++ 5 files changed, 123 insertions(+), 12 deletions(-) create mode 100644 docs/development/how_to_debug.rst create mode 100644 setup/optionals.txt create mode 100644 setup/requirements.txt diff --git a/docs/development/how_to_debug.rst b/docs/development/how_to_debug.rst new file mode 100644 index 00000000..1026f3ff --- /dev/null +++ b/docs/development/how_to_debug.rst @@ -0,0 +1,85 @@ +How to debug +============== + +Log debug info +-------------------------- +Mace defines two sorts of logs: one is for users (LOG), the other is for developers (VLOG). + +LOG includes four levels, i.e, ``INFO``, ``WARNING``, ``ERROR``, ``FATAL``; +Environment variable ``MACE_CPP_MIN_LOG_LEVEL`` can be set to specify log level of users, e.g., +``set MACE_CPP_MIN_LOG_LEVEL=0`` will enable ``INFO`` log level, while ``set MACE_CPP_MIN_LOG_LEVEL=4`` will enable ``FATAL`` log level. + + +VLOG level is specified by numbers, e.g., 0, 1, 2. Environment variable ``MACE_CPP_MIN_VLOG_LEVEL`` can be set to specify vlog level. +Logs with higher levels than which is specified will be printed. So simply specifying a very large level number will make all logs printed. + +By using Mace run tool, vlog level can be easily set by option, e.g., + + .. code:: sh + + python tools/converter.py run --config /path/to/model.yml --vlog_level=2 + + +If models are run on android, you might need to use ``adb logcat`` to view logs. + + +Debug memory usage +-------------------------- +The simplest way to debug process memory usage is to use ``top`` command. With ``-H`` option, it can also show thread info. +For android, if you need more memory info, e.g., memory used of all categories, ``adb shell dumpsys meminfo`` will help. +By watching memory usage, you can check if memory usage meets expectations or if any leak happens. + + +Debug using GDB +-------------------------- +GDB can be used as the last resort, as it is powerful that it can trace stacks of your process. If you run models on android, +things may be a little bit complicated. + + .. code:: sh + + # push gdbserver to your phone + adb push $ANDROID_NDK_HOME/prebuilt/android-arm64/gdbserver/gdbserver /data/local/tmp/ + + + # set system env, pull system libs and bins to host + export SYSTEM_LIB=/path/to/android/system_lib + export SYSTEM_BIN=/path/to/android/system_bin + mkdir -p $SYSTEM_LIB + adb pull /system/lib/. $SYSTEM_LIB + mkdir -p $SYSTEM_BIN + adb pull /system/bin/. $SYSTEM_BIN + + + # Suppose ndk compiler used to compile Mace is of android-21 + export PLATFORMS_21_LIB=$ANDROID_NDK_HOME/platforms/android-21/arch-arm/usr/lib/ + + + # start gdbserver，make gdb listen to port 6000 + # adb shell /data/local/tmp/gdbserver :6000 /path/to/binary/on/phone/example_bin + adb shell LD_LIBRARY_PATH=/dir/to/dynamic/library/on/phone/ /data/local/tmp/gdbserver :6000 /data/local/tmp/mace_run/example_bin + # or attach a running process + adb shell /data/local/tmp/gdbserver :6000 --attach 8700 + # forward tcp port + adb forward tcp:6000 tcp:6000 + + + # use gdb on host to execute binary + $ANDROID_NDK_HOME/prebuilt/linux-x86_64/bin/gdb [/path/to/binary/on/host/example_bin] + + + # connect remote port after starting gdb command + target remote :6000 + + + # set lib path + set solib-search-path $SYSTEM_LIB:$SYSTEM_BIN:$PLATFORMS_21_LIB + + # then you can use it as host gdb, e.g., + bt + + + + + + + diff --git a/docs/index.rst b/docs/index.rst index 7545f2aa..c73aa1d3 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -39,6 +39,7 @@ The main documentation is organized into the following sections: development/contributing development/adding_a_new_op development/how_to_run_tests + development/how_to_debug development/memory_layout .. toctree:: diff --git a/docs/installation/env_requirement.rst b/docs/installation/env_requirement.rst index 82ae4730..b2cfb36e 100644 --- a/docs/installation/env_requirement.rst +++ b/docs/installation/env_requirement.rst @@ -18,25 +18,30 @@ Required dependencies * - Bazel - `bazel installation guide `__ - 0.13.0 - * - CMake - - apt-get install cmake - - >= 3.11.3 * - Jinja2 - - pip install -I jinja2==2.10 + - pip install jinja2==2.10 - 2.10 * - PyYaml - - pip install -I pyyaml==3.12 + - pip install pyyaml==3.12 - 3.12.0 * - sh - - pip install -I sh==1.12.14 + - pip install sh==1.12.14 - 1.12.14 * - Numpy - - pip install -I numpy==1.14.0 + - pip install numpy==1.14.0 - Required by model validation * - six - - pip install -I six==1.11.0 + - pip install six==1.11.0 - Required for Python 2 and 3 compatibility +For Bazel, install it following installation guide. For python dependencies, + + .. code:: sh + + pip install -U --user setup/requirements.txt + + + Optional dependencies --------------------- @@ -49,25 +54,35 @@ Optional dependencies * - Android NDK - `NDK installation guide `__ - Required by Android build, r15b, r15c, r16b, r17b + * - CMake + - apt-get install cmake + - >= 3.11.3 * - ADB - apt-get install android-tools-adb - Required by Android run, >= 1.0.32 * - TensorFlow - - pip install -I tensorflow==1.8.0 + - pip install tensorflow==1.8.0 - Required by TensorFlow model * - Docker - `docker installation guide `__ - Required by docker mode for Caffe model * - Scipy - - pip install -I scipy==1.0.0 + - pip install scipy==1.0.0 - Required by model validation * - FileLock - - pip install -I filelock==3.0.0 + - pip install filelock==3.0.0 - Required by run on Android * - ONNX - - pip install onnx + - pip install onnx==1.3.0 - Required by ONNX model +For python dependencies, + + .. code:: sh + + pip install -U --user setup/optionals.txt + + .. note:: - For Android build, `ANDROID_NDK_HOME` must be confifigured by using ``export ANDROID_NDK_HOME=/path/to/ndk`` diff --git a/setup/optionals.txt b/setup/optionals.txt new file mode 100644 index 00000000..94187950 --- /dev/null +++ b/setup/optionals.txt @@ -0,0 +1,4 @@ +tensorflow>=1.8.0 +scipy>=1.0.0 +filelock>=3.0.0 +onnx>=1.3.0 \ No newline at end of file diff --git a/setup/requirements.txt b/setup/requirements.txt new file mode 100644 index 00000000..a3250b40 --- /dev/null +++ b/setup/requirements.txt @@ -0,0 +1,6 @@ +python>=2.7.0 +jinja2>=2.10 +pyyaml>=3.12 +sh>=1.12.14 +numpy>=1.14.0 +six>=1.11.0 \ No newline at end of file -- GitLab