Merge branch 'doc' into 'master'

Complement HOW-TO-DEBUG docs

See merge request !941

docs/conf.py

@@ -25,3 +25,10 @@ html_theme = "sphinx_rtd_theme"
 html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
 html_static_path = ['_static']
 smartquotes = False
+
+extensions = [
+    'sphinx.ext.autodoc',
+    'sphinx.ext.viewcode',
+    'sphinx.ext.todo',
+    'sphinx.ext.mathjax',
+]

docs/development/how_to_debug.rst

 How to debug
-==============
+==========================
 
-Log debug info
+Debug correctness
+--------------------------
+
+MACE provides tools to examine correctness of model execution by comparing model's output of MACE with output of training platform (e.g., Tensorflow, Caffe).
+Three metrics are used as comparison results:
+
+* **Cosine Similarity**:
+
+.. math::
+
+	Cosine\ Similarity = \frac{X \cdot X'}{\|X\| \|X'\|}
+
+This metric will be approximately equal to 1 if output is correct.
+
+* **SQNR** (Signal-to-Quantization-Noise Ratio):
+
+.. math::
+
+	SQNR = \frac{P_{signal}}{P_{noise}} = \frac{\|X\|^2}{\|X - X'\|^2}
+
+It is usually used to measure quantization accuracy. The higher SQNR is, the better accuracy will be.
+
+* **Pixel Accuracy**:
+
+.. math::
+
+   Pixel\ Accuracy = \frac{\sum^{batch}_{b=1} equal(\mathrm{argmax} X_b, \mathrm{argmax} X'_b)}{batch}
+
+It is usually used to measure classification accuracy. The higher the better.
+
+where :math:`X` is expected output (from training platform) whereas :math:`X'` is actual output (from MACE) .
+
+
+MACE automatically validate these metrics by running models with synthetic inputs.
+If you want to specify input data to use, you can add an option in yaml config under 'subgraphs', e.g.,
+
+.. code:: yaml
+
+	models:
+	  mobilenet_v1:
+	    platform: tensorflow
+	    model_file_path: https://cnbj1.fds.api.xiaomi.com/mace/miai-models/mobilenet-v1/mobilenet-v1-1.0.pb
+	    model_sha256_checksum: 71b10f540ece33c49a7b51f5d4095fc9bd78ce46ebf0300487b2ee23d71294e6
+	    subgraphs:
+	      - input_tensors:
+	          - input
+	        input_shapes:
+	          - 1,224,224,3
+	        output_tensors:
+	          - MobilenetV1/Predictions/Reshape_1
+	        output_shapes:
+	          - 1,1001
+	        validation_inputs_data:
+	          - https://cnbj1.fds.api.xiaomi.com/mace/inputs/dog.npy
+
+
+If model's output is suspected to be incorrect, it might be useful to debug your model layer by layer by specifying an intermediate layer as output,
+or use binary search method until suspicious layer is found.
+
+
+Debug memory usage
+--------------------------
+The simplest way to debug process memory usage is to use ``top`` command. With ``-H`` option, it can also show thread info.
+For android, if you need more memory info, e.g., memory used of all categories, ``adb shell dumpsys meminfo`` will help.
+By watching memory usage, you can check if memory usage meets expectations or if any leak happens.
+
+
+Debug performance
+--------------------------
+Using MACE, you can benchmark a model by examining each layer's duration as well as total duration. Or you can benchmark a single op.
+The detailed information is in :doc:`../user_guide/benchmark`.
+
+
+Debug model conversion
+--------------------------
+After model is converted to MACE model, a literal model graph is generated in directory `mace/codegen/models/your_model`.
+You can refer to it when debugging model conversion.
+
+
+Debug engine using log
 --------------------------
 Mace defines two sorts of logs: one is for users (LOG), the other is for developers (VLOG).
 
@@ -23,14 +102,7 @@ By using Mace run tool, vlog level can be easily set by option, e.g.,
 If models are run on android, you might need to use ``adb logcat`` to view logs.
 
 
-Debug memory usage
---------------------------
-The simplest way to debug process memory usage is to use ``top`` command. With ``-H`` option, it can also show thread info.
-For android, if you need more memory info, e.g., memory used of all categories, ``adb shell dumpsys meminfo`` will help.
-By watching memory usage, you can check if memory usage meets expectations or if any leak happens.
-
-
-Debug using GDB
+Debug engine using GDB
 --------------------------
 GDB can be used as the last resort, as it is powerful that it can trace stacks of your process. If you run models on android,
 things may be a little bit complicated.