refactor(doc): refactor and simplify script

GitOrigin-RevId: 3cdc2d677ff7a6c56062e20361c36d8f8af1fdcd

README.md

@@ -11,6 +11,25 @@
 
     For more information about RST, please visit https://sphinx-doc-zh.readthedocs.io/en/latest/rest.html.
 
+- Install doxygen and exhale for C++ doc building
+
+    Make sure you have installed necessary build tools (i.e. g++, python, cmake, flex, bison)
+
+    Install doxygen: 
+    ```
+    git clone https://github.com/doxygen/doxygen.git
+    cd doxygen
+    mkdir build
+    cd build
+    cmake -G "Unix Makefiles" ..
+    make
+    make install
+    ```
+    Install exhale:
+    ```
+    pip install exhale
+    ```
+
 ## Generate API document
 
 1. Make sure you have installed [MegEngine](https://github.com/MegEngine/MegEngine).
@@ -25,11 +44,11 @@
     git clone git@git-core.megvii-inc.com:brain-sdk/MegBrain.git
     ```
 
-3. Run [gen_python_docs/gen_html.sh](gen_python_docs/gendoc.sh) to generate HTML files.
+3. Run [gen_docs/entry.sh](gen_docs/entry.sh) to generate HTML files.
     The script accepts the MegEngine installation and MegBrain clone path as the argument.
 
     ```bash
-    ./gen_python_docs/gen_html.sh $MGB_ROOT $MGE_ROOT(optional)
+    ./gen_docs/entry.sh $MGB_ROOT $MGE_ROOT(optional)
     ```
 
     Note that the RST files generated from python docstring are put under `source/autogen`.
@@ -46,16 +65,16 @@
     2. Run sphinx tool to generate RST files from python docstring.
     3. Generate HTML files from RST.
 
-    Refer to [gen_python_docs/gendoc.sh](gen_python_docs/gendoc.sh) for more details.
+    Refer to [gen_docs/build.sh](gen_docs/build.sh) for more details.
 
-* Example python docstring: see [gen_python_docs/example/example.py](gen_python_docs/example/example.py).
+* Example python docstring: see [gen_docs/example/example.py](gen_docs/example/example.py).
 
 ## Run doctest in API document
 
 API docstring also contains examples written by [doctest](https://docs.python.org/3/library/doctest.html). Run the tests by
 
 ```bash
-gen_python_docs/gen_html.sh $MGB_ROOT ~/.local/lib/python3.6/site-packages/megengine
+gen_docs/entry.sh $MGB_ROOT $MGE_ROOT(optional)
 sphinx-build -b doctest source build/doctest
 ```
 
@@ -82,9 +101,12 @@ find the class rst file and copy its name and replace the doc with
 :ref:`exhale_class_<filename without .rst>`
 ```
 
-2. For file referencing；
+2. For file referencing:
 
 find the file and copy its name and replace the doc with
 ```
 :ref:`file_file_<filename>`
 ```
+
+## Process of generate document
+!["entry.sh process"](source/entry.png)
\ No newline at end of file

gen_python_docs/gendoc.sh → gen_docs/build.sh

@@ -27,8 +27,4 @@ rm -f $AUTOGEN/modules.rst
 
 cd ..
 
-if [[ ! -n $BUILD_LANG ]]; then
-    sphinx-build -j$(nproc) source build/html
+sphinx-build ${BUILDLANG} -j$(nproc) source build/html
\ No newline at end of file
-else
-    sphinx-build -D language="zh_CN" -j$(nproc) source build/html
-fi

gen_python_docs/gen_html.sh → gen_docs/entry.sh

@@ -8,23 +8,6 @@ rm -rf source/cpp_api
 
 set -e
 
-if [ ! -n "$1" ]
-then
-    echo "MegBrain directory not provided"
-    exit 1
-else
-    MGB_ROOT=$1
-fi
-
-exec 3<"source/include/h_file.txt"
-exec 4<"source/include/h_location.txt"
-
-while read line1<&3 && read line2<&4
-do
-    mkdir -p "source/include/file${line2%/*}"
-    cp ${MGB_ROOT}${line1} "source/include/file${line2%/*}"
-done
-
 if [ ! -n "$2" ]; then
     MGE_ROOT=`python3 -c "import os; \
                           import megengine; \
@@ -33,18 +16,30 @@ else
     MGE_ROOT=$2
 fi
 
-./gen_python_docs/gendoc.sh $MGE_ROOT
+# generate English python document
+export BUILDLANG="-D language=en_US"
+./gen_docs/build.sh $MGE_ROOT
 
-if [[ ! -f .tmp ]]
-then
-    mkdir .tmp
-fi
+rm -rf en_python_doc
+mkdir en_python_doc
+cp -r build/html/api/* en_python_doc
+
+# copy MegBrain source file to source/include
+MGB_ROOT="$1"
+cat source/include/h_file.txt | while read line; do
+    line2=${line#*include}
+    mkdir -p "source/include/file${line2%/*}"
+    cp ${MGB_ROOT}${line} "source/include/file${line2%/*}"
+done
 
-cp -r build/html/api/* .tmp/
-export BUILD_LANG="zh_CN"
 export API_DIR="api_zh"
-./gen_python_docs/gendoc.sh $MGE_ROOT
+
+# generate Chinese python document, tutorial, and English c++ document
+export BUILDLANG="-D language=zh_CN"
+./gen_docs/build.sh $MGE_ROOT
+
+# copy and replace English python document with previous one 
 rm -rf build/html/api
 mkdir build/html/api
-cp .tmp/* build/html/api
-rm -rf .tmp
+cp en_python_doc/* build/html/api
+rm -rf en_python_doc

source/conf.py

@@ -53,7 +53,6 @@ exhale_args = {
     "rootFileName":          "library_root.rst",
     "rootFileTitle":         "C++ API",
     "doxygenStripFromPath":  "include",
-    # "doxygenStripFromPath":  "../../../MegBrain/src",
     # Suggested optional arguments
     "createTreeView":        True,
     # TIP: if using the sphinx-bootstrap-theme, you need
@@ -61,8 +60,6 @@ exhale_args = {
     "exhaleExecutesDoxygen": True,
     "exhaleDoxygenStdin": textwrap.dedent('''
         INPUT      = include
-        # Using `=` instead of `+=` overrides
-        PREDEFINED = FOO="12"
     '''),
     # "createTreeView": True,
     "verboseBuild": True,

source/entry.dot

+digraph entry{
+
+    subgraph cluster_1{
+        label = "entry.sh"
+        "build.sh(with language = English)"->"copy Megbrain files to source/include";
+        "copy Megbrain files to source/include"->"build.sh(with language = Chinese)";
+    }
+
+    subgraph cluster_2{
+        label = "output documents";
+        "English documents";
+        "C++ document";
+        "Chinese python document";
+        "Tutorial";
+    }
+
+    subgraph cluster_3{
+        label = "discard";
+        "English python document(with bug)";
+    }
+
+    "build.sh(with language = English)"->"English documents";
+    "build.sh(with language = Chinese)"->"English python document(with bug)";
+    "build.sh(with language = Chinese)"->"C++ document";
+    "build.sh(with language = Chinese)"->"Chinese python document";
+    "build.sh(with language = Chinese)"->"Tutorial";
+    "English documents"->"final document";
+    "C++ document"->"final document";
+    "Chinese python document"->"final document";
+    "Tutorial"->"final document";
+}

source/include/h_location.txt

-/megbrain/serialization/load_dump_config.h
-/megbrain/serialization/serializer.h
-/megbrain/serialization/file.h
-/megbrain/plugin/*
-/megbrain/gopt/inference.h
-/megbrain/comp_node.h
-/megbrain/dtype.h
-/megbrain/exception.h
-/megbrain/graph/cg.h
-/megbrain/graph/operator_node.h
-/megbrain/graph/event.h
-/megbrain/graph/symbol_var.h
-/megbrain/graph/var_node.h
-/megbrain/tensor.h
-/megbrain/utils/debug.h