!749 Add more tutorial for Profiler.

Merge pull request !749 from yuximiao/master

tutorials/source_en/advanced_use/performance_profiling.md

@@ -29,6 +29,10 @@ Performance data like operators' execution time is recorded in files and can be
 - Start MindInsight and specify the profiler data directory using startup parameters. After MindInsight is started, access the visualization page based on the IP address and port number. The default access IP address is `http://127.0.0.1:8080`.
 - Find the training in the list, click the performance profiling link, and view the data on the web page.
 
+## Preparing the Environment
+
+Before using Profiler, make sure the process of ada in background running right. The ada process must using the root user to run. The start command is `/usr/local/Ascend/driver/tools/ada`.
+
 ## Preparing the Training Script
 
 To enable the performance profiling of neural networks, MindSpore Profiler APIs should be added into the script. At first, the MindSpore `Profiler` object need to be set after set context and before the network and HCCL initialization. Then, at the end of the training, `Profiler.analyse()` should be called to finish profiling and generate the perforamnce analyse results.
@@ -183,4 +187,8 @@ W/A/S/D can be applied to zoom in and out of the Timeline graph.
 ## Specifications
 
 - To limit the data size generated by the Profiler, MindInsight suggests that for large neural network, the profiled steps should better below 10.
+
+  > How to limit step count, Please refer to data preparation tutorial:
+  > <https://www.mindspore.cn/tutorial/en/master/use/data_preparation/data_preparation.html>
+
 - The parse of Timeline data is time consuming, and several step's data is usually enough for analysis. In order to speed up the data parse and UI display, Profiler will show at most 20M data (Contain 10+ step information for large networks).

tutorials/source_en/advanced_use/performance_profiling_gpu.md

@@ -23,7 +23,7 @@ Performance data like operators' execution time is recorded in files and can be
 
 > The GPU operation process is the same as that in Ascend chip.
 >
-> https://www.mindspore.cn/tutorial/en/master/advanced_use/performance_profiling.html#id3
+> <https://www.mindspore.cn/tutorial/en/master/advanced_use/performance_profiling.html#id3>
 
 ## Preparing the Training Script
 
@@ -31,7 +31,37 @@ To enable the performance profiling of neural networks, MindSpore Profiler APIs
 
 > The sample code is the same as that in Ascend chip：
 >
-> https://www.mindspore.cn/tutorial/en/master/advanced_use/performance_profiling.html#id4
+> <https://www.mindspore.cn/tutorial/en/master/advanced_use/performance_profiling.html#id4>
+
+Users can get profiling data by user-defined callback:
+
+```python
+class StopAtStep(Callback):
+    def __init__(self, start_step, stop_step):
+        super(StopAtStep, self).__init__()
+        self.start_step = start_step
+        self.stop_step = stop_step
+        self.already_analysed = False
+        
+    def step_begin(self, run_context):
+        cb_params = run_context.original_args()
+        step_num = cb_params.cur_step_num
+        if step_num == self.start_step:
+            self.profiler = Profiler()
+
+    def step_end(self, run_context):
+        cb_params = run_context.original_args()
+        step_num = cb_params.cur_step_num
+        if step_num == self.stop_step and not self.already_analysed:
+            self.profiler.analyse()
+            self.already_analysed = True
+            
+    def end(self, run_context):
+        if not self.already_analysed:
+            self.profiler.analyse()
+```
+
+The code above is just a example. Users should implement callback by themselves.
 
 ## Launch MindInsight
 

tutorials/source_zh_cn/advanced_use/performance_profiling.md

@@ -29,6 +29,9 @@
 - 启动MindInsight，并通过启动参数指定Profiler文件目录，启动成功后，根据IP和端口访问可视化界面，默认访问地址为 `http://127.0.0.1:8080`。
 - 在训练列表找到对应训练，点击性能分析，即可在页面中查看训练性能数据。
 
+## 环境准备
+在使用性能分析工具之前，要确保后台工具进程（ada）正确启动，要求用户使用root启动ada进程，启动命令为：`/usr/local/Ascend/driver/tools/ada`。
+
 ## 准备训练脚本
 
 为了收集神经网络的性能数据，需要在训练脚本中添加MindSpore Profiler相关接口。  
@@ -183,4 +186,9 @@ Timeline主要包含如下几个部分：
 ## 规格
 
 - 为了控制性能测试时生成数据的大小，大型网络建议性能调试的step数目限制在10以内。
+
+  > 如何控制step数目请参考数据准备教程：
+  >
+  > <https://www.mindspore.cn/tutorial/zh-CN/master/use/data_preparation/data_preparation.html>
+
 - Timeline数据的解析比较耗时，且一般几个step的数据即足够分析出结果。出于数据解析和UI展示性能的考虑，Profiler最多展示20M数据（对大型网络20M可以显示10+条step的信息）。

tutorials/source_zh_cn/advanced_use/performance_profiling_gpu.md

@@ -23,7 +23,7 @@
 
 > 操作流程可以参考Ascend 910上profiler的操作：
 >
-> https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/performance_profiling.html#id3
+> <https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/performance_profiling.html#id3>
 
 ## 准备训练脚本
 
@@ -33,7 +33,37 @@
 
 > 样例代码与Ascend使用方式一致可以参考：
 >
-> https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/performance_profiling.html#id4
+> <https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/performance_profiling.html#id4>
+
+GPU场景下还可以用自定义callback的方式收集性能数据，示例如下：
+
+```python
+class StopAtStep(Callback):
+    def __init__(self, start_step, stop_step):
+        super(StopAtStep, self).__init__()
+        self.start_step = start_step
+        self.stop_step = stop_step
+        self.already_analysed = False
+        
+    def step_begin(self, run_context):
+        cb_params = run_context.original_args()
+        step_num = cb_params.cur_step_num
+        if step_num == self.start_step:
+            self.profiler = Profiler()
+
+    def step_end(self, run_context):
+        cb_params = run_context.original_args()
+        step_num = cb_params.cur_step_num
+        if step_num == self.stop_step and not self.already_analysed:
+            self.profiler.analyse()
+            self.already_analysed = True
+            
+    def end(self, run_context):
+        if not self.already_analysed:
+            self.profiler.analyse()
+```
+
+以上代码仅供参考，用户可根据所需场景自由实现。
 
 ## 启动MindInsight