Merge pull request !462 from 黄伟锋/myMaster

Merge pull request !460 from zyhStack/zyh_mindinsight

Merge pull request !457 from LiHongzhang/npu_timeout

Merge pull request !459 from luopengting/lineage_log

Merge pull request !455 from yelihua/dev_profiler

Merge pull request !449 from LiHongzhang/npu_timeout

Merge pull request !453 from LiHongzhang/v060

!451 use forkserver multiprocess context to avoid forking child process with locked stream resource(eg stdout)
Merge pull request !451 from wenkai/wk5_fix_log_stream_flush_stuck

Merge pull request !450 from LiHongzhang/no_readable

use forkserver multiprocess context to avoid forking child process with locked stream resource(eg stdout)

Merge pull request !446 from luopengting/fix_lineage

!442 convert the object of `NAN`、｀-INF｀and ｀INF｀ to string so that browser can view these tensor data
Merge pull request !442 from wangshuide/wsd_handle_special_tensor

Merge pull request !447 from wangshuide/wsd_limit_tensor_count

Merge pull request !441 from yuximiao/yuximiao_profiler

Merge pull request !445 from wenkai/wk4_0721_ci_pickle_error_fix

!443 Optimize number of max processes used for computing and limit number of thread running load_data_in_thread()
Merge pull request !443 from wenkai/perf_opt2_0721

Optimize number of max processes used for computing and limit number of thread running load_data_in_thread()

For max processes used for computing, we need to make sure every summary directory has a process to load data. We also should not use too many processes to avoid system problems (eg. out of memory). So we calc the max processes cnt in _calc_default_max_processes_cnt

For _load_data_in_thread_wrapper, because self._load_data_in_thread() will create process pool when loading files, we can not afford to run multiple self._load_data_in_thread() simultaneously. So we use a lock to make sure that only one self._load_data_in_thread() is running.

!440 Change the mindinsight multiprocessing computing code to use a unified manager, add new  features
Merge pull request !440 from wenkai/pref_opt_0720_1cp1

1. delete original message in HistogramContainer
2. Wrap image content in ImageContainer

Merge pull request !444 from 冯学峰/myMaster02

Merge pull request !427 from LiHongzhang/hw_board

Main features:

1. Use the ComputingResourceManager to manage all computing workers.
2. Ensure fair worker number between summary directories at first. So every summary directories in cache will be loaded simultaneously.
3. When a summary directory is loaded, it's worker will be released, and other unfinished summary directories can use the released workers to speed up. This way we solved the slow worker problem.

Code changes:

1. Added computing_resource_mgr.py
2. Passed ComputingResourceManager instances instead of workers_count
3. Simplified the _load_single_fine() function a bit.

- collect_cpu
- collect_mem
- collect_npu

bug fix. convert the object of `NAN`、｀-INF｀and ｀INF｀ to string so that browser can show these tensor data.

Merge pull request !439 from 冯学峰/myMaster01

Merge pull request !437 from wangshuide/wsd_multiple_processes_for_file_parsing

1. To accelerate summary file parsing, multiple processes are used. As the first step to mindinsight parsing performance optimization, we only made changes to _load_single_file function.

2. This PR will imporve summary parsing throughput dramatically (about cpu_count times)

3. Changes are mainly about _load_single_file function

In the future, a more global concurrent computing framework is needed for mindinsight. See the gitee wiki doc for details.