1. 21 7月, 2020 1 次提交
    • W
      Refactor the mindinsight multiprocessing computing code to use a unified manager. · 26fabf47
      wenkai 提交于
      Main features:
      
      1. Use the ComputingResourceManager to manage all computing workers.
      2. Ensure fair worker number between summary directories at first. So every summary directories in cache will be loaded simultaneously.
      3. When a summary directory is loaded, it's worker will be released, and other unfinished summary directories can use the released workers to speed up. This way we solved the slow worker problem.
      
      Code changes:
      
      1. Added computing_resource_mgr.py
      2. Passed ComputingResourceManager instances instead of workers_count
      3. Simplified the _load_single_fine() function a bit.
      26fabf47
  2. 20 7月, 2020 2 次提交
    • M
      !437 Use multiple processes to calc events · 008bdf56
      mindspore-ci-bot 提交于
      Merge pull request !437 from wangshuide/wsd_multiple_processes_for_file_parsing
      008bdf56
    • W
      Use multiple processes to calc events. · 7877f33b
      wangshuide2020 提交于
      1. To accelerate summary file parsing, multiple processes are used. As the first step to mindinsight parsing performance optimization, we only made changes to _load_single_file function.
      
      2. This PR will imporve summary parsing throughput dramatically (about cpu_count times)
      
      3. Changes are mainly about _load_single_file function
      
      In the future, a more global concurrent computing framework is needed for mindinsight. See the gitee wiki doc for details.
      7877f33b
  3. 18 7月, 2020 4 次提交
  4. 17 7月, 2020 9 次提交
  5. 16 7月, 2020 8 次提交
  6. 15 7月, 2020 3 次提交
  7. 14 7月, 2020 5 次提交
  8. 13 7月, 2020 2 次提交
  9. 10 7月, 2020 6 次提交