Created by: heavengate
PR types
Bug fixes
PR changes
APIs
Describe
fix DataLoader performance decrease in some reader intensive model by #25558.
The _send_idx and _rcvd_idx checking should be only in the IterableDataset case, Map-Style dataset may enter this while loop in the last few samples(when some of subprocess exit and others not) and cost extra time.
In CI unittest, the time cost in while loop may lead main process check subprocess status error and exit
| Model | original Speed | #25558 Speed | This PR Speed |
|---|---|---|---|
| MobileNetV1 | 1047.742 img/s | 913.665 img/s | 1030.650 img/s |
| MobileNetV2 | 816.83 img/s | 574.08 img/s | 808.304 img/s |