This way the scheduler doesn't wait in it's work thread any more.

v2: fix race conditions
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>