We should be disabling clocks when wake from USB is not needed. Since
this wasn't done, we had a clock imbalance since clocks were always
being enabled on resume.

Fixes: ae532b2b ("phy: usb: Add "wake on" functionality for newer Synopsis XHCI controllers")
Fixes: b0c0b66c ("phy: usb: Add support for wake and USB low power mode for 7211 S2/S5")
Signed-off-by: NJustin Chen <justinpopo6@gmail.com>
Acked-by: NFlorian Fainelli <f.fainelli@gmail.com>
Link: https://lore.kernel.org/r/1665005418-15807-7-git-send-email-justinpopo6@gmail.comSigned-off-by: NVinod Koul <vkoul@kernel.org>