[ Upstream commit f98e8a572bddbf27032114127d2fcc78fa5e6a9d ]

When the fixed factor clock is created by devicetree,
of_clk_add_provider is called.  Add a call to
of_clk_del_provider in the remove function to balance
it out.
Reported-by: NAlan Tull <atull@kernel.org>
Fixes: 971451b3 ("clk: fixed-factor: Convert into a module platform driver")
Signed-off-by: NRicardo Ribalda Delgado <ricardo.ribalda@gmail.com>
Signed-off-by: NStephen Boyd <sboyd@kernel.org>
Signed-off-by: NSasha Levin <sashal@kernel.org>