Commit f566e1fb ("kbuild: make multiple directory targets work")
broke single target builds for external modules. Fix this.

Fixes: f566e1fb ("kbuild: make multiple directory targets work")
Reported-by: NBjørn Mork <bjorn@mork.no>
Signed-off-by: NMasahiro Yamada <masahiroy@kernel.org>
Tested-by: NBjørn Mork <bjorn@mork.no>