'make clean' descends into ./Kbuild, but does not clean anything
since everything is added to no-clean-files.

There is no need to descend to ./Kbuild in the first place.
We can drop the no-clean-files assignment.

With this, there is no more user of no-clean-files. I will keep it
for a while to see whether a new user will appear.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

'make /' is just an alias for 'make ./'; this builds all objects of an
external module, but skips the modpost stage.

I am not a big fan of 'make /' since it looks as if it were touching
the root directory of the system. I like 'make ./' better.

I do not know how many people are using it, but let's show a hint if
it is used. Also, move it close to the external module rules since
this only makes sense for external modules.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

$(objtree)/Module.symvers is not required for descending into
sub-directories. It is needed for the modpost stage.

Move the Module.symvers check to the right place.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

A compilation -Wimplicit-fallthrough warning was enabled by commit
a035d552 ("Makefile: Globally enable fall-through warning")

Even though clang 10.0.0 does not currently support this warning without
a patch, clang currently does not support a value for this option.

  Link: https://bugs.llvm.org/show_bug.cgi?id=39382

The gcc default for this warning is 3 so removing the =3 has no effect
for gcc and enables the warning for patched versions of clang.

Also remove the =3 from an existing use in a parisc Makefile:
arch/parisc/math-emu/Makefile
Signed-off-by: NJoe Perches <joe@perches.com>
Reviewed-and-tested-by: NNathan Chancellor <natechancellor@gmail.com>
Cc: Gustavo A. R. Silva <gustavo@embeddedor.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Since commit ff9b45c5 ("kbuild: modpost: read modules.order instead
of $(MODVERDIR)/*.mod"), a module is no longer built in the following
pattern:

  [Makefile]
  subdir-y := some-module

  [some-module/Makefile]
  obj-m := some-module.o

You cannot write Makefile this way in upstream because modules.order is
not correctly generated. subdir-y is used to descend to a sub-directory
that builds tools, device trees, etc.

For external modules, the modules order does not matter. So, the
Makefile above was known to work.

I believe the Makefile should be re-written as follows:

  [Makefile]
  obj-m := some-module/

  [some-module/Makefile]
  obj-m := some-module.o

However, people will have no idea if their Makefile suddenly stops
working. In fact, I received questions from multiple people.

Show a warning for a while if obj-m is specified in a Makefile visited
by subdir-y or subdir-m.

I touched the %/ rule to avoid false-positive warnings for the single
target.

Cc: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Tom Stonecypher <thomas.edwardx.stonecypher@intel.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: NJan Kiszka <jan.kiszka@siemens.com>

I removed the single target %.ko in commit ff9b45c5 ("kbuild:
modpost: read modules.order instead of $(MODVERDIR)/*.mod") because
the modpost stage does not work reliably. For instance, the module
dependency, modversion, etc. do not work if we lack symbol information
from the other modules.

Yet, some people still want to build only one module in their interest,
and it may be still useful if it is used within those limitations.

Fixes: ff9b45c5 ("kbuild: modpost: read modules.order instead of $(MODVERDIR)/*.mod")
Reported-by: NDon Brace <don.brace@microsemi.com>
Reported-by: NArend Van Spriel <arend.vanspriel@broadcom.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Define and export OBJSIZE variable for "size" tool from binutils to be
used in architecture specific Makefiles (naming the variable just "SIZE"
would be too risky). In particular this tool is useful to perform checks
that early boot code is not using bss section (which might have not been
zeroed yet or intersects with initrd or other files boot loader might
have put right after the linux kernel).

Link: http://lkml.kernel.org/r/patch-1.thread-2257a1.git-188f5a3d81d5.your-ad-here.call-01565088755-ext-5120@work.hoursAcked-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: NVasily Gorbik <gor@linux.ibm.com>

CLANG_FLAGS is initialized by the following line:

  CLANG_FLAGS     := --target=$(notdir $(CROSS_COMPILE:%-=%))

..., which is run only when CROSS_COMPILE is set.

Some build targets (bindeb-pkg etc.) recurse to the top Makefile.

When you build the kernel with Clang but without CROSS_COMPILE,
the same compiler flags such as -no-integrated-as are accumulated
into CLANG_FLAGS.

If you run 'make CC=clang' and then 'make CC=clang bindeb-pkg',
Kbuild will recompile everything needlessly due to the build command
change.

Fix this by correctly initializing CLANG_FLAGS.

Fixes: 238bcbc4 ("kbuild: consolidate Clang compiler flags")
Cc: <stable@vger.kernel.org> # v5.0+
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: NNathan Chancellor <natechancellor@gmail.com>
Acked-by: NNick Desaulniers <ndesaulniers@google.com>

Now that all the fall-through warnings have been addressed in the
kernel, enable the fall-through warning globally.

Also, update the deprecated.rst file to include implicit fall-through
as 'deprecated' so people can be pointed to a single location for
justification.

Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Kees Cook <keescook@chromium.org>
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: NGustavo A. R. Silva <gustavo@embeddedor.com>

The gcc -fcf-protection=branch option is not compatible with
-mindirect-branch=thunk-extern. The latter is used when
CONFIG_RETPOLINE is selected, and this will fail to build with
a gcc which has -fcf-protection=branch enabled by default. Adding
-fcf-protection=none when building with retpoline enabled
prevents such build failures.
Signed-off-by: NSeth Forshee <seth.forshee@canonical.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Now that there is no rule for 'prepare1', it can go away.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

While descending directories, Kbuild produces objects for modules,
but do not link final *.ko files; it is done in the modpost.

To keep track of modules, Kbuild creates a *.mod file in $(MODVERDIR)
for every module it is building. Some post-processing steps read the
necessary information from *.mod files. This avoids descending into
directories again. This mechanism was introduced in 2003 or so.

Later, commit 551559e1 ("kbuild: implement modules.order") added
modules.order. So, we can simply read it out to know all the modules
with directory paths. This is easier than parsing the first line of
*.mod files.

$(MODVERDIR) has a flat directory structure, that is, *.mod files
are named only with base names. This is based on the assumption that
the module name is unique across the tree. This assumption is really
fragile.

Stephen Rothwell reported a race condition caused by a module name
conflict:

  https://lkml.org/lkml/2019/5/13/991

In parallel building, two different threads could write to the same
$(MODVERDIR)/*.mod simultaneously.

Non-unique module names are the source of all kind of troubles, hence
commit 3a48a919 ("kbuild: check uniqueness of module names")
introduced a new checker script.

However, it is still fragile in the build system point of view because
this race happens before scripts/modules-check.sh is invoked. If it
happens again, the modpost will emit unclear error messages.

To fix this issue completely, create *.mod with full directory path
so that two threads never attempt to write to the same file.

$(MODVERDIR) is no longer needed.

Since modules with directory paths are listed in modules.order, Kbuild
is still able to find *.mod files without additional descending.

I also killed cmd_secanalysis; scripts/mod/sumversion.c computes MD4 hash
for modules with MODULE_VERSION(). When CONFIG_DEBUG_SECTION_MISMATCH=y,
it occurs not only in the modpost stage, but also during directory
descending, where sumversion.c may parse stale *.mod files. It would emit
'No such file or directory' warning when an object consisting a module is
renamed, or when a single-obj module is turned into a multi-obj module or
vice versa.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: NNicolas Pitre <nico@fluxnic.net>

Towards the goal of removing MODVERDIR, read out modules.order to get
the list of modules to be processed. This is simpler than parsing *.mod
files in $(MODVERDIR).

For external modules, $(KBUILD_EXTMOD)/modules.order should be read.

I removed the single target %.ko from the top Makefile. To make sure
modpost works correctly, vmlinux and the other modules must be built.
You cannot build a particular .ko file alone.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Removing the 'kernel/' prefix will make our life easier because we can
simply do 'cat modules.order' to get all built modules with full paths.

Currently, we parse the first line of '*.mod' files in $(MODVERDIR).
Since we have duplicated functionality here, I plan to remove MODVERDIR
entirely.

In fact, modules.order is generated also for external modules in a
broken format. It adds the 'kernel/' prefix to the absolute path of
the module, like this:

  kernel//path/to/your/external/module/foo.ko

This is fine for now since modules.order is not used for external
modules. However, I want to sanitize the format everywhere towards
the goal of removing MODVERDIR.

We cannot change the format of installed module.{order,builtin}.
So, 'make modules_install' will add the 'kernel/' prefix while copying
them to $(MODLIB)/.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Currently, $(objtree)/modules.order is touched in two places.

In the 'prepare0' rule, scripts/Makefile.build creates an empty
modules.order while processing 'obj=.'

In the 'modules' rule, the top-level Makefile overwrites it with
the correct list of modules.

While this might be a good side-effect that modules.order is made
empty every time (probably this is not intended functionality),
I personally do not like this behavior.

Create modules.order only when it is sensible to do so.

This avoids creating the following pointless files:

  scripts/basic/modules.order
  scripts/dtc/modules.order
  scripts/gcc-plugins/modules.order
  scripts/genksyms/modules.order
  scripts/mod/modules.order
  scripts/modules.order
  scripts/selinux/genheaders/modules.order
  scripts/selinux/mdp/modules.order
  scripts/selinux/modules.order

Going forward, $(objtree)/modules.order lists the modules that
was built in the last successful build.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

It takes somewhat long time to generate these tag files.
Keep such precious files until we run 'make distclean'.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

As commit 1e022137 ("mips: vdso: drop unnecessary cc-ldoption")
explained, these flags are supported by the minimal required version
of binutils. They are supported by ld.lld too.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: NNathan Chancellor <natechancellor@gmail.com>
Tested-by: NNathan Chancellor <natechancellor@gmail.com>

When cross-compiling an out-of-tree build with an unclean source tree
directory, the build fails with:

  /path/to/kernel/source/tree is not clean, please run 'make mrproper'
  in the '/path/to/kernel/source/tree' directory.

However, doing so does not fix the problem, as "make mrproper" now
requires passing the target architecture to the make command, else it
won't remove $(srctree)/arch/$(SRCARCH)/include/generated.
"git ls-files -o" doesn't give a clue, as it doesn't list (empty)
directories, only files.

Improve usability by including the ARCH= option in the error output.

Fixes: a788b2ed ("kbuild: check arch/$(SRCARCH)/include/generated before out-of-tree build")
Signed-off-by: NGeert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

In old days, Kbuild always used an absolute path for $(srctree).

Since commit 890676c6 ("kbuild: Use relative path when building in
the source tree"), $(srctree) is '.' when O= was not passed from the
command line.

Yet, using absolute paths is useful in some cases even without O=, for
instance, to create a cscope file with absolute path tags.

'O=.' was known to work as a workaround to force Kbuild to use absolute
paths even when you are building in the source tree.

Since commit 25b146c5 ("kbuild: allow Kbuild to start from any
directory"), Kbuild is too clever to be tricked. Even if you pass 'O=.'
Kbuild notices you are building in the source tree, then use '.' for
$(srctree).

So, 'make O=. cscope' is no help to create absolute path tags.

We cannot force one or the other according to commit e93bc1a0
("Revert "kbuild: specify absolute paths for cscope""). Both of
relative path and absolute path have pros and cons.

This commit adds a new flag KBUILD_ABS_SRCTREE to allow users to
choose the absolute path for $(srctree).

'make KBUILD_ABS_SRCTREE=1 cscope' will work as a replacement of
'make O=. cscope'.
Reported-by: NPawan Gupta <pawan.kumar.gupta@linux.intel.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Commit 25b146c5 ("kbuild: allow Kbuild to start from any directory")
deprecated KBUILD_SRCTREE.

It is only used in tools/testing/selftest/ to distinguish out-of-tree
build. Replace it with a new boolean flag, building_out_of_srctree.

I also replaced the conditional ($(srctree),.) because the next commit
will allow an absolute path to be used for $(srctree) even when building
in the source tree.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Replace $(src) and $(obj) with $(srctree) and $(objtree), respectively.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

The headers in include/ are globally used in the kernel source tree
to provide common APIs. They are included from external modules, too.

It will be useful to make as many headers self-contained as possible
so that we do not have to rely on a specific include order.

There are more than 4000 headers in include/. In my rough analysis,
70% of them are already self-contained. With efforts, most of them
can be self-contained.

For now, we must exclude more than 1000 headers just because they
cannot be compiled as standalone units. I added them to header-test-.
The blacklist was mostly generated by a script, so the reason of the
breakage should be checked later.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: NJani Nikula <jani.nikula@intel.com>
Reviewed-by: NJoel Fernandes (Google) <joel@joelfernandes.org>

header-test-y does not work with headers in sub-directories.

For example, you may want to write a Makefile, like this:

include/linux/Kbuild:

  header-test-y += mtd/nand.h

This entry will create a wrapper include/linux/mtd/nand.hdrtest.c
with the following content:

  #include "mtd/nand.h"

To make this work, we need to add $(srctree)/include/linux to the
header search path. It would be tedious to add ccflags-y.

Instead, we could change the *.hdrtest.c rule to wrap:

  #include "nand.h"

This works for in-tree build since #include "..." searches in the
relative path from the header with this directive. For O=... build,
we need to add $(srctree)/include/linux/mtd to the header search path,
which will be even more tedious.

After all, I thought it would be handier to compile headers directly
without creating wrappers.

I added a new build rule to compile %.h into %.h.s

The target is %.h.s instead of %.h.o because it is slightly faster.
Also, as for GCC, an empty assembly is smaller than an empty object.

I wrote the build rule:

  $(CC) $(c_flags) -S -o $@ -x c /dev/null -include $<

instead of:

  $(CC) $(c_flags) -S -o $@ -x c $<

Both work fine with GCC, but the latter is bad for Clang.

This comes down to the difference in the -Wunused-function policy.
GCC does not warn about unused 'static inline' functions at all.
Clang does not warn about the ones in included headers, but does
about the ones in the source. So, we should handle headers as
headers, not as source files.

In fact, this has been hidden since commit abb2ea7d ("compiler,
clang: suppress warning for unused static inline functions"), but we
should not rely on that.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: NJani Nikula <jani.nikula@intel.com>
Tested-by: NJani Nikula <jani.nikula@intel.com>

Multiple people have suggested compile-testing UAPI headers to ensure
they can be really included from user-space. "make headers_check" is
obviously not enough to catch bugs, and we often leak unresolved
references to user-space.

Use the new header-test-y syntax to implement it. Please note exported
headers are compile-tested with a completely different set of compiler
flags. The header search path is set to $(objtree)/usr/include since
exported headers should not include unexported ones.

We use -std=gnu89 for the kernel space since the kernel code highly
depends on GNU extensions. On the other hand, UAPI headers should be
written in more standardized C, so they are compiled with -std=c90.
This will emit errors if C++ style comments, the keyword 'inline', etc.
are used. Please use C style comments (/* ... */), '__inline__', etc.
in UAPI headers.

There is additional compiler requirement to enable this test because
many of UAPI headers include <stdlib.h>, <sys/ioctl.h>, <sys/time.h>,
etc. directly or indirectly. You cannot use kernel.org pre-built
toolchains [1] since they lack <stdlib.h>.

I reused CONFIG_CC_CAN_LINK to check the system header availability.
The intention is slightly different, but a compiler that can link
userspace programs provide system headers.

For now, a lot of headers need to be excluded because they cannot
be compiled standalone, but this is a good start point.

[1] https://mirrors.edge.kernel.org/pub/tools/crosstool/index.htmlSigned-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: NSam Ravnborg <sam@ravnborg.org>

Commit 0126be38 ("kbuild: announce removal of SUBDIRS if used")
added a hint about the 'SUBDIRS' replacement, but it was not clear
enough.

Multiple people sent me similar questions, patches. For instance,

  https://lkml.org/lkml/2019/1/17/456

I did not mean to use M= for building a subdirectory in the kernel tree.

From commit 669efc76 ("net: hns3: fix compile error"), people
already (ab)use M=... to do that because it seems to work to some extent.

Documentation/kbuild/kbuild.txt says M= and KBUILD_EXTMOD are used for
building external modules.

In fact, Kbuild supports the single target '%/' for this purpose, but
this may not be noticed much.

Kindly add more hints.

Makefile:213: ================= WARNING ================
Makefile:214: 'SUBDIRS' will be removed after Linux 5.3
Makefile:215:
Makefile:216: If you are building an individual subdirectory
Makefile:217: in the kernel tree, you can do like this:
Makefile:218: $ make path/to/dir/you/want/to/build/
Makefile:219: (Do not forget the trailing slash)
Makefile:220:
Makefile:221: If you are building an external module,
Makefile:222: Please use 'M=' or 'KBUILD_EXTMOD' instead
Makefile:223: ==========================================
Suggested-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: NSam Ravnborg <sam@ravnborg.org>

There are some people interested in experimenting with Clang's
integrated assembler. To make it easy to do so without source
modification, allow the user to specify 'AS=clang' as part of the
make command to avoid adding '-no-integrated-as' to the {A,C}FLAGS.

Link: https://github.com/ClangBuiltLinux/linux/issues/577Suggested-by: NDmitry Golovin <dima@golovin.in>
Signed-off-by: NNathan Chancellor <natechancellor@gmail.com>
Reviewed-by: NNick Desaulniers <ndesaulniers@google.com>
Tested-by: NNick Desaulniers <ndesaulniers@google.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

modules.order is a real target. Split its build rule out like
modules.builtin
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Unlike modules.order, modules.builtin is not rebuilt every time.
Once modules.builtin is created, it will not be updated until
auto.conf or tristate.conf is changed.

So, it does not  notice a change in Makefile, for example, the rename
of modules.

Kbuild must always descend into directories for modules.builtin too.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

In commit ebcc5928 ("arm64: Silence gcc warnings about arch ABI
drift"), the arm64 Makefile added -Wno-psabi to KBUILD_CFLAGS, which is
a GCC only option so clang rightfully complains:

warning: unknown warning option '-Wno-psabi' [-Wunknown-warning-option]

https://clang.llvm.org/docs/DiagnosticsReference.html#wunknown-warning-option

However, by default, this is merely a warning so the build happily goes
on with a slew of these warnings in the process.

Commit c3f0d0bc ("kbuild, LLVMLinux: Add -Werror to cc-option to
support clang") worked around this behavior in cc-option by adding
-Werror so that unknown flags cause an error. However, this all happens
silently and when an unknown flag is added to the build unconditionally
like -Wno-psabi, cc-option will always fail because there is always an
unknown flag in the list of flags. This manifested as link time failures
in the arm64 libstub because -fno-stack-protector didn't get added to
KBUILD_CFLAGS.

To avoid these weird cryptic failures in the future, make clang behave
like gcc and immediately error when it encounters an unknown flag by
adding -Werror=unknown-warning-option to CLANG_FLAGS. This can be added
unconditionally for clang because it is supported by at least 3.0.0,
according to godbolt [1] and 4.0.0, according to its documentation [2],
which is far earlier than we typically support.

[1]: https://godbolt.org/z/7F7rm3
[2]: https://releases.llvm.org/4.0.0/tools/clang/docs/DiagnosticsReference.html#wunknown-warning-option

Link: https://github.com/ClangBuiltLinux/linux/issues/511
Link: https://github.com/ClangBuiltLinux/linux/issues/517Suggested-by: NPeter Smith <peter.smith@linaro.org>
Signed-off-by: NNathan Chancellor <natechancellor@gmail.com>
Tested-by: NNick Desaulniers <ndesaulniers@google.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Sometimes it's useful to be able to explicitly ensure certain headers
remain self-contained, i.e. that they are compilable as standalone
units, by including and/or forward declaring everything they depend on.

Add special target header-test-y where individual Makefiles can add
headers to be tested if CONFIG_HEADER_TEST is enabled. This will
generate a dummy C file per header that gets built as part of extra-y.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Reviewed-by: NSam Ravnborg <sam@ravnborg.org>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Now that hdr-inst is used only in the top Makefile, move it there
from scripts/Kbuild.include.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Since commit fcc8487d ("uapi: export all headers under uapi
directories"), the headers in uapi directories are all exported by
default although exceptional cases are still allowed by the syntax
'no-export-headers'.

The traditional directory descending has been kept (in a somewhat
hacky way), but it is actually unneeded.

Get rid of it to simplify the code.

Also, handle files one by one instead of the previous per-directory
processing. This will emit much more log, but I like it.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>