While timing a "no-op" build of the kernel (incrementally building the
kernel even though nothing changed) in the Chrome OS build system I
found that it was much slower than I expected.

Digging into things a bit, I found that quite a bit of the time was
spent invoking the C compiler even though we weren't actually building
anything.  Currently in the Chrome OS build system the C compiler is
called through a number of wrappers (one of which is written in
python!) and can take upwards of 100 ms to invoke even if we're not
doing anything difficult, so these invocations of the compiler were
taking a lot of time.  Worse the invocations couldn't seem to take
advantage of the multiple cores on my system.

Certainly it seems like we could make the compiler invocations in the
Chrome OS build system faster, but only to a point.  Inherently
invoking a program as big as a C compiler is a fairly heavy
operation.  Thus even if we can speed the compiler calls it made sense
to track down what was happening.

It turned out that all the compiler invocations were coming from
usages like this in the kernel's Makefile:

KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,)

Due to the way cc-option and similar statements work the above
contains an implicit call to the C compiler.  ...and due to the fact
that we're storing the result in KBUILD_CFLAGS, a simply expanded
variable, the call will happen every time the Makefile is parsed, even
if there are no users of KBUILD_CFLAGS.

Rather than redoing this computation every time, it makes a lot of
sense to cache the result of all of the Makefile's compiler calls just
like we do when we compile a ".c" file to a ".o" file.  Conceptually
this is quite a simple idea.  ...and since the calls to invoke the
compiler and similar tools are centrally located in the Kbuild.include
file this doesn't even need to be super invasive.

Implementing the cache in a simple-to-use and efficient way is not
quite as simple as it first sounds, though.  To get maximum speed we
really want the cache in a format that make can natively understand
and make doesn't really have an ability to load/parse files. ...but
make _can_ import other Makefiles, so the solution is to store the
cache in Makefile format.  This requires coming up with a valid/unique
Makefile variable name for each value to be cached, but that's
solvable with some cleverness.

After this change, we'll automatically create a ".cache.mk" file that
will contain our cached variables.  We'll load this on each invocation
of make and will avoid recomputing anything that's already in our
cache.  The cache is stored in a format that it shouldn't need any
invalidation since anything that might change should affect the "key"
and any old cached value won't be used.
Signed-off-by: NDouglas Anderson <dianders@chromium.org>
Tested-by: NIngo Molnar <mingo@kernel.org>
Tested-by: NGuenter Roeck <linux@roeck-us.net>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

When compiling with `make CC=clang HOSTCC=clang`, I was seeing warnings
that clang did not recognize -fno-delete-null-pointer-checks for HOSTCC
targets.  These were added in commit 61163efa ("kbuild: LLVMLinux:
Add Kbuild support for building kernel with Clang").

Clang does not support -fno-delete-null-pointer-checks, so adding it to
HOSTCFLAGS if HOSTCC is clang does not make sense.

It's not clear why the other warnings were disabled, and just for
HOSTCFLAGS, but I can remove them, add -Werror to HOSTCFLAGS and compile
with clang just fine.
Suggested-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: NNick Desaulniers <nick.desaulniers@gmail.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

The top Makefile is divided into some sections such as mixed targets,
config targets, build targets, etc.

When we build mixed targets, Kbuild just invokes submake to process
them one by one.  In this case, compiler-related variables like CC,
KBUILD_CFLAGS, etc. are unneeded.

Check what kind of targets we are building first, and parse variables
for building only when necessary.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

The first "_all" occurrence around line 120 is only visible when
KBUILD_SRC is unset.

If O=... is specified, the working directory is relocated, then the
only second occurrence around line 193 is visible, that is not set
to PHONY.

Move the first one to an always visible place.  This clarifies "_all"
is our default target and it is always set to PHONY.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: NDouglas Anderson <dianders@chromium.org>

Since commit 5e538790 ("sparc,sparc64: unify Makefile"), hdr-arch
and SRCARCH always match.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: NDouglas Anderson <dianders@chromium.org>

Since commit 1f2bfbd0 ("kbuild: link of vmlinux moved to a
script"), it is easy to increment .version without using a temporary
file .old_version.

I do not see anybody who creates the .tmp_version.  Probably it is a
left-over of commit 4e25d8bb ("[PATCH] kbuild: adjust .version
updating").  Just remove it.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

kselftest and kselftest-clean targets fail when object directory is
specified to relocate objects. Fix it so it can find the source tree
to build from.

make O=/tmp/kselftest_top kselftest

make[1]: Entering directory '/tmp/kselftest_top'
make[2]: Entering directory '/tmp/kselftest_top'
make[2]: *** tools/testing/selftests: No such file or directory.  Stop.
make[2]: Leaving directory '/tmp/kselftest_top'
./linux-kselftest/Makefile:1185: recipe for target
'kselftest' failed
make[1]: *** [kselftest] Error 2
make[1]: Leaving directory '/tmp/kselftest_top'
Makefile:145: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>
Acked-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Commit 5620a0d1 ("firmware: delete in-kernel firmware") removed the
entire firmware directory.  Unfortunately it thereby also removed the
support for built-in firmware.

This restores the ability to build firmware directly into the kernel by
pruning the original Makefile to the necessary minimum.  The default for
EXTRA_FIRMWARE_DIR is now the standard directory /lib/firmware/.

Fixes: 5620a0d1 ("firmware: delete in-kernel firmware")
Signed-off-by: NMarkus Trippelsdorf <markus@trippelsdorf.de>
Acked-by: NGreg K-H <gregkh@linuxfoundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The last firmware change for the in-kernel firmware source code was back
in 2013.  Everyone has been relying on the out-of-tree linux-firmware
package for a long long time.

So let's drop it, it's baggage we don't need to keep dragging around
(and having to fix random kbuild issues over time...)

Cc: Kyle McMartin <kyle@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Michal Marek <mmarek@suse.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: NDavid Woodhouse <dwmw2@infradead.org>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Kbuild conventionally uses $(shell cd ... && /bin/pwd) idiom to get
the absolute path of the directory because GNU Make 3.80, the minimal
supported version at that time, did not support $(abspath ...) or
$(realpath ...).

Commit 37d69ee3 ("docs: bump minimal GNU Make version to 3.81")
dropped the GNU Make 3.80 support, so we are now allowed to use those
make-builtin helpers.

This conversion will provide better portability without relying on
the pwd command or its location /bin/pwd.

I am intentionally using $(realpath ...) instead $(abspath ...) in
some places.  The difference between the two is $(realpath ...)
returns an empty string if the given path does not exist.  It is
convenient in places where we need to error-out if the makefile fails
to create an output directory.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: NThierry Reding <treding@nvidia.com>

kselftest-clean isn't in the PHONY target list. Add it.
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Commit 971a69db ("Xen: don't warn about 2-byte wchar_t in efi")
added the --no-wchar-size-warning to the Makefile to avoid this
harmless warning:

arm-linux-gnueabi-ld: warning: drivers/xen/efi.o uses 2-byte wchar_t yet the output is to use 4-byte wchar_t; use of wchar_t values across objects may fail

Changing kbuild to use thin archives instead of recursive linking
unfortunately brings the same warning back during the final link.

The kernel does not use wchar_t string literals at this point, and
xen does not use wchar_t at all (only efi_char16_t), so the flag
has no effect, but as pointed out by Jan Beulich, adding a wchar_t
string literal would be bad here.

Since wchar_t is always defined as u16, independent of the toolchain
default, always passing -fshort-wchar is correct and lets us
remove the Xen specific hack along with fixing the warning.

Link: https://patchwork.kernel.org/patch/9275217/
Fixes: 971a69db ("Xen: don't warn about 2-byte wchar_t in efi")
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NDavid Vrabel <david.vrabel@citrix.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

This is a bunch of trivial fixes and cleanups.
Signed-off-by: NCao jin <caoj.fnst@cn.fujitsu.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

After removal of DocBook, those targets are bogus.
Reported-by: NJim Davis <jim.epost@gmail.com>
Signed-off-by: NMauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

I made the mistake of upgrading my desktop to the new Fedora 26 that
comes with gcc-7.1.1.

There's nothing wrong per se that I've noticed, but I now have 1500
lines of warnings, mostly from the new format-truncation warning
triggering all over the tree.

We use 'snprintf()' and friends in a lot of places, and often know that
the numbers are fairly small (ie a controller index or similar), but gcc
doesn't know that, and sees an 'int', and thinks that it could be some
huge number.  And then complains when our buffers are not able to fit
the name for the ten millionth controller.

These warnings aren't necessarily bad per se, and we probably want to
look through them subsystem by subsystem, but at least during the merge
window they just mean that I can't even see if somebody is introducing
any *real* problems when I pull.

So warnings disabled for now.
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This fixes the following build error for me when building on an 32 bit
machine using an XFS file system:

	$ make scripts/basic/fixdep
	  HOSTCC  scripts/basic/fixdep
	fixdep: error fstat'ing depfile: scripts/basic/.fixdep.d: Value too large for defined data type
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Acked-by: NBen Hutchings <ben@decadent.org.uk>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

When we install headers, we are interested only in headers under uapi
directories.  Split out uapi-asm-generic target and make headers_install
depend on it.  It will avoid generating unneeded asm-generic wrappers.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

We can always pass dst= from the top Makefile.  This will simplify
the logic in Makefile.headersinst.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Commit 61562f98 ("uapi: export all arch specifics directories")
changed the dst from asm-<arch> to arch-<arch> for headers_install_all
or headers_check_all.  Update the comment.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Original comments is confusing on "OBJ directory", make it clear.
Bonus: move comments close to what it wants to comment.
Signed-off-by: NCao jin <caoj.fnst@cn.fujitsu.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

The thin archives build currently puts all lib.a and built-in.o
files together and links them with --whole-archive.

This works because thin archives can recursively refer to thin
archives. However some architectures include libgcc.a, which may
not be a thin archive, or it may not be constructed with the "P"
option, in which case its contents do not get linked correctly.

So don't pull .a libs into the root built-in.o archive. These
libs should already have symbol tables and indexes built, so they
can be direct linker inputs. Move them out of the --whole-archive
option, which restore the conditional linking behaviour of lib.a
to thin archives builds.
Signed-off-by: NNicholas Piggin <npiggin@gmail.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

cc-option uses KBUILD_CFLAGS and KBUILD_CPPFLAGS when it determines
whether an option is supported or not. This is fine for options used to
build the kernel itself, however some components like the x86 boot code
use a different set of flags.

Add the new macro __cc-option which is a more generic version of
cc-option with additional parameters. One parameter is the compiler
with which the check should be performed, the other the compiler options
to be used instead KBUILD_C*FLAGS.

Refactor cc-option and hostcc-option to use __cc-option and move
hostcc-option to scripts/Kbuild.include.
Suggested-by: NArnd Bergmann <arnd@arndb.de>
Suggested-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: NMatthias Kaehlcke <mka@chromium.org>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NMichal Marek <mmarek@suse.com>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

There were a few bits and pieces left over from the now-disused DocBook
toolchain; git rid of them.
Reported-by: NMarkus Heiser <markus.heiser@darmarit.de>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

clang generates plenty of these warnings in different parts of the code,
to an extent that the warnings are little more than noise. Disable the
'address-of-packed-member' warning.
Signed-off-by: NMatthias Kaehlcke <mka@chromium.org>
Reviewed-by: NDouglas Anderson <dianders@chromium.org>
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

Commit 90ac086b ("Makefile: include arch/*/include/generated/uapi
before .../generated") introduced this for bisect'ability.  The commit
chose to promote arch/*/include/generated/uapi in the search path
rather than cleaning stale headers.

After all, we found that approach was not enough, and ended up with
cleaning stale headers by commit cda2c65f ("kbuild: Remove stale
asm-generic wrappers").

So, the extra search path is no longer needed because Kbuild invokes
scripts/Makefile.asm-generic and remove stale headers before it starts
descending.

This commit is also reverting commit dc33db7c ("Kbuild: avoid
duplicate include path") because we have no more duplicated path.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>