In commit a8ba798b ("selftests: enable O and KBUILD_OUTPUT"), added
support to generate compile targets in a user specified directory. OUTPUT
variable controls the location which is undefined when tests are built in
the test directory or with "make -C tools/testing/selftests/x86".

make -C tools/testing/selftests/x86/
make: Entering directory '/lkml/linux_4.11/tools/testing/selftests/x86'
Makefile:44: warning: overriding recipe for target 'clean'
../lib.mk:51: warning: ignoring old recipe for target 'clean'
gcc -m64 -o /single_step_syscall_64 -O2 -g -std=gnu99 -pthread -Wall  single_step_syscall.c -lrt -ldl
/usr/bin/ld: cannot open output file /single_step_syscall_64: Permission denied
collect2: error: ld returned 1 exit status
Makefile:50: recipe for target '/single_step_syscall_64' failed
make: *** [/single_step_syscall_64] Error 1
make: Leaving directory '/lkml/linux_4.11/tools/testing/selftests/x86'

Same failure with "cd tools/testing/selftests/x86/;make" run.

Fix this with a change to lib.mk to define OUTPUT to be the pwd when
MAKELEVEL is 0. This covers both cases mentioned above.
Reported-by: NIngo Molnar <mingo@kernel.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

Both these rules incorrectly use $< (first prerequisite) rather than
$^ (all prerequisites), meaning they don't work if we're using more than
one .S file as input. Switch them to using $^.

They also don't include $(CPPFLAGS) and other variables used in the
default rules, which breaks targets that require those. Fix that by
using the builtin $(COMPILE.S) and $(LINK.S) rules.

Fixes: a8ba798b ("selftests: enable O and KBUILD_OUTPUT")
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
Tested by: Bamvor Jian Zhang <bamvor.zhangjian@linaro.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

Currently we can't build some tests, for example:

  $ make -C tools/testing/selftests/ TARGETS=vm
  ...
  gcc -Wall -I ../../../../usr/include   -lrt -lpthread ../../../../usr/include/linux/kernel.h userfaultfd.c -o tools/testing/selftests/vm/userfaultfd
  /tmp/ccmOkQSM.o: In function `stress':
  userfaultfd.c:(.text+0xc60): undefined reference to `pthread_create'
  userfaultfd.c:(.text+0xca5): undefined reference to `pthread_create'
  userfaultfd.c:(.text+0xcee): undefined reference to `pthread_create'
  userfaultfd.c:(.text+0xd30): undefined reference to `pthread_create'
  userfaultfd.c:(.text+0xd77): undefined reference to `pthread_join'
  userfaultfd.c:(.text+0xe7d): undefined reference to `pthread_join'
  userfaultfd.c:(.text+0xe9f): undefined reference to `pthread_cancel'
  userfaultfd.c:(.text+0xec6): undefined reference to `pthread_join'
  userfaultfd.c:(.text+0xf14): undefined reference to `pthread_join'
  /tmp/ccmOkQSM.o: In function `userfaultfd_stress':
  userfaultfd.c:(.text+0x13e2): undefined reference to `pthread_attr_setstacksize'
  collect2: error: ld returned 1 exit status

This is because the rule for linking .c files to binaries is incorrect.

The first bug is that it uses $< (first prerequisite) instead of $^ (all
preqrequisites), fix it by using ^$.

Secondly the ordering of the prerequisites vs $(LDLIBS) is wrong,
meaning on toolchains that use --as-needed we fail to link (as above).
Fix that by placing $(LDLIBS) *after* ^$.

Finally switch to using the default rule $(LINK.c), so that we get
$(CPPFLAGS) etc. included.

Fixes: a8ba798b ("selftests: enable O and KBUILD_OUTPUT")
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
Tested by: Bamvor Jian Zhang <bamvor.zhangjian@linaro.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

In commit 88baa78d ("selftests: remove duplicated all and clean
target"), the "all" target was removed from individual Makefiles and
added to lib.mk.

However the "all" target was added to lib.mk *after* the existing
"runtests" target. This means "runtests" becomes the first (default)
target for most of our Makefiles.

This has the effect of causing a plain "make" to build *and run* the
tests. Which is at best rude, but depending on which tests are run could
oops someone's build machine.

  $ make -C tools/testing/selftests/
  ...
  make[1]: Entering directory 'tools/testing/selftests/bpf'
  gcc -Wall -O2 -I../../../../usr/include   test_verifier.c -o tools/testing/selftests/bpf/test_verifier
  gcc -Wall -O2 -I../../../../usr/include   test_maps.c -o tools/testing/selftests/bpf/test_maps
  gcc -Wall -O2 -I../../../../usr/include   test_lru_map.c -o tools/testing/selftests/bpf/test_lru_map
  #0 add+sub+mul FAIL
  Failed to load prog 'Function not implemented'!
  #1 unreachable FAIL
  Unexpected error message!
  #2 unreachable2 FAIL
  ...

Fix it by moving the "all" target to the start of lib.mk, making it the
default target.

Fixes: 88baa78d ("selftests: remove duplicated all and clean target")
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
Tested by: Bamvor Jian Zhang <bamvor.zhangjian@linaro.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

Enable O and KBUILD_OUTPUT for kselftest. User could compile kselftest
to another directory by passing O or KBUILD_OUTPUT. And O is high
priority than KBUILD_OUTPUT.
Signed-off-by: NBamvor Jian Zhang <bamvor.zhangjian@linaro.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

Some testcases need the clean extra data after running. This patch
introduce the "EXTRA_CLEAN" variable to address this requirement.

After KBUILD_OUTPUT is enabled in later patch, it will be easy to
decide to if we need do the cleanup in the KBUILD_OUTPUT path(if the
testcase ran immediately after compiled).
Signed-off-by: NBamvor Jian Zhang <bamvor.zhangjian@linaro.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

There are difference rules for compiling c source file in different
testcases. In order to enable KBUILD_OUTPUT support in later patch,
this patch introduce the default rules in
"tools/testing/selftest/lib.mk" and remove the existing rules in each
testcase.
Acked-by: NMichael Ellerman <mpe@ellerman.id.au>
Signed-off-by: NBamvor Jian Zhang <bamvor.zhangjian@linaro.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

The TEST_DIRS was introduced in Commit e8c1d7cd ("selftests: copy
TEST_DIRS to INSTALL_PATH") for coping a whole directory in ftrace.

After rsync(with -a) is introduced by Commit 900d65ee ("selftests:
change install command to rsync"). Rsync could handle the directory
without the definition of TEST_DIRS.

This patch simply replace TEST_DIRS with TEST_FILES in ftrace and remove
the TEST_DIRS in tools/testing/selftest/lib.mk
Acked-by: NMichael Ellerman <mpe@ellerman.id.au>
Signed-off-by: NBamvor Jian Zhang <bamvor.zhangjian@linaro.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

Currently, kselftest use TEST_PROGS, TEST_PROGS_EXTENDED, TEST_FILES to
indicate the test program, extended test program and test files. It is
easy to understand the purpose of these files. But mix of compiled and
uncompiled files lead to duplicated "all" and "clean" targets.

In order to remove the duplicated targets, introduce TEST_GEN_PROGS,
TEST_GEN_PROGS_EXTENDED, TEST_GEN_FILES to indicate the compiled
objects.

Also, the later patch will make use of TEST_GEN_XXX to redirect these
files to output directory indicated by KBUILD_OUTPUT or O.

And add this changes to "Contributing new tests(details)" of
Documentation/kselftest.txt.
Signed-off-by: NBamvor Jian Zhang <bamvor.zhangjian@linaro.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

The command of install could not handle the special files in exec
testcases, change the default rule to rsync to fix this.

The installation is unchanged after this commit.
Suggested-by: NMichael Ellerman <mpe@ellerman.id.au>
Signed-off-by: NBamvor Jian Zhang <bamvor.zhangjian@linaro.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

When the test cases is not supported by the current architecture
the install files(TEST_PROGS, TEST_PROGS_EXTENDED and TEST_FILES)
will be empty. Check it before installation to dismiss a failure
reported by install program.
Signed-off-by: NBamvor Jian Zhang <bamvor.zhangjian@linaro.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

Loop over all TEST_DIRS and recursively copy them to the INSTALL_PATH. Tests
such as ftrace require a directory and all of it's contents to execute the
test properly, thus these directories and files need to be copied when we
perform an install.
Acked-by: NMichael Ellerman <mpe@ellerman.id.au>
Signed-off-by: NTyler Baker <tyler.baker@linaro.org>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

This avoids repeating the logic in every Makefile. We mimic the
top-level Makefile and use $(CROSS_COMPILE)gcc.
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

Change the timers Makefile to make use of shared run and install logic
in lib.mk. Destructive tests are installed but not run by default.

Add a new variable, TEST_PROGS_EXTENDED, which is a list of extra
programs to install, but which are not run by the default run_tests
logic.
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

This adds make install support to selftests. The basic usage is:

$ cd tools/testing/selftests
$ make install

That installs into tools/testing/selftests/install, which can then be
copied where ever necessary.

The install destination is also configurable using eg:

$ INSTALL_PATH=/mnt/selftests make install

The implementation uses two targets in the child makefiles. The first
"install" is expected to install all files into $(INSTALL_PATH).

The second, "emit_tests", is expected to emit the test instructions (ie.
bash script) on stdout. Separating this from install means the child
makefiles need no knowledge of the location of the test script.
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>

This adds a Make include file which most selftests can then include to
get the run_tests logic.

On its own this has the advantage of some reduction in repetition, and
also means the pass/fail message is defined in fewer places.

However the key advantage is it will allow us to implement install very
simply in a subsequent patch.

The default implementation just executes each program in $(TEST_PROGS).

We use a variable to hold the default implementation of $(RUN_TESTS)
because that gives us a clean way to override it if necessary, ie. using
override. The mount, memory-hotplug and mqueue tests use that to provide
a different implementation.

Tests are not run via /bin/bash, so if they are scripts they must be
executable, we add a+x to several.
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
Signed-off-by: NShuah Khan <shuahkh@osg.samsung.com>