Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

The IF statements for DEFAULTS and TEST_START sections now handle
complex statements (&&,||)

Example:

  TEST_START IF (DEFINED ALL_TESTS || ${MYTEST} == boottest) && ${MACHINE} == gandalf
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Have IF statements process if a config variable or option has been
defined or not. Can use NOT DEFINED in the case for telling if
a variable or option has not been defined.

DEFAULTS IF NOT DEFINED SSH_USER
SSH_USER = root
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

The OVERRIDE keyword will allow options defined in the given
DEFAULTS section to override options defined in previous DEFAULT
sections.

Normally, options will error if they were previous defined.
The OVERRIDE keyword allows options that have been previously
defined to be changed in the given section.

Note, the same option can not be defined in the same DEFAULT section
even if that section is marked as OVERRIDE.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Have the reading of the config file allow reading of other config
files using the INCLUDE keyword. This allows multiple config files
to share config options.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Allow ==, !=, <=, >=, <, and > to be used in IF statements
to compare if a section should be processed or not.

For example:

BITS := 32

DEFAULTS IF ${BITS} == 32
MIN_CONFIG = ${CONFIG_DIR}/config-32
ELSE
MIN_CONFIG = ${CONFIG_DIR}/config-64
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Add IF keyword to sections within the config. Also added an ELSE
keyword that allows different config options to be set for a given
section.

For example:

TYPE := 1
STATUS := 0

DEFAULTS IF ${TYPE}
[...]
ELSE IF ${STATUS}
[...]
ELSE
[...]

The above will process the first section as $TYPE is true. If it
was false, it would process the last section as $STATUS is false.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

When ktest.pl reboots, it will usuall wait SLEEP_TIME seconds of idle
console before starting the next test. By setting the
REBOOT_SUCCESS_LINE, ktest will not wait SLEEP_TIME when it detects the
line while rebooting to a new kernel.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

There's cases where running the same kernel over and over again
is useful, and being able to not install the same kernel can
save time between tests.

Add a NO_INSTALL option that tells ktest.pl to not install the
new kernel.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

In testing one of my boxes, I found that I only wanted to build and
install the kernel. I wanted to manually reboot the box and test it.
Adding a TEST_TYPE option "install" allows this to happen.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

If the defined OUTPUT_MIN_CONFIG in the make_min_config test exists,
then give a prompt to ask the user if they want to use that config
instead, as it is very often the case, especially when the test has been
interrupted. The OUTPUT_MIN_CONFIG is usually the config that one wants
to use to continue the test where they left off.

But if START_MIN_CONFIG is defined (thus the MIN_CONFIG is not the
default), then do not prompt, as it will be annoying if the user has
this as one of many tests, and the test pauses waiting for input, while
the user is sleeping.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

To save time, the test does not just grab any option and test
it. The Kconfig files are examined to determine the dependencies
of the configs. If a config is chosen that depends on another
config, that config will be checked first. By checking the
parents first, we can eliminate whole groups of configs that
may have been enabled.

For example, if a USB device config is chosen and depends on
CONFIG_USB, the CONFIG_USB will be tested before the device.
If CONFIG_USB is found not to be needed, it, as well as all
configs that depend on it, will be disabled and removed from
the current min_config.

Note, the code from streamline_config (make localmodconfig)
was copied and used to find the dependencies in the Kconfig file.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

After doing a make localyesconfig, your kernel configuration may
not be the most useful minimum configuration. Having a true minimum
config that you can use against other configs is very useful if
someone else has a config that breaks on your code. By only forcing
those configurations that are truly required to boot your machine
will give you less of a chance that one of your set configurations
will make the bug go away. This will give you a better chance to
be able to reproduce the reported bug matching the broken config.

Note, this does take some time, and may require you to run the
test over night, or perhaps over the weekend. But it also allows
you to interrupt it, and gives you the current minimum config
that was found till that time.

Note, this test automatically assumes a BUILD_TYPE of oldconfig
and its test type acts like boot.

TODO: add a test version that makes the config do more than just
      boot, like having network access.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Doing a patchcheck test, there may be warnings that gcc produces which
may be OK, and the test should not fail on that commit. By adding a
IGNORE_WARNINGS option to list a space delimited SHA1s that are ignored
lets the user avoid having the test fail on certain commits.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

As multiple tests may be executed by the same server, have the test
machine name add uniqueness to the value of the temp directory.
Otherwise the temp directories may overwrite each other's tests.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

There are some cases that a patch may be needed to apply to the kernel
in patchcheck or bisect tests. Adding a PRE_BUILD option to apply the
patch and POST_BUILD to remove it, allows for this to be done easily.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Searching through several tests, it gets confusing which test result
is for which test. By adding the TEST_NAME option, the user can tell
which test result belongs to which test.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Currently the config_bisect compares the min config with the
CONFIG_BISECT config. There may be another config that we know
is good that we want to ignore configs on. By passing in this
config it will ignore the options that are set in the good config.

Note: This only ignores the config, it does not (yet) handle
options that are different between the two configs. If the good
config has "SLAB" set and the bad config has "SLUB" it will not
find the bug if the bug had to do with changing these two options.

This is something that I intend to implement in the future.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

When a triple fault happens in a test, no call trace nor panic
is displayed. Instead, the system reboots to the good kernel.
Since the good kernel may display a boot prompt that matches the
success string, ktest may think that the test succeeded, when it
did not.

Detecting triple faults is tricky because it is hard to generalize
what a reboot looks like. The best that we can come up with for now
is to examine the Linux banner. If we detect that the Linux banner
matches the test we want to test, then look to see if we hit another
Linux banner with a different kernel is booted. This can be assumed
to be a triple fault.

We can't just check for two Linux banners because things like
early printk may cause the Linux banner to be displayed twice. Checking
for different kernel versions should be the safe bet.

If this for some reason detects a false triple boot. A new ktest
config option is also created:

 DETECT_TRIPLE_FAULT

This can be set to 0 to disable this checking.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

There are cases where one ktest option may be used within another
ktest option. Allow them to be reused just like config variables
but there are evaluated at time of test not config processing time.

Thus having something like:

MAKE_CMD = make ARCH=${ARCH}

TEST_START
ARCH = powerpc

TEST_START
ARCH = arm

Will have the arch defined for each test iteration.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

I found that I constantly reuse information for each test case.
It would be nice to just define a variable to reuse.

For example I may have:

TEST_START
[...]
TEST = ssh root@mybox /path/to/my/script

TEST_START
[...]
TEST = ssh root@mybox /path/to/my/script

[etc]

The issue is, I may wont to change that script or one of the other
fields. Then I need to update each line individually.

With the addition of config variables (variables only used during parsing
the config) we can simplify the config files. These variables can
also be defined multiple times and each time the new value will
overwrite the old value.

The convention to use a config variable over a ktest option is to use :=
instead of =.

Now we could do:

USER := root
TARGET := mybox
TEST_SCRIPT := /path/to/my/script
TEST_CASE := ${USER}@${TARGET} ${TEST_SCRIPT}

TEST_START
[...]
TEST = ${TEST_CASE}

TEST_START
[...]
TEST = ${TEST_CASE}

[etc]

Now we just need to update the variables at the top.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

The patches being checked may not leave the kernel in a state
that the next run will allow the new kernel to be copied to the
machine. Reboot to a known good kernel before continuing to the
next kernel to test.

Added option PATCHCHECK_SLEEP_TIME for the max time to sleep between
patchcheck reboots.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Currently, if a test causes constant output but never reaches a
boot prompt, or crashes, the test will never stop. Add STOP_TEST_AFTER
to create a variable that will stop (and fail) the test after it has run
for this amount of time. The default is 10 minutes. Setting this
variable to -1 will disable it.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Add the config option BISECT_FILES that allows the user to
specify what path in the kernel to run the git bisect on.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

If a during a git bisect, ktest fails on something other than
what it is testing (if BISECT_TYPE is test but it fails on build),
if BISECT_SKIP is set, then it will do a "git bisect skip" instead
of just failing the bisect and letting the user find a good commit
to test.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

For both git bisect and config bisect, if BISECT_MANUAL is set to 1,
then bisect will stop between iterations and ask the user for the
result. The actual result is ignored. This makes it possible to
use ktest.pl for bisecting configs and git and let the user examine
the results themselves and enter their own results.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Added the options STOP_AFTER_SUCCESS and STOP_AFTER_FAILURE to
allow the user to give a time (in seconds) to stop the monitor
after a stack trace or login has been detected. Sometimes the
kernel constantly prints out to the console and this may cause
the test to run indefinitely.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Added documentation for SSH_EXEC, SCP_TO_TARGET, REBOOT,
and CONFIG_BISECT and friends.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Change the config to use TEST_START where the options after a
TEST_START automatically get the [] as it is read and they do
not need to exist in the config file;

TEST_START
MIN_CONFIG = myconfig

is the same as

MIN_CONFIG[1] = myconfig

The benefit is that you no longer need to keep track of test numbers
with tests.

Also process the commit ids that are passed to the options
to get the actually SHA1 so it is no longer relative to the branch.
Ie, saying HEAD will get the current SHA1 and then that will
be used, and will work even if another branch is checked out.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Added the options POWEROFF_AFTER_HALT to handle boxes that do not
really shut off after a halt is called.

Added POWERCYCLE_AFTER_REBOOT to force a power cycle for boxes that
don't reboot but get stuck during the reboot.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Add a POST_INSTALL option that runs after the build and install
but before rebooting to the test kernel. This alls the user to
run a script that will install an initrd (or anything else that may
be special) before booting.

An environment variable KERNEL_VERSION is set.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>

Added sample.conf as a nice document to show new users.

Use a %default hash to separate out the options that are default
and allow us to complain about options being set twice.
Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>