Add support for locating an image header in an image.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Add support for locating an image's Fdt map which is used to determine
the contents and structure of the image.
Signed-off-by: NSimon Glass <sjg@chromium.org>

It is useful to be able to summarise all the entries in an image, e.g. to
display this to this user. Add a new ListEntries() method to Entry, and
set up a way to call it through the Image class.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Detect when an unknown or unsupported file type is specified and report
an error.
Signed-off-by: NSimon Glass <sjg@chromium.org>

It is useful to add the CBFS file information (offset, size, etc.) into
the FDT so that the layout is complete. Add support for this.
Signed-off-by: NSimon Glass <sjg@chromium.org>

The purpose of this badly named field is a bit ambiguous. Adjust the code
to use it only to store the uncompressed length of a file, leaving it set
to None if there is no compression used. This makes it easy to see if the
value in this field is relevant / useful.

Also set data_len for compressed fields, since it should be the length of
the compressed data, not the uncompressed data.
Signed-off-by: NSimon Glass <sjg@chromium.org>

At present a file with no explicit CBFS offset is placed in the next
available location but there is no way to find out where it ended up.
Update and rename the get_data() function to provide this information.
Signed-off-by: NSimon Glass <sjg@chromium.org>

At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Add support for detecting entries that change size after they have already
been packed, and re-running packing when it happens.

This removes the limitation that entry size cannot change after
PackEntries() is called.
Signed-off-by: NSimon Glass <sjg@chromium.org>

We plan to support changing the size of entries after they have been
packed. For now it will always be enabled. But to aid testing of both
cases (in the event that we want to add a command-line flag, for example),
add a setting to control it.
Signed-off-by: NSimon Glass <sjg@chromium.org>

At present if this function tries to update the contents such that the
size changes, it raises an error. We plan to add the ability to change
the size of entries after packing is completed, since in some cases it is
not possible to determine the size in advance.

An example of this is with a compressed device tree, where the values
of the device tree change in SetCalculatedProperties() or
ProcessEntryContents(). While the device tree itself does not change size,
since placeholders for any new properties have already bee added by
AddMissingProperties(), we cannot predict the size of the device tree
after compression. If a value changes from 0 to 0x1234 (say), then the
compressed device tree may expand.

As a first step towards supporting this, make ProcessContentsUpdate()
return a value indicating whether the content size is OK. For now this is
always True (since otherwise binman raises an error), but later patches
will adjust this.
Signed-off-by: NSimon Glass <sjg@chromium.org>

SetContents() should only be called to set the contents of an entry from
within the ObtainContents() call, since it has no guard against increasing
the size of the contents, thus triggering incorrect operation.

Change all such calls to use ProcessUpdateContents() instead.
Signed-off-by: NSimon Glass <sjg@chromium.org>

This function raises an exception with its arguments around the wrong way
so the message is incorrect. Fix this as well as a few minor comment
problems.
Signed-off-by: NSimon Glass <sjg@chromium.org>

At present entry modules can only be accessed using Entry.Lookup() or
Entry.Create(). Most of the time this is fine, but sometimes a module
needs to provide constants or helper functions useful to other modules.
It is easier in this case to use 'import'.

Add an __init__ file to permit this.
Signed-off-by: NSimon Glass <sjg@chromium.org>

The first argument is not used. Remove it.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Compression is currently available only with blobs. However we want to
report the compression algorithm and uncompressed size for all entries,
so that other entry types can support compression. This will help with
the forthcoming 'list' feature which lists entries in the image.

Move the compression properties into the base class. Also fix up the docs
which had the wrong property name.
Signed-off-by: NSimon Glass <sjg@chromium.org>

This class is the new way to handle arguments in Python. Convert binman
over to use it. At the same time, introduce commands so that we can
separate out the different parts of binman functionality.
Signed-off-by: NSimon Glass <sjg@chromium.org>

It is useful to be able to quickly locate the FDT map in the image. An
easy way to do this is with a pointer at the start or end of the image.

Add an 'image header' entry, which places a magic number followed by a
pointer to the FDT map. This can be located at the start or end of the
image, or at a chosen location.

As part of this, update GetSiblingImagePos() to detect missing siblings.
Signed-off-by: NSimon Glass <sjg@chromium.org>

An FDT map is an entry which holds a full description of the image
entries, in FDT format. It can be discovered using the magic string at
its start. Tools can locate and read this entry to find out what entries
are in the image and where each entry is located.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Quite a few tests will use a real device tree and need it updated with the
binman metadata. Add a helper function for this.
Signed-off-by: NSimon Glass <sjg@chromium.org>

A few new features have been added. This has rendered part of the README
obsolete. Update it.
Signed-off-by: NSimon Glass <sjg@chromium.org>

A few features have been completed and a few items are added.
Signed-off-by: NSimon Glass <sjg@chromium.org>

The current test for the 'entry' module is a bit convoluted since it has
to import the module multiple times. It also relies on ordering, in that
test1EntryNoImportLib() must run before test2EntryImportLib() if they are
running in the same Python process.

This is unreliable since neither the ordering of tests nor the process
that they run in is defined.

Fix this by always reloading the entry in these two tests. Also add a
check that the expected value of have_importlib is obtained.

This corrects a code-coverage problem in the 'entry' module on some
systems.
Signed-off-by: NSimon Glass <sjg@chromium.org>

A feature of CBFS is that it allows files to be positioned at particular
offset (as with binman in general). This is useful to support
execute-in-place (XIP) code, since this may not be relocatable.

Add a new cbfs-offset property to control this.
Signed-off-by: NSimon Glass <sjg@chromium.org>

When there is lots of open space in a CBFS it is normally padded with
'empty' files so that sequentially scanning the CBFS can skip from one to
the next without a break.

Add support for this.
Signed-off-by: NSimon Glass <sjg@chromium.org>

An Integrated Firmware Image is used to hold various binaries used for
booting with Apollolake and some later devices. Add support for this.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Add support for putting CBFSs (Coreboot Filesystems) in an image. This
allows binman to produce firmware images used by coreboot to boot.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Coreboot uses a simple flash-based filesystem called Coreboot Filesystem
(CBFS) to organise files used during boot. This allows files to be named
and their position in the flash to be set. It has special features for
dealing with x86 devices which typically memory-map their SPI flash to the
top of 32-bit address space and need a 'boot block' ending there.

Create a library to help create and read CBFS files. This includes a
writer class, a reader class and associated other helpers. Only a subset
of features are currently supported.
Signed-off-by: NSimon Glass <sjg@chromium.org>

This tool has quite a few arguments and options, so put the functionality
in a function so that we call it from one place and hopefully get it
right.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Tools like ifwitool may not be available in the PATH, but are available in
the build. These tools may be needed by tests, so allow tests to use the
--toolpath flag.

Also use this flag with travis.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Sometimes when debugging tests it is useful to keep the input and output
directories so they can be examined later. Add an option for this and
update the binman tests to support it. This affects both the test class
and the tearDown() function called after each test.
Signed-off-by: NSimon Glass <sjg@chromium.org>

At present the -v flag is ignored with tests, so that (for example) -v2
does not have any effect. Update binman to pass this flag through to tests
so that they work just like running binman normally, except in a few
special cases where we are actually testing behaviour with different
levels of verbosity.
Signed-off-by: NSimon Glass <sjg@chromium.org>

This should be -u, not -up, since we don't need to preserve the output
directory in this case.
Signed-off-by: NSimon Glass <sjg@chromium.org>

We currenty support using the ELF file in U-Boot proper and SPL, but not
TPL. Add this as it is useful both with sandbox and for CBFS to allow
adding TPL as a 'stage'.
Signed-off-by: NSimon Glass <sjg@chromium.org>

This comment mentions the wrong default filename. Fix it.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Avoid duplicate code here by using the new compression function in the
tools module.
Signed-off-by: NSimon Glass <sjg@chromium.org>

The -D option enables debug mode, but we only need to add -D to the
command line once. Drop the duplicate code. Also drop the comment about
enabling debugging since this can be done with -D.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Update the compression test to use the tools module to decompress the
output data.
Signed-off-by: NSimon Glass <sjg@chromium.org>

Add utility functions to compress and decompress using lz4 and lzma
algorithms. In the latter case these use the legacy lzma support favoured
by coreboot's CBFS.

No tests are provided as these functions will be tested by the CBFS
tests in a separate patch.
Signed-off-by: NSimon Glass <sjg@chromium.org>

At present text entries use an indirect method to specify the text to use,
with a label pointing to the text itself.

Allow the text to be directly written into the node. This is more
convenient in cases where the text is constant.
Signed-off-by: NSimon Glass <sjg@chromium.org>