If there are multiple sections with the same section name, the current
implementation results in several sections by the same heading, with the
content duplicated from the last section to all. Even if there's the
error message, a more graceful approach is to combine all the
identically named sections into one, with concatenated contents.

With the supported sections already limited to select few, there are
massively fewer collisions than there used to be, but this is still
useful for e.g. when function parameters are documented in the middle of
a documentation comment, with description spread out above and
below. (This is not a recommended documentation style, but used in the
kernel nonetheless.)

We can now also demote the error to a warning.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

kernel-doc currently identifies anything matching "section header:"
(specifically a string of word characters and spaces followed by a
colon) as a new section in the documentation comment, and renders the
section header accordingly.

Unfortunately, this turns all uses of colon into sections, mostly
unintentionally. Considering the output, erroneously creating sections
when not intended is always worse than erroneously not creating sections
when intended. For example, a line with "http://example.com" turns into
a "http" heading followed by "//example.com" in normal text style, which
is quite ugly. OTOH, "WARNING: Beware of the Leopard" is just fine even
if "WARNING" does not turn into a heading.

It is virtually impossible to change all the kernel-doc comments, either
way. The compromise is to pick the most commonly used and depended on
section headers (with variants) and accept them as section headers.

The accepted section headers are, case insensitive:

 * description:
 * context:
 * return:
 * returns:

Additionally, case sensitive:

 * @return:

All of the above are commonly used in the kernel-doc comments, and will
result in worse output if not identified as section headers. Also,
kernel-doc already has some special handling for all of them, so there's
nothing particularly controversial in adding more special treatment for
them.

While at it, improve the whitespace handling surrounding section
names. Do not consider the whitespace as part of the name.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Yes, for our purposes the type should contain typedef.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

The latter isn't special to rst.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

If a param description spans multiple lines, check any leading
whitespace in the first continuation line, and remove same amount of
whitespace from following lines.

This allows indentation in the multi-line parameter descriptions for
aesthetical reasons while not causing accidentally significant
indentation in the rst output.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Handle whitespace on the first line of param text as if it was the empty
string. There is no need to add the newline in this case. This improves
the rst output in particular, where blank lines may be problematic in
parameter lists.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Move away from field lists, and simply use **strong emphasis** for
section headings on lines of their own. Do not use rst section headings,
because their nesting depth depends on the surrounding context, which
kernel-doc has no knowledge of. Also, they do not need to end up in any
table of contexts or indexes.

There are two related immediate benefits. Field lists are typically
rendered in two columns, while the new style uses the horizontal width
better. With no extra indent on the left, there's no need to be as fussy
about it. Field lists are more susceptible to indentation problems than
the new style.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

The inline member markup allows whitespace lines before the actual
documentation starts. Strip the leading blank lines. This improves the
rst output.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Current approach leads to two blank lines, while one is enough.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

No functional changes.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

The use of these is confusing in the script, and per this grep, they're
not used anywhere anyway:

$ git grep " \* [%$&][a-zA-Z0-9_]*:" -- *.[ch] | grep -v "\$\(Id\|Revision\|Date\)"

While at it, throw out the constants array, nothing is ever put there
again.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Let the user use @foo, &bar, %baz, etc. in the first kernel-doc purpose
line too.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

This bit is already done by xml_unescape() above.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Link "&foo->bar", "&foo->bar()", "&foo.bar", and "&foo.bar()" to the
struct/union/enum foo definition. The members themselves do not
currently have anchors to link to, but this is better than nothing, and
promotes a universal notation.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Let the user use "&union foo" and "&typedef foo" to reference foo. The
difference to using "union &foo", "typedef &foo", or just "&foo" (which
are valid too) is that "union" and "typedef" become part of the link
text.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

It's possible to use &foo to reference structs, enums, typedefs, etc. in
the Sphinx C domain. Thus do not prefix the links with "struct".
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

The Sphinx C domain spec says function references should include the
parens ().
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

If the user requests a specific DOC: section by name, do not output its
section title. In these cases, the surrounding context already has a
heading, and the DOC: section title is only used as an identifier and a
heading for clarity in the source file.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Make the output selection a bit more readable by adding constants for
the various types of output selection. While at it, actually call the
variable for choosing what to output $output_selection.

No functional changes.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Make the state machine a bit more readable by adding constants for
parser states and inline member documentation parser substates. While at
it, rename the "split" documentation to "inline" documentation.

No functional changes.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Currently we use docproc to figure out which symbols are exported, and
then docproc calls kernel-doc on specific functions, to get
documentation on exported functions. According to git blame and docproc
comments, this is due to historical reasons, as functions and their
corresponding EXPORT_SYMBOL* may have been in different files. However
for more than ten years the recommendation in CodingStyle has been to
place the EXPORT_SYMBOL* immediately after the closing function brace
line.

Additionally, the kernel-doc comments for functions are generally placed
above the function definition in the .c files (i.e. where the
EXPORT_SYMBOL* is) rather than above the declaration in the .h
files. There are some exceptions to this, but AFAICT none of these are
included in DocBook documentation using the "!E" docproc directive.

Therefore, assuming the EXPORT_SYMBOL* and kernel-doc are with the
function definition, kernel-doc can extract the exported vs. not
information by making two passes on the input file. Add support for that
via the new -export and -internal parameters.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

I'm not quite sure why the errors below are happening, but this fixes
them.

Use of uninitialized value in string ne at ./scripts/kernel-doc line 1819, <IN> line 6494.
Use of uninitialized value $_[0] in join or string at ./scripts/kernel-doc line 1759, <IN> line 6494.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

First, the headings for structs, enums and typedefs will be similar to
functions. Second, this provides a kind of namespace for cross
references. Third, and most importantly, the return and parameter types
from .. c:function:: definitions will automagically become cross
references to the documented types.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

If given the -rst flag, output will now be in RestructuredText.  Various
glitches to be worked out yet.

In the end I decided not to use RST section headings within the kerneldoc
comments.  gpu.tmpl already has headings five levels deep; adding more is
not going to bring clarity.

This is really just Jani Nikula's asciidoc change with the serial numbers
filed off.  It's a hack job that doubtless needs a lot of cleaning up.
Signed-off-by: NJonathan Corbet <corbet@lwn.net>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

Instead of having the kernel-doc usage in both comments and in output to
the user, merge them all to one here document. While at it, imrove the
text and make it pretty. Give shoemaker's children some shoes.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

Some documented structures in the kernel use DECLARE_BITMAP to create
arrays of unsigned longs to store information using the bitmap functions.
These have to be replaced with a parsable version for kernel-doc.

For example a simple input like

    /**
     * struct something - some test
     * @members: active members
     */
    struct something {
    	DECLARE_BITMAP(members, MAX_MEMBERS);
    };

resulted in parsing warnings like

    warning: No description found for parameter 'MAX_MEMBERS)'
    warning: Excess struct/union/enum/typedef member 'members' description in 'something'
Signed-off-by: NConchúr Navid <conchur@web.de>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

Some enumerations in the kernel headers use #ifdef to reduce their size
based on the the configuration. These lines have to be stripped to avoid
parsing problems.

For example a simple input like

    /**
     * enum flags - test flags
     * @flag1: first flag
     * @flag2: second flag
     */
    enum flags {
    	flag1 = BIT(0),
    #ifdef SECOND_FLAG
    	flag2 = BIT(1),
    #endif
    };

resulted in parsing warnings like

    warning: Enum value '#ifdef SECOND_FLAG;flag2 = BIT(1)' not described in enum 'flags'
    warning: Enum value '#endif;' not described in enum 'flags'
Signed-off-by: NConchúr Navid <conchur@web.de>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

The regex to strip single line #define's in enumerations depends on the
fact that the defines are still stored on separate lines. But the
surrounding code already removed newlines and replaced them with
semicolons.

For example a simple input like

    /**
     * enum flags - test flags
     * @flag1: first flag
     * @flag2: second flag
     * @flag3: third flag
     * @flag4: fourth flag
     */
    enum flags {
    	flag1 = BIT(0),
    	flag2 = BIT(1),
    #define flags_small (flag1 | flag2)
    	flag3 = BIT(2),
    	flag4 = BIT(3),
    #define flags_big (flag2 | flag3)
    };

resulted in parsing warnings like

    warning: Enum value '#define flags_small (flag1 | flag2);flag3 = BIT(2)' not described in enum 'flags'
    warning: Enum value '#define flags_big (flag2 | flag3);' not described in enum 'flags'
Signed-off-by: NConchúr Navid <conchur@web.de>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

Changeset 4d732701('scripts/kernel-doc: Replacing highlights
hash by an array') broke compatibility of the kernel-doc script with
older versions of perl by using "keys ARRAY" syntax with is available
only on Perl 5.12 or newer, according with:
	http://perldoc.perl.org/functions/keys.html

Restore backward compatibility by replacing "foreach my $k (keys ARRAY)"
by a C-like variant: "for (my $k = 0; $k < !ARRAY; $k++)"
Signed-off-by: NMauro Carvalho Chehab <mchehab@osg.samsung.com>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

On some places, people could use Private: to tag the private fields
of an struct. So, be case-insensitive when parsing "private:"
meta-tag.
Signed-off-by: NMauro Carvalho Chehab <mchehab@osg.samsung.com>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

A typedef function looks more likely a function and not a
normal typedef. Change the code to use the output_function_*,
in order to properly parse the function prototype parameters.
Signed-off-by: NMauro Carvalho Chehab <mchehab@osg.samsung.com>

The current typedef parser only works for non-function typedefs.

As we need to also document some function typedefs, add a
parser for it.
Signed-off-by: NMauro Carvalho Chehab <mchehab@osg.samsung.com>

The "highlight" code is very sensible to the order of the hash keys,
but the order of the keys cannot be predicted. It generates
faulty DocBook entries like:
	- @<function>device_for_each_child</function>

Sorting the result is not enough some times (as it's deterministic but
we can't control it).
We should use an array for that job, so we can guarantee that the order
of the regex execution on dohighlight is correct.

[jc: I think this is kind of papering around the real problem, that people
 are saying @function() when "function" is not a parameter.  But this makes
 things better than they were before, so...]
Signed-off-by: NDanilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

Currently kernel-doc generates a dummy DocBook file when asked to
convert a C source file with no structured comments.  For an
out-of-tree build (objtree != srctree), the title of the output file
is the absolute path name of the C source file, which later results
in a manual page being created alongside the C source file.

Change the title to be a relative path.
Signed-off-by: NBen Hutchings <ben@decadent.org.uk>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

Docproc processes the EXPORT_SYMBOL(f1) macro and uses -nofunc f1 to
avoid duplicated documentation in the next call.
It works for most of the cases, but there are some specific situations
where a struct has the same name of an already-exported function.

Current kernel-doc behavior ignores those structs and does not add them
to the final documentation. This patch fixes it.

This is unusual, the only case I've found is the drm_modeset_lock
(function and struct) defined in drm_modeset_lock.h and
drm_modeset_lock.c. Considering this, it should only affect the DRM
documentation by including struct drm_modeset_lock to the final Docbook.
Signed-off-by: NDanilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

Editors like emacs and vi recognize a number of error message formats.
The format used by the kerneldoc tool is not recognized by emacs.

Change the kerneldoc error message format to the GNU style such that the
emacs prev-error and next-error commands can be used to navigate through
kerneldoc error messages.  For more information about the GNU error
message format, see also
  https://www.gnu.org/prep/standards/html_node/Errors.html.

This patch has been generated via the following sed command:

  sed -i.orig 's/Error(\${file}:\$.):/\${file}:\$.: error:/g;s/Warning(\${file}:\$.):/\${file}:\$.: warning:/g;s/Warning(\${file}):/\${file}:1: warning:/g;s/Info(\${file}:\$.):/\${file}:\$.: info:/g' scripts/kernel-doc
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Cc: Johannes Berg <johannes.berg@intel.com>
Acked-by: NRandy Dunlap <rdunlap@infradead.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The kernel-doc script gets confused by __attribute__(()) strings in
structures, so just clean the out.  Also ignore the CRYPTO_MINALIGN_ATTR
macro used in the crypto subsystem.
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

Describing arguments at top of a struct definition works fine
for small/medium size structs, but it definitely doesn't work well
for struct with a huge list of elements.

Keeping the arguments list inside the struct body makes it easier
to maintain the documentation.
ie:
/**
 * struct my_struct - short description
 * @a: first member
 * @b: second member
 *
 * Longer description
 */
struct my_struct {
    int a;
    int b;
    /**
     * @c: This is longer description of C
     *
     * You can use paragraphs to describe arguments
     * using this method.
     */
    int c;
};

This patch allows the use of this kind of syntax. Only one argument
per comment and user can use how many paragraphs he needs. It should
start with /**, which is already being used by kernel-doc. If those
comment doesn't follow those rules, it will be ignored.
Signed-off-by: NDanilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Stephan Mueller <smueller@chronox.de>
Cc: Michal Marek <mmarek@suse.cz>
Cc: linux-kernel@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
Cc: dri-devel <dri-devel@lists.freedesktop.org>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

Together with the preceding changes, this allows man pages to be built
reproducibly.
Signed-off-by: NBen Hutchings <ben@decadent.org.uk>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>

Regular expressions for highlights in kernel-doc are stored in a Perl
hash.  These hashes are ordered differently for each Perl run. This will
prevent kernel-doc to behave deterministically when parsing “@foo()” as
in some runs it will be interpreted as a parameter and in the others it
will be interpreted as a function.

We now sort the %highlights hash to get the same behavior on every run.
Signed-off-by: NJérémy Bobbio <lunar@debian.org>
Signed-off-by: NBen Hutchings <ben@decadent.org.uk>
Signed-off-by: NJonathan Corbet <corbet@lwn.net>