Function pointers are not void*.  See also
ce4ea956
8427fca4

In the case that shinking the entries buffer to the exact size
failed, `dirp` is already closed.  Found by mame with Coverity
Scan.

Sort the results which matched single wildcard or character set in
binary ascending order, unless `sort: false` is given.  The order
of an Array of pattern strings and braces are not affected.

Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead.  This would significantly
speed up incremental builds.

We take the following inclusion order in this changeset:

1.  "ruby/config.h", where _GNU_SOURCE is defined (must be the very
    first thing among everything).
2.  RUBY_EXTCONF_H if any.
3.  Standard C headers, sorted alphabetically.
4.  Other system headers, maybe guarded by #ifdef
5.  Everything else, sorted alphabetically.

Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).

This removes the related tests, and puts the related specs behind
version guards.  This affects all code in lib, including some
libraries that may want to support older versions of Ruby.

This removes the security features added by $SAFE = 1, and warns for access
or modification of $SAFE from Ruby-level, as well as warning when calling
all public C functions related to $SAFE.

This modifies some internal functions that took a safe level argument
to no longer take the argument.

rb_require_safe now warns, rb_require_string has been added as a
version that takes a VALUE and does not warn.

One public C function that still takes a safe level argument and that
this doesn't warn for is rb_eval_cmd.  We may want to consider
adding an alternative method that does not take a safe level argument,
and warn for rb_eval_cmd.

Coverity Scan points out that all the return values of GLOB_ALLOC_N are
NULL-checked except this call.

This changeset basically replaces `ruby_xmalloc(x * y)` into
`ruby_xmalloc2(x, y)`.  Some convenient functions are also
provided for instance `rb_xmalloc_mul_add(x, y, z)` which allocates
x * y + z byes.

* dir.c (file_s_fnmatch): ensure that pattern does not contain a
  NUL character.  https://hackerone.com/reports/449617

Cfuncs that use rb_scan_args with the : entry suffer similar keyword
argument separation issues that Ruby methods suffer if the cfuncs
accept optional or variable arguments.

This makes the following changes to : handling.

* Treats as **kw, prompting keyword argument separation warnings
  if called with a positional hash.

* Do not look for an option hash if empty keywords are provided.
  For backwards compatibility, treat an empty keyword splat as a empty
  mandatory positional hash argument, but emit a a warning, as this
  behavior will be removed in Ruby 3.  The argument number check
  needs to be moved lower so it can correctly handle an empty
  positional argument being added.

* If the last argument is nil and it is necessary to treat it as an option
  hash in order to make sure all arguments are processed, continue to
  treat the last argument as the option hash. Emit a warning in this case,
  as this behavior will be removed in Ruby 3.

* If splitting the keyword hash into two hashes, issue a warning, as we
  will not be splitting hashes in Ruby 3.

* If the keyword argument is required to fill a mandatory positional
  argument, continue to do so, but emit a warning as this behavior will
  be going away in Ruby 3.

* If keyword arguments are provided and the last argument is not a hash,
  that indicates something wrong. This can happen if a cfunc is calling
  rb_scan_args multiple times, and providing arguments that were not
  passed to it from Ruby.  Callers need to switch to the new
  rb_scan_args_kw function, which allows passing of whether keywords
  were provided.

This commit fixes all warnings caused by the changes above.

It switches some function calls to *_kw versions with appropriate
kw_splat flags. If delegating arguments, RB_PASS_CALLED_KEYWORDS
is used.  If creating new arguments, RB_PASS_KEYWORDS is used if
the last argument is a hash to be treated as keywords.

In open_key_args in io.c, use rb_scan_args_kw.
In this case, the arguments provided come from another C
function, not Ruby.  The last argument may or may not be a hash,
so we can't set keyword argument mode.  However, if it is a
hash, we don't want to warn when treating it as keywords.

In Ruby files, make sure to appropriately use keyword splats
or literal keywords when calling Cfuncs that now issue keyword
argument separation warnings through rb_scan_args.  Also, make
sure not to pass nil in place of an option hash.

Work around Kernel#warn warnings due to problems in the Rubygems
override of the method.  There is an open pull request to fix
these issues in Rubygems, but part of the Rubygems tests for
their override fail on ruby-head due to rb_scan_args not
recognizing empty keyword splats, which this commit fixes.

Implementation wise, adding rb_scan_args_kw is kind of a pain,
because rb_scan_args takes a variable number of arguments.
In order to not duplicate all the code, the function internals need
to be split into two functions taking a va_list, and to avoid passing
in a ton of arguments, a single struct argument is used to handle
the variables previously local to the function.

After 5e86b005, I now think ANYARGS is
dangerous and should be extinct.  This commit deletes ANYARGS from
rb_ensure, which also revealed many arity / type mismatches.

I guess that this check was to detect re-entrance (maybe callcc?).
But currently it does not work as intended.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67337 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* dir.c (ruby_glob0): expand braces if a glob pattern starts with
  brace.
  [ruby-core:91728] [Bug #15649]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67332 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* dir.c (join_path_from_pattern): add the last slash for directory
  matching.

* test/ruby/test_dir.rb (test_glob_recursive_directory): add a test
  for above.
  [ruby-core:91110] [Bug #15540]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66838 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66762 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66205 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* dir.c (glob_helper): initialize `args.pathtype`.
  missed at r64810.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64849 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Fixed bug that glob with recursive and braces (**/{a,b}) pattern
fails.
[Feature #13167]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64836 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

join_path uses malloc. So free is required.
[Feature #13167]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64835 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Braces were expended before ruby_glob0(). This caused to call
replace_real_basename() for same plain patterns repeatedly.
Move blace expansion into glob_helper() in ruby_glob0() to reduce
replace_real_basename() call.
This fix changes the order of glob results.
[Feature #13167] [Fix GH-1864]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64810 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

[Fix GH-1958]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64808 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* dir.c (do_stat, do_lstat, do_opendir): need the length of the base
  path for fstatat() when fd is valid.

* dir.c (glob_helper): fix for platforms where DT_UNKNOWN is not
  available, e.g. Solaris.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64067 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

This reverts commit r63982.
It breaks build on Solaris 11.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* dir.c (do_stat, do_lstat): need the length of the base path for
  fstatat() when fd is valid.

* dir.c (glob_helper): fix for platforms where DT_UNKNOWN is not
  available, e.g. Solaris.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63982 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* dir.c (glob_helper): fix directory glob which resulted in lacking
  the first byte.  adjust the length of basename to be appended as
  well as removing the heading path, not the length of the joined
  path.  [ruby-dev:50588] [Bug #14899]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63909 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

SuSE 10 has openat(), but not O_CLOEXEC

Reported-by: wangpeiwen

[ruby-core:87591] [Bug #14864]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63720 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* dir.c (rb_push_glob): warn NUL-separated glob patterns.
  [Feature #14643]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63190 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

```
File.fnmatch("a*b", "a\nb") # => true
/\Aa.*b\z/x.match?("a\nb")  # => false
/\Aa.*b\z/mx.match?("a\nb") # => true
```

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63115 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* dir.c (dir_s_glob): [DOC] added an example of Dir.glob using
  pattern list.  [ci skip]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63041 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* dir.c (rb_push_glob): do not assume string is NUL terminated
  always, shared substring may not in the future.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63034 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* dir.c (GlobPathValue): should be used in rb_push_glob only.
  other methods should use FilePathValue.
  https://hackerone.com/reports/302338

* dir.c (rb_push_glob): expand GlobPathValue

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62989 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62544 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

* dir.c (dir_each_child_m): new instance methods Dir#each_child
  and Dir#children.  [Feature #13969]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62022 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62008 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e