We set CDB_PALLOC_TAGS regardless of USE_ASSERT_CHECKING on 81627e27. This bloated chunk header in release build, resulting in significantly more memory consumption in certain cases.

In this PR we are restoring the original behavior by setting CDB_PALLOC_TAGS only when USE_ASSERT_CHECKING is set. We are also removing unused code.
Signed-off-by: NFoyzur Rahman <foyzur@gmail.com>

Previously accounting tree and reference to accounts were pointer based.
This prevents us from freeing accounts at the end of each statement as
later statements of that transaction can potentially access memory and
the corresponding accounts. This resulted in a perceived memory leak as
the accounts were not freed per-statement.

In this implementation we convert pointers to index that can refer to an
array of accounts. This allows us to partition indexes based on
"generation" to define which accounts are live and which are already
dead. This is also a soft-pointer so we don't crash accessing it. This
also allows us to optimize the previous heavyweight "rollover" at the
time of generation overflow as we no longer need to detect the validity
of accounts based on generation, and rather we can just refer to logical
range of indexes to identify the liveness.

[#120028219]

    The GPDB Vmem is the lowest layer of memory allocator that supports higher allocators such as AllocSet. This layer (mostly defined in memprot.c) is in charge of actually calling malloc/realloc/free to allocate/reallocate/free memory. In this process this layer is also in charge of reserving "virtual" memory or Vmem, which is a GPDB specific shared memory counter to track per-segment combined allocations across all the GPDB processes under Vmem umbrella. The Vmem counter is managed by a separate module Vmem_Tracker, and the memprot functions (such as gp_malloc, gp_free2 and gp_realloc) call the APIs provided by VmemTracker.

    Previously the memprot allocators (gp_alloc/gp_realloc/gp_free) were only allocating/freeing memory but were not adding any additional metadata in the header (and there was no header) to track the size of allocations. Therefore, there was no gp_free as freeing memory requires the size of the free to adjust Vmem counter inside VmemTracker. This was patched by explicitly passing size info in gp_free2.

    In this PR we do the following:

    * We add allocation size in Vmem header (along with checksums which are only available in debug build to detect header and footer boundary, and buffer overruns).

    * We remove size information from the block header of AllocSet.

    * We rename gp_free2 to gp_free as the second parameter (size information) is now obtained from the header and therefore no longer necessary

    * We modify all the consumers of memprot.c APIs to use the new APIs

    * We add unit tests to test the metadata and the correctness of the new Vmem allocators

    This is the first step to integrate external modules and third party allocations with Vmem. A long running issue in GPDB is its inability to track allocations by external components including libraries such as ORCA. Therefore, the central Vmem counter is often way off from the underlying allocations, and this may run the system out of memory. By maintaining the size information in the Vmem header, we now have a self-contained allocator that can be exposed to external allocators such as GPOS allocators, without forcing them to manage size information separately.

    This fixes #117269929.
Signed-off-by: NMarc Spehlmann <marc.spehlmann@gmail.com>

third party includes (like tcl) that define DLLIMPORT.

back-stamped for this.

Tag appropriate files for rc3

Also performed an initial run through of upgrading our Copyright date to
extend to 2005 ... first run here was very simple ... change everything
where: grep 1996-2004 && the word 'Copyright' ... scanned through the
generated list with 'less' first, and after, to make sure that I only
picked up the right entries ...

CurrentMemoryContext is DLLIMPORT on Win32.  Work around that by
creating stubs in the backend for palloc/pstrdup.

Also fix pg_dumpall to do proper quoting on Win32.

make sure the $Id tags are converted to $PostgreSQL as well ...

practice of evaluating MemSet's arguments multiple times, except for
the special case of newNode(), where we can assume the argument is
a constant sizeof() operator.
Also, add GetMemoryChunkContext() to mcxt.c's API, in preparation for
fixing recent GEQO breakage.

away by the compiler;  used by palloc0.

initdb/regression tests pass.

tests pass.

are now separate files "postgres.h" and "postgres_fe.h", which are meant
to be the primary include files for backend .c files and frontend .c files
respectively.  By default, only include files meant for frontend use are
installed into the installation include directory.  There is a new make
target 'make install-all-headers' that adds the whole content of the
src/include tree to the installed fileset, for use by people who want to
develop server-side code without keeping the complete source tree on hand.
Cleaned up a whole lot of crufty and inconsistent header inclusions.

for details).  It doesn't really do that much yet, since there are no
short-term memory contexts in the executor, but the infrastructure is
in place and long-term contexts are handled reasonably.  A few long-
standing bugs have been fixed, such as 'VACUUM; anything' in a single
query string crashing.  Also, out-of-memory is now considered a
recoverable ERROR, not FATAL.
Eliminate a large amount of crufty, now-dead code in and around
memory management.
Fix problem with holding off SIGTRAP, SIGSEGV, etc in postmaster and
backend startup.

  * Portions Copyright (c) 1996-2000, PostgreSQL, Inc

to all files copyright Regents of Berkeley.  Man, that's a lot of files.

error-proof. Rearranged some old code and removed dead sections.

Jan

Another PGINDENT run that changes variable indenting and case label indenting.  Also static variable indenting.