Summary: Remove the extra guard to allow G1 to use ReduceInitialCardMarks
Reviewed-by: jmasa, tonyp, johnc, ysr

Summary: Fixed a few inconsistencies in the monitoring data, in particular when reported from jstat.
Reviewed-by: jmasa, brutisso, johnc

Summary: G1 now uses two reference processors - one is used by concurrent marking and the other is used by STW GCs (both full and incremental evacuation pauses). In an evacuation pause, the reference processor is embedded into the closures used to scan objects. Doing so causes causes reference objects to be 'discovered' by the reference processor. At the end of the evacuation pause, these discovered reference objects are processed - preserving (and copying) referent objects (and their reachable graphs) as appropriate.
Reviewed-by: ysr, jwilhelm, brutisso, stefank, tonyp

Summary: As a result of the changes for 7080389, an evacuation failure during an initial mark pause may result in some root objects not being marked. Pass whether the caller is a root scanning closure into the evacuation failure handling code so that the thread that successfully forwards an object to itself also marks the object.
Reviewed-by: ysr, brutisso, tonyp

Summary: Refactor code marking code in the evacuation pause copy closures so that an evacuated object is only marked by the thread that successfully copies it.
Reviewed-by: stefank, brutisso, tonyp

Summary: Removed the possibility to turn off generational mode for G1.
Reviewed-by: johnc, ysr, tonyp

Summary: Refactor the allocation code during GC to use the G1AllocRegion abstraction. Use separate subclasses of G1AllocRegion for survivor and old regions. Avoid BOT updates and dirty survivor cards incrementally for the former.
Reviewed-by: brutisso, johnc, ysr

Summary: In root processing, move the scanning of the reference processor's discovered lists to before RSet updating and scanning. When scanning the reference processor's discovered lists, use a buffering closure so that the time spent copying any reference object is correctly attributed. Also removed a couple of unused and irrelevant timers.
Reviewed-by: ysr, jmasa

Summary: Extend and make more consistent the output from the G1PrintHeapRegions flag.
Reviewed-by: johnc, jmasa

Summary: Remove two unnecessary iterations over the collection set which are supposed to prepare the RSet's of the CSet regions for parallel iterations (we'll make sure this is done incrementally). I'll piggyback on this CR the removal of the G1_REM_SET_LOGGING code.
Reviewed-by: brutisso, johnc

Summary: Remove two unused parameters from the mem_allocate() method and update its uses accordingly.
Reviewed-by: stefank, johnc

Summary: Perform a heap verification after the first phase of G1's full GC using objects' mark words to determine liveness. The third parameter of the heap verification routines, which was used in G1 to determine which marking bitmap to use in liveness calculations, has been changed from a boolean to an enum with values defined for using the mark word, and the 'prev' and 'next' bitmaps.
Reviewed-by: tonyp, ysr

7042285: G1: native memory leak during humongous object allocation
6804436: G1: heap region indices should be size_t
Summary: A series of fixes and improvements to the HeapRegionSeq class: a) replace the _regions growable array with a standard C array, b) avoid de-allocating / re-allocating HeapRegion instances when the heap shrinks / grows (fix for 7042285), c) introduce fast method to map address to HeapRegion via a "biased" array pointer, d) embed the _hrs object in G1CollectedHeap, instead of pointing to it via an indirection, e) assume that all the regions added to the HeapRegionSeq instance are contiguous, f) replace int's with size_t's for indexes (and expand that to HeapRegion as part of 6804436), g) remove unnecessary / unused methods, h) rename a couple of fields (_alloc_search_start and _seq_bottom), i) fix iterate_from() not to always start from index 0 irrespective of the region passed to it, j) add a verification method to check the HeapRegionSeq assumptions, k) always call the wrappers for _hrs.iterate(), _hrs_length(), and _hrs.at() from G1CollectedHeap, not those methods directly, and l) unify the code that expands the sequence (by either re-using or creating a new HeapRegion) and make it robust wrt to a HeapRegion allocation failing.
Reviewed-by: stefank, johnc, brutisso

Summary: This changeset extends the logging information generated by +PrintGCDetails to also print out separate size transitions for the eden, survivors, and old regions.
Reviewed-by: ysr, brutisso

Summary: Implement a more accurate is_scavengable()
Reviewed-by: stefank, jcoomes, ysr

7035144: G1: nightly failure: Non-dirty cards in region that should be dirty (failures still exist...)
Summary: We should only undirty cards after we decide that they are not on a young region, not before. The fix also includes improvements to the verify_dirty_region() method which print out which cards were not found dirty.
Reviewed-by: johnc, brutisso

Summary: Added counters for jstat
Reviewed-by: tonyp, jwilhelm, stefank, ysr, johnc

Summary: It introduces non-product cmd line parameter G1DummyRegionsPerGC which indicates how many "dummy" regions to allocate at the end of each GC. This allows the G1 heap to grow artificially and makes concurrent marking cycles more frequent irrespective of what the application that is running is doing. The dummy regions will be found totally empty during cleanup so this parameter can also be used to stress the concurrent cleanup operation.
Reviewed-by: brutisso, johnc

Summary: Some G1 code cleanup.
Reviewed-by: johnc, jcoomes, jwilhelm

Summary: Make the G1 specific version of is_in_closed_subset() available in all builds.
Reviewed-by: tonyp, jcoomes

Summary: Repurpose the existing flag G1PrintRegionLivenessInfo to print out the liveness distribution across the regions in the heap at the end of marking.
Reviewed-by: iveresov, jwilhelm

7023151: G1: refactor the code that operates on _cur_alloc_region to be re-used for allocs by the GC threads
7018286: G1: humongous allocation attempts should take the GC locker into account
Summary: First, this change replaces the asymmetric locking scheme in the G1 slow alloc path by a summetric one. Second, it factors out the code that operates on _cur_alloc_region so that it can be re-used for allocations by the GC threads in the future.
Reviewed-by: stefank, brutisso, johnc

Summary: What the synopsis says.
Reviewed-by: jwilhelm, iveresov, johnc

7007446: G1: expand the heap with a single step, not one region at a time
Summary: Changed G1CollectedHeap::expand() to expand the committed space by calling VirtualSpace::expand_by() once rather than for every region in the expansion amount. This allows the success or failure of the expansion to be determined before creating any heap regions. Introduced a develop flag G1ExitOnExpansionFailure (false by default) that, when true, will exit the VM if the expansion of the committed space fails. Finally G1CollectedHeap::expand() returns a status back to it's caller so that the caller knows whether to attempt the allocation.
Reviewed-by: brutisso, tonyp

Summary: A race between the concurrent cleanup thread and the VM thread while it is processing the "expanded sparse table list" causes both threads to try to free the same sparse table entry and either causes one of the threads to fail or leaves the entry in an inconsistent state. The solution is purge all entries on the expanded list that correspond go regions that are being cleaned up.
Reviewed-by: brutisso, johnc

Summary: This changeset removes the zero-filling thread from G1 and collapses the two free region lists we had before (the "free" and "unclean" lists) into one. The new free list uses the new heap region sets / lists abstractions that we'll ultimately use it to keep track of all regions in the heap. A heap region set was also introduced for the humongous regions. Finally, this change increases the concurrency between the thread that completes freeing regions (after a cleanup pause) and the rest of the system (before we'd have to wait for said thread to complete before allocating a new region). The changest also includes a lot of refactoring and code simplification.
Reviewed-by: jcoomes, johnc

Summary: First attempt to allocate out the current alloc region using a CAS instead of taking the Heap_lock (first level of G1's slow allocation path). Only if that fails and it's necessary to replace the current alloc region take the Heap_lock (that's the second level of G1's slow allocation path).
Reviewed-by: johnc, brutisso, ysr

Summary: During evacuation failure handling we refine the BOT to reflect the location of all the objects in the regions we scan. The changeset includes some minor cleanup: a) non-product print_on() method on the G1 BOT class, b) added more complete BOT verification during heap / region verification, c) slight modification to the BOT set up for humongous regions to be more consistent with the BOT set up during evac failure handling, and d) removed a couple of unused methods.
Reviewed-by: johnc, ysr

Summary: Initialize the _is_alive_non_header field of G1's reference processor with an instance of the G1CMIsAliveClosure. This will stop adding reference objects with live referents to the discovered reference lists unnecessarily.
Reviewed-by: tonyp, ysr, jwilhelm, brutisso

7000559: G1: assertion failure !outer || (full_collections_started == _full_collections_completed + 1)
Summary: The concurrent marking thread can complete its operation and increment the full GC counter during a Full GC. This causes the nesting of increments to the start and end of Full GCs that we are expecting to be wrong. the fix is for the marking thread to join the suspendible thread set before incrementing the counter so that it's blocked until the Full GC (or any other safepoint) is finished. The change also includes some minor code cleanup (I renamed a parameter).
Reviewed-by: brutisso, ysr

Summary: Allow the eden to the expanded up to a point when the GC locker is active.
Reviewed-by: jwilhelm, johnc, ysr, jcoomes

Summary: Replaced MakeDeps and the includeDB files with more standardized solutions.
Reviewed-by: coleenp, kvn, kamg

Summary: When an entry is stolen, don't push it on the task queue but process it directly.
Reviewed-by: iveresov, ysr, jcoomes

Reviewed-by: iveresov, tonyp

6942771: SEGV in ParScanThreadState::take_from_overflow_stack
Reviewed-by: apetrusenko, ysr, pbk

Summary: Associate number of GC workers with the workgang as opposed to the task.
Reviewed-by: johnc, ysr

Summary: This change revamps the slow allocation path of G1. Improvements include the following: a) Allocations directly to old regions are now totally banned. G1 now only allows allocations out of young regions (with the only exception being humongous regions). b) The thread that allocates a new region (which is now guaranteed to be young) does not dirty all its cards. Each thread that successfully allocates out of a young region is now responsible for dirtying the cards that corresponding to the "block" that just got allocated. c) allocate_new_tlab() and mem_allocate() are now implemented differently and TLAB allocations are only done by allocate_new_tlab(). d) If a thread schedules an evacuation pause in order to satisfy an allocation request, it will perform the allocation at the end of the safepoint so that the thread that initiated the GC also gets "first pick" of any space made available by the GC. e) If a thread is unable to allocate a humongous object it will schedule an evacuation pause in case it reclaims enough regions so that the humongous allocation can be satisfied aftewards. f) The G1 policy is more careful to set the young list target length to be the survivor number +1. g) Lots of code tidy up, removal, refactoring to make future changes easier.
Reviewed-by: johnc, ysr

Summary: as the title says, sometimes we are allocating humongous objects in young regions and we shouldn't.
Reviewed-by: ysr, johnc

Reviewed-by: tonyp, ysr

Summary: The per-worker _new_refs array is used to hold references that point into the collection set. It is populated during RSet updating and subsequently processed. In the event of an evacuation failure it processed again to recreate the RSets of regions in the collection set. Remove the per-worker _new_refs array by processing the references directly. Use a DirtyCardQueue to hold the cards containing the references so that the RSets of regions in the collection set can be recreated when handling an evacuation failure.
Reviewed-by: iveresov, jmasa, tonyp