Summary: There is a high number of mispredicted branches associated with calling BitMap::iteratate() from within CMBitMapRO::iterate(). Implement a version of CMBitMapRO::iterate() directly using inline-able routines.
Reviewed-by: tonyp, iveresov

Summary: This change simplifies the interaction between GC and concurrent marking. By disabling survivor spaces during the initial-mark pause we don't need to propagate marks of objects we copy during each GC (since we never need to copy an explicitly marked object).
Reviewed-by: johnc, brutisso

Summary: Replaced the code with a ShouldNotReachHere
Reviewed-by: tonyp, jmasa

Summary: Parallelize the removal of self forwarding pointers etc. by wrapping in a HeapRegion closure, which is then wrapped inside an AbstractGangTask.
Reviewed-by: tonyp, iveresov

Summary: Running G1 with ParallelGCThreads==0 results in various crashes and asserts. Most of these are caused by unguarded references to the worker threads array or an incorrect number of active workers.
Reviewed-by: jmasa, tonyp

Summary: Change variables representing the number of GC workers to uint from int and size_t.  Change the parameter in work(int i) to work(uint worker_id).
Reviewed-by: brutisso, tonyp

Summary: Parallelize the serial code that was used to mark objects reachable from survivor objects in the collection set. Some minor improvments in the timers used to track the freeing of the collection set along with some tweaks to PrintGCDetails.
Reviewed-by: tonyp, brutisso

Summary: Keep track of all the old regions in the heap with a heap region set.
Reviewed-by: brutisso, johnc

Summary: Removed unused fields and methods, removed the G1CollectoryPolicy_BestRegionsFirst class and folded its functionality into the G1CollectorPolicy class.
Reviewed-by: ysr, brutisso, jcoomes

Summary: Declare GrainBytes, GrainWords, and CardsPerRegion as size_t.
Reviewed-by: jcoomes, tonyp, jmasa

Summary: G1 now uses two reference processors - one is used by concurrent marking and the other is used by STW GCs (both full and incremental evacuation pauses). In an evacuation pause, the reference processor is embedded into the closures used to scan objects. Doing so causes causes reference objects to be 'discovered' by the reference processor. At the end of the evacuation pause, these discovered reference objects are processed - preserving (and copying) referent objects (and their reachable graphs) as appropriate.
Reviewed-by: ysr, jwilhelm, brutisso, stefank, tonyp

Summary: Cast HeapRegion::GrainBytes to size_t in output statement.
Reviewed-by: ysr, brutisso, pbk, tonyp

Summary: Displays the elapsed time taken to perform reference processing during remark as part of the PrintGCDetails output.
Reviewed-by: ysr

Summary: It introduces ergonomic decision logging in G1 for the following heuristics: heap sizing, collection set construction, concurrent cycle initiation, and partially-young GC start/end. The code has a bit of refactoring in a few places to make the decision logging possible. It also replaces alternative ad-hoc logging that we have under different parameters and switches (G1_DEBUG, G1PolicyVerbose).
Reviewed-by: johnc, ysr

Summary: Removed the possibility to turn off generational mode for G1.
Reviewed-by: johnc, ysr, tonyp

Summary: Select number of GC threads dynamically based on heap usage and number of Java threads
Reviewed-by: johnc, ysr, jcoomes

Summary: Various cleanups and formatting changes identified during a code walk through of the changes for 6484965 ("G1: piggy-back liveness accounting phase on marking").
Reviewed-by: brutisso, tonyp

Summary: Extend and make more consistent the output from the G1PrintHeapRegions flag.
Reviewed-by: johnc, jmasa

Summary: Only cosmetic changes to make the concurrentMark.* more consistent, code-style-wise, with the rest of the codebase.
Reviewed-by: johnc, ysr

Summary: Perform a heap verification after the first phase of G1's full GC using objects' mark words to determine liveness. The third parameter of the heap verification routines, which was used in G1 to determine which marking bitmap to use in liveness calculations, has been changed from a boolean to an enum with values defined for using the mark word, and the 'prev' and 'next' bitmaps.
Reviewed-by: tonyp, ysr

Summary: Some optimizations to improve the concurrent marking phase: specialize the main oop closure, make sure a few methods in the fast path are properly inlined, a few more bits and pieces, and some cosmetic fixes.
Reviewed-by: stefank, johnc

Summary: During an evacuation pause clear the region fields of any concurrent marking task whose local finger points into the collection set as the values in the region fields will become stale. Clearing these fields causes the concurrent mark task to claim a new region when marking restarts after the pause.
Reviewed-by: tonyp, iveresov

7035144: G1: nightly failure: Non-dirty cards in region that should be dirty (failures still exist...)
Summary: We should only undirty cards after we decide that they are not on a young region, not before. The fix also includes improvements to the verify_dirty_region() method which print out which cards were not found dirty.
Reviewed-by: johnc, brutisso

7034139: G1: assert(Thread::current()->is_ConcurrentGC_thread()) failed: only a conc GC thread can call this.
Summary: We were calling STS join and leave during a STW pause and we are not suppoesed to. I now only call those during  concurrent phase. I also added stress code in the non-product builds to force an overflows (the condition that ws uncovering the bug) to make sure it does not happen again.
Reviewed-by: johnc, brutisso

Summary: Use _max_num_q = max(discovery_degree, processing_degree), and let balance_queues() redistribute from discovery_degree to processing_degree of queues. This should also allow a more dynamic and flexible parallelism policy in the future.
Reviewed-by: jmasa, johnc

Summary: Some G1 code cleanup.
Reviewed-by: johnc, jcoomes, jwilhelm

Summary: Disable reference discovery and processing during concurrent marking by disabling fix for 6994628.
Reviewed-by: tonyp, ysr

Summary: Repurpose the existing flag G1PrintRegionLivenessInfo to print out the liveness distribution across the regions in the heap at the end of marking.
Reviewed-by: iveresov, jwilhelm

Summary: move symbols from permgen into C heap and reference count them
Reviewed-by: never, acorn, jmasa, stefank

Summary: A race between the concurrent cleanup thread and the VM thread while it is processing the "expanded sparse table list" causes both threads to try to free the same sparse table entry and either causes one of the threads to fail or leaves the entry in an inconsistent state. The solution is purge all entries on the expanded list that correspond go regions that are being cleaned up.
Reviewed-by: brutisso, johnc

Summary: Implement support for ParallelRefProcEnabled in the reference processing that takes place at the end of G1 concurrent marking.
Reviewed-by: tonyp, ysr

Summary: This changeset removes the zero-filling thread from G1 and collapses the two free region lists we had before (the "free" and "unclean" lists) into one. The new free list uses the new heap region sets / lists abstractions that we'll ultimately use it to keep track of all regions in the heap. A heap region set was also introduced for the humongous regions. Finally, this change increases the concurrency between the thread that completes freeing regions (after a cleanup pause) and the rest of the system (before we'd have to wait for said thread to complete before allocating a new region). The changest also includes a lot of refactoring and code simplification.
Reviewed-by: jcoomes, johnc

Summary: This changeset introduces filtering of SATB buffers at the point when they are about to be enqueued. If this filtering clears enough entries on each buffer, the buffer can then be re-used and not enqueued. This cuts down the number of SATB buffers that need to be processed by the concurrent marking threads.
Reviewed-by: johnc, ysr

Summary: Make JvmtiGCMark safe to run non-safepoint and instrument CMS
Reviewed-by: ysr, dcubed

Summary: Initialize the _is_alive_non_header field of G1's reference processor with an instance of the G1CMIsAliveClosure. This will stop adding reference objects with live referents to the discovered reference lists unnecessarily.
Reviewed-by: tonyp, ysr, jwilhelm, brutisso

Summary: Enable reference discovery during concurrent marking by setting the reference processor field of the concurrent marking closure. Keep reference objects on the discovered reference lists alive during incremental evacuation pauses until they are processed at the end of concurrent marking.
Reviewed-by: ysr, tonyp

Summary: Replaced MakeDeps and the includeDB files with more standardized solutions.
Reviewed-by: coleenp, kvn, kamg

6988458: G1: assert(mr.end() <= _cm->finger()) failed: otherwise the region shouldn't be on the stack
Summary: The changes from 6941395 did not clear the CMTask::_aborted_region fields when concurrent marking aborted because of overflow. As a result, the next time around we could see a memory region whose start address was above the global finger and the assertion tripped. Moved the clearing of the aborted regions to ConcurrentMark::clear_marking_state, which is executed on all of the exit paths.
Reviewed-by: tonyp, ysr, jmasa

Summary: Re-enable use of the lock-free versions of region stack push() and pop() by recording aborted regions in a thread-local structure, which are then processed when scanning of the region stack restarts. The previous locking versions of these routines are retained for diagnostic purposes.
Reviewed-by: tonyp, ysr

Summary: Associate number of GC workers with the workgang as opposed to the task.
Reviewed-by: johnc, ysr