Summary: as title

Test Plan: as part 1

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27789

Summary: as title

Test Plan: same as part 1

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27705

Summary: as title

Test Plan: same as part 1

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27693

Summary:
DBTest has several functions (Reopen(), TryReopen(), ChangeOptins(), etc
that takes a pointer to options), depending on if it is nullptr, it uses
different options underneath. This makes it really hard to track what
options is used in different test case. We should just kill the default
value and make it being passed into explicitly. It is going to be very
hairy. I will start with simple ones.

Test Plan:
make db_test
stacked diffs, will run test with full stack

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27687

Summary:
Remove an unnecessary include file in version_edit.cc

Summary:
This diff replaces BlockBasedTable in flush_job_test with TableMock, making it depend on less things and making it closer to an unit test than integration test.

It also introduces a framework to compile mock classes -- Any file named *mock.cc will not be compiled into the build. It will only get compiled into the tests. What way we can mock out most other classes, Version, VersionSet, DBImpl, etc.

Test Plan: flush_job_test

Reviewers: ljin, rven, yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27681

Summary:
Improve the robustness of PartialCompactionFailure test again.

Test Plan:
./db_test

Summary:
Improve the robustness of PartialCompactionFailure test.

Test Plan:
./db_test

Summary:
This diff has two fixes.

1. Fix the bug where compaction does not fail when RocksDB can't create a new file.
2. When NewWritableFiles() fails in OpenCompactionOutputFiles(), previously such fail-to-created file will be still be included as a compaction output.  This patch also fixes this bug.
3. Allow VersionEdit::EncodeTo() to return Status and add basic check.

Test Plan:
./version_edit_test
export ROCKSDB_TESTS=FileCreationRandomFailure
./db_test

Reviewers: ljin, sdong, nkg-, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D25581

[RocksJava] - BackupInfos & Restore-/BackupableDB enhancements

Summary:
Abstract out FlushProcess and take it out of DBImpl.
This also includes taking DeletionState outside of DBImpl.

Currently this diff is only doing the refactoring. Future work includes:
1. Decoupling flush_process.cc, make it depend on less state
2. Write flush_process_test, which will mock out everything that FlushProcess depends on and test it in isolation

Test Plan: make check

Reviewers: rven, yhchiang, sdong, ljin

Reviewed By: ljin

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27561

Summary: No need to expose them in .h

Test Plan: make release

Reviewers: igor, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27645

Parameter types for BackupID are now aligned to int.

Summary:
- BackupableDB deleteBackup method
- BackupableDB purgeOldBackups bugfix
- BackupInfos now available in Restorable-/BackupableDB
- Extended BackupableDBTest to cover more of the currently implemented functionality.

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D27027

Summary: as title

Test Plan:
make release
will run full test on all stacked diffs

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27597

Summary: as title

Test Plan:
make release
will run full test on all stacked diffs before committing

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27591

Summary:
We have several different types of data structures for file information.
FileLevel is kinda of confusing since it only contains file range and
fd. Rename it to LevelFilesBrief to make it clear.
Unfriend CompactedDBImpl as a by product

Test Plan:
make release / make all
will run full test with all stacked diffs

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27585

Summary: as title

Test Plan:
make release
I will run make all check for all stacked diffs before commit

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27573

Summary: as title

Test Plan: running make all check

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27549

[RocksJava] Minor correction to the previous pull request merge

Added merge operators to RocksJava

[RocksJava] - Hardening RocksIterator

RocksIterator will sometimes Sigsegv on dispose. Mainly thats related
to dispose order. If the related RocksDB instance is freed beforehand
RocksIterator.dispose() will fail.

Within this commit there is a major change to RocksIterator. RocksIterator
will hold a private reference to the RocksDB instance which created the
RocksIterator. So even if RocksDB is freed in the same GC cycle the
RocksIterator instances will be freed prior to related RocksDB instances.

Another aspect targets the dispose logic if the RocksDB is freed previously
and already gc`ed. On dispose of a RocksIterator the dispose logic will check
if the RocksDB instance points to an initialized DB. If not the dispose logic
will not perform any further action.

The crash can be reproduced by using the related test provided within this
commit.

Related information: This relates to @adamretter`s facebook rocksdb-dev group
post about SigSegv on RocksIterator.dispose().

[RocksJava] Support Snapshots

Summary: RocksDB already depends on C++11, so we might as well all the goodness that C++11 provides. This means that we don't need AtomicPointer anymore. The less things in port/, the easier it will be to port to other platforms.

Test Plan: make check + careful visual review verifying that NoBarried got memory_order_relaxed, while Acquire/Release methods got memory_order_acquire and memory_order_release

Reviewers: rven, yhchiang, ljin, sdong

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D27543

RocksJava Fix after MutableCFOptions change. (relates to f1841985)

Summary: Allow SetOptions() during db_stress test

Test Plan: make crash_test

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25497

Summary:
Make inplace_update_support and inplace_update_num_locks dynamic.
inplace_callback becomes immutable
We are almost free of references to cfd->options() in db_impl

Test Plan: unit test

Reviewers: igor, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25293

Summary:
Snapshots integration into RocksJava. Added support for the following functionalities:

- getSnapshot
- releaseSnapshot
- ReadOptions support to set a Snapshot
- ReadOptions support to retrieve Snapshot
- SnapshotTest

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D24801