It's now possible to set a LogLevel in Options and
DBOptions to control LOG verbosity.

[RocksJava] Test framework support

+ 7% coverage + 3% branch coverage

Summary:
As we had the discussion some weeks ago. Java needs a test framework and should support code coverage analysis. This pull request includes:

    Move Tests from main method functionality to Junit4
    Move WriteBatchTest to test package
    Adjust the Makefile to run Junit4
    Download dependencies from Make (once if not-present)
    Adjustment of the rocksjni.pom to run coverage analysis using jacoco
    Javadoc excludes now tests
    Two bugfixes regarding GC cleanup which came up within the test runs

Make can be used as beforehand to build and run RocksJava. make test runs tests using the command-line version of Junit4.

Maven can be used to retrieve code coverage reports using mvn -f rocksjni.pom package. Code coverage reports can then be found as usual in the site folder.

Testing libraries available within Java

    Junit4 (incl. hamcrest-core dependency)
    AssertJ (providing fluent syntax for assertions, cglib dependency)
    Mockito to provide mocktests

Libraries as said before are not statically within this commit or filesystem instead they are downloaded using curl. Make checks if files are present, if so it will perform tests without downloading the libraries again.

Note: Libraries are only necessary to compile & run tests.

Next steps after merge:

    Get the maven build into travis-ci and coveralls.io
    Filling up the missing test spots (based on coverage data)

Test Plan:
make rocksdbjava
make jtest

Reviewers: yhchiang, ankgup87, adamretter

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D28209

Summary: Store links to live files in directory on same disk

Test Plan:
Take snapshot and open it. Added a test GetSnapshotLink in
db_test.

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28713

Summary:
This is just a simple test that passes two files though a compaction. It shows the framework so that people can continue building new compaction *unit* tests.
In the future we might want to move some Compaction* tests from DBTest here. For example, CompactBetweenSnapshot seems a good candidate.

Hopefully this test can be simpler when we mock out VersionSet.

Test Plan: this is a test

Reviewers: ljin, rven, yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28449

Relax the block count check on deallocation in env_test

Summary:
Make db_stress built for ROCKSDB_LITE.
The test doesn't pass tough. It seg fault quickly. But I took a look and it doesn't seem to be related to lite version. Likely to be a bug inside RocksDB.

Test Plan: make db_stress

Reviewers: yhchiang, rven, ljin, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D28797

It seems that on some FS we get more blocks than we ask for. This is
already handled when checking the allocated number of blocks, but
after the file is closed it checks for an exact number of blocks,
which fails on my machine.

I changed the test to add one full page to the size, then calculate
the expected number of blocks and check if the actual number of blocks
is less or equal to that.

Summary: Add a unit test in db_test to verify the behavior when both of merge operator and compaction filter apply to a key when merging.

Test Plan: Run the new test

Reviewers: ljin, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28455

Summary: As a short-term fix, let's go back to previous way of calculating NeedsCompaction(). SIGSEGV happens because NeedsCompaction() can happen before super_version (and thus MutableCFOptions) is initialized.

Test Plan: make check

Reviewers: ljin, sdong, rven, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28875

Summary:
CorruptionTest for backupable_db_test did not call
GarbageCollect() after deleting a corrupt backup,
which sometimes lead to test failures as the newly created backup
would reuse the same backup ID and files and fail the consistency
check.

Moved around some of the test logic to ensure that GarbageCollect()
is called at the right time.

Test Plan:
Run backupable_db_test eight times and make sure
it passes repeatedly. Also run make check to make sure other
tests don't fail.

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28863

Summary: It adds lots of code.

Test Plan: compile for iOS, compile for mac. works.

Reviewers: rven, sdong, ljin, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28857

Summary:
Move NeedsCompaction() from VersionStorageInfo to CompactionPicker
to allow different compaction strategy to have their own way to
determine whether doing compaction is necessary.

When compaction style is set to kCompactionStyleNone, then
NeedsCompaction() will always return false.

Test Plan:
export ROCKSDB_TESTS=Compact
./db_test

Reviewers: ljin, sdong, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28719

Summary: This will speed up our import times

Test Plan: Added simple unit test just to get code coverage

Reviewers: sdong, ljin, yhchiang, rven, mohaps

Reviewed By: mohaps

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28869

Summary: So iOS size_t is 32-bit, so we need to static_cast<size_t> any uint64_t :(

Test Plan: TARGET_OS=IOS make static_lib

Reviewers: dhruba, ljin, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28743

Summary: I found that db_stress sometimes segfault on my machine. Fix the bug.

Test Plan: make all check. Run db_stress

Reviewers: ljin, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D28803

Summary:
Improve the backup engine by not deleting the corrupted
backup when it is detected; instead leaving it to the client
to delete the corrupted backup.

Also add a BackupEngine::Open() call.

Test Plan:
Add check to CorruptionTest inside backupable_db_test
to check that the corrupt backups are not deleted. The previous
version of the code failed this test as backups were deleted,
but after the changes in this commit, this test passes.

Run make check to ensure that no other tests fail.

Reviewers: sdong, benj, sanketh, sumeet, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28521

[RocksJava] Cleanup 

[RocksJava] ColumnFamilyOptions extension 3.6

Java8 is more restrictive than Java7 with generating
JavaDocs. This commit resolves current existing Java8
warnings.

- RocksDB ColumnFamilySample adjusted to C++ sample.
- DefaultColumnFamily is available now as constant in RocksDB.