Summary: Most of the approach is copied from WebSQL's MySQL branch. It's nice that we can do this without touching core RocksDB code.

Test Plan: Compiles and runs. Didn't test flashback code, as I don't have flashback device and most if it is c/p

Reviewers: MarkCallaghan, sdong

Reviewed By: sdong

Subscribers: rven, lgalanis, kradhakrishnan, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35391

Summary:
This fixes two bugs: "make clean" would never remove the generated
file, util/build_version.cc, and since D33591, would be regenerated
only if it were absent.
* Makefile (clean): Remove the generated file.
(util/build_version.cc): Depend on the no-prereq FORCE target,
so that this target's rules are always run.
Since this is a generated file, make it read-only.
Also, be sure to remove the temporary file when it is the same
as the original.

Test Plan:
Ensure that we attempt regeneration every time.
Make it empty with an up-to-date time stamp and demonstrate
that it is rebuilt with the expected content:

  $ : > util/build_version.cc
  $ make util/build_version.o
   GEN      util/build_version.cc
   GEN      util/build_version.d
   GEN      util/build_version.cc
   CC       util/build_version.o
  $ cat util/build_version.cc
  #include "build_version.h"
  const char* rocksdb_build_git_sha = "rocksdb_build_git_sha:v3.10-2-gb30e72a";
  const char* rocksdb_build_git_date = "rocksdb_build_git_date:2015-03-27";
  const char* rocksdb_build_compile_date = __DATE__;

Reviewers: igor.sugak, sdong, ljin, igor, rven

Reviewed By: rven

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D36087

Summary:
Assign the string properties to const string variables under the
DB::Properties namespace. This helps catch typos during compilation and
also consolidates the property definition in one place.

Test Plan: Run rocksdb unit tests

Reviewers: sdong, yoshinorim, igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35991

Summary:
Whenever we add new tests in db_sanity_test.cc, the verification test
will fail since the old version db_sanity_test.cc does not have the
newly added test.  This patch makes auto_sanity_test.sh always use
the db_sanity_test.cc of the newer commit.

As a result, a macro guard is added to allow db_sanity_test.cc to be
backward compatible.

Test Plan: tools/auto_sanity_check.sh

Reviewers: sdong, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35997

Fix interger overflow on i386 arch

The error was:

util/logging.cc: In function 'int rocksdb::AppendHumanMicros(uint64_t, char*, int)':
error: util/logging.cc:41:39: integer overflow in expression [-Werror=overflow]
} else if (micros < 1000000l * 60 * 60) {
^
error: util/logging.cc:41:39: comparison between signed and unsigned integer expressions [-Werror=sign-compare]

Summary: It's useful to know if we have compression support or no

Test Plan:
Observed this in my LOG:

      2015/03/26-10:34:35.460681 7f5b322b7840 Snappy supported
      2015/03/26-10:34:35.460682 7f5b322b7840 Zlib supported
      2015/03/26-10:34:35.460686 7f5b322b7840 Bzip supported
      2015/03/26-10:34:35.460687 7f5b322b7840 LZ4 NOT supported

Reviewers: sdong, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35955

fix compilation error (same as fix #284)

[maa@srv2-nskb-devg2 rocksdb-master]$ CXX=/usr/local/CC/gcc-4.7.4/bin/g++ EXTRA_CXXFLAGS=-std=c++11 DISABLE_WARNING_AS_ERROR=1  make db_bench
  CC       db/db_bench.o
db/db_bench.cc: In member function 'rocksdb::Slice rocksdb::Benchmark::AllocateKey(std::unique_ptr<const char []>*)':
db/db_bench.cc:1434:41: error: use of deleted function 'void std::unique_ptr<_Tp [], _Dp>::reset(_Up) [with _Up = char*; _Tp = const char; _Dp = std::default_delete<const char []>]'
In file included from /usr/local/CC/gcc-4.7.4/lib/gcc/x86_64-unknown-linux-gnu/4.7.4/../../../../include/c++/4.7.4/memory:86:0,
                 from ./include/rocksdb/db.h:14,
                 from ./db/dbformat.h:14,
                 from ./db/db_impl.h:21,
                 from db/db_bench.cc:33:

Summary:
* Makefile (COMPILE_WITH_TSAN): Avoid a link failure by disabling
-pg when building with TSAN enabled.
Now that "make check" builds all $(PROGRAMS), it is linking
a few programs that were not normally linked before.
For example, this would fail to link with the following diagnostic:

  COMPILE_WITH_TSAN=1 make -j40 log_and_apply_bench

    CCLD     log_and_apply_bench
  ld: /usr/lib/../lib64/gcrt1.o: relocation R_X86_64_32S against `__libc_csu_fini' can not be used when making a shared object; recompile with -fPIC
  /usr/lib/../lib64/gcrt1.o: error adding symbols: Bad value
  collect2: error: ld returned 1 exit status
  Makefile:511: recipe for target 'log_and_apply_bench' failed
  make: *** [log_and_apply_bench] Error 1

Since removing -pg is sufficient to get past this link
failure, and no one cares about profiling TSAN-enabled
binaries anyway, we will refrain from linking with -pg
when TSAN testing is enabled.  Use a new variable, "pg"
which is set to "-pg" in most cases, but that is made
empty when COMPILE_WITH_TSAN is set.

Test Plan:
  Now, this succeeds:

    rm -f log_and_apply_bench
    COMPILE_WITH_TSAN=1 make -j40 log_and_apply_bench

Reviewers: igor.sugak, rven, sdong, ljin, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35943

Summary:
Add a missing section title in HISTORY.md

Test Plan:
no code change

Summary:
Fixed a typo in RocksDBSample.java

Test Plan:
make clean
make rocksdbjava -j32
make jtest

Summary:
* Makefile (check): Cause "make check" to build all $(PROGRAMS),
so that it verifies that the few benchmark-only source files that
not already built via "make check" do compile and link successfully.

Test Plan:
run "make clean; make check", and verify that
table/table_reader_bench.cc is now compiled.
Before, it was not, which led to an incomplete fix
for a build break.

Reviewers: ljin, igor.sugak, rven, igor, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35883

Test Plan: Running make all

Reviewers: sdong

Reviewed By: sdong

Subscribers: rven, yhchiang, igor, meyering, dhruba

Differential Revision: https://reviews.facebook.net/D35889

Summary: Update HISTORY.md for 3.10.0

Test Plan: no code chagne.

Reviewers: sdong, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35871

Summary: Fixing issues with get context function.

Test Plan: Run make commit-prereq

Reviewers: sdong, meyering, yhchiang

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35853

[RocksJava] Added static lz4 support for roccksjavastatic

[RocksJava] Maven build fix

Summary: Fix compile error when NROCKSDB_THREAD_STATUS is not used.

Test Plan: make dbg OPT=-DNROCKSDB_THREAD_STATUS -j32

Reviewers: sdong, igor, rven

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35847

Summary:
We have addded new stats and perf_context for measuring the merge and filter operation time consumption.
We have bounded all the merge operations within the GUARD statment and collected the total time for these operations in the DB.

Test Plan: WIP

Reviewers: rven, yhchiang, kradhakrishnan, igor, sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D34377

[RocksJava] Expose MemEnv in RocksJava

Usage:
make clean rocksdbjavastatic

Dependency:
LZ4

To install LZ4:
git clone https://github.com/Cyan4973/lz4.git && \
cd lz4/lib && \
make install

Alternativel installation method:
curl -O https://codeload.github.com/Cyan4973/lz4/tar.gz/r127
mv r127 lz4-r127.tar.gz
tar xvzf lz4-r127.tar.gz
cd lz4-r127/lib && \
make install
Signed-off-by: NPooya Shareghi <shareghi@gmail.com>

[RocksJava] Add compression per level to API

Summary:
Report elapsed time of a thread operation in micros in ThreadStatus
instead of start time of a thread operation in seconds since the
Epoch, 1970-01-01 00:00:00 (UTC).

Test Plan:
./db_bench --benchmarks=fillrandom --num=100000 --threads=40 \
--max_background_compactions=10 --max_background_flushes=3 \
--thread_status_per_interval=1000 --key_size=16 --value_size=1000 \
--num_column_families=10

Sample Output:
            ThreadID ThreadType                    cfName    Operation  ElapsedTime                                         Stage        State
     140667724562496   High Pri column_family_name_000002        Flush   772.419 ms                    FlushJob::WriteLevel0Table
     140667728756800   High Pri                   default        Flush   617.845 ms                    FlushJob::WriteLevel0Table
     140667732951104   High Pri column_family_name_000005        Flush   772.078 ms                    FlushJob::WriteLevel0Table
     140667875557440    Low Pri column_family_name_000008   Compaction  1409.216 ms                        CompactionJob::Install
     140667737145408    Low Pri
     140667749728320    Low Pri
     140667816837184    Low Pri column_family_name_000007   Compaction  1071.815 ms      CompactionJob::ProcessKeyValueCompaction
     140667787477056    Low Pri column_family_name_000009   Compaction   772.516 ms      CompactionJob::ProcessKeyValueCompaction
     140667741339712    Low Pri
     140667758116928    Low Pri column_family_name_000004   Compaction   620.739 ms      CompactionJob::ProcessKeyValueCompaction
     140667753922624    Low Pri
     140667842003008    Low Pri column_family_name_000006   Compaction  1260.079 ms      CompactionJob::ProcessKeyValueCompaction
     140667745534016    Low Pri

Reviewers: sdong, igor, rven

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35769

Summary: There is no alternative to GetLiveFiles() function

Test Plan: none

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35805

Summary:
Improve ThreadStatusSingleCompaction in two ways:
1. Use SYNC_POINT to ensure compaction won't happen
   before the test finishes its "Put Phase" instead of
   using sleep.
2. In Put Phase, it continues until we have sufficient
   number of L0 files.  Note that during the put phase,
   there won't be any compaction that consumes L0 files
   because of item 1.

Test Plan: ./db_test  --gtest_filter="*ThreadStatusSingleCompaction*"

Reviewers: sdong, igor, rven

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35727

Summary:
Limiting verbose printing to "command=scan"

Test Plan:
Run make check and manual testing of sst_dump_test

Reviewers: sdong

CC: leveldb

Task ID: #6575982

Blame Rev:

[RocksJava] Add missing JavaDoc annotations

[RocksJava] Java sample bugfix

One of the latest commits broke the sample code. This
resolves the introduced bug.

Extended DBBenchmark with option to enable RocksMemEnv.

Summary:
In 3.10 the C++ code was extended with a MemEnv implementation. This
is now also available in RocksJava.

Changes:
- Extraced abstract super class Env
- Introduced RocksMemEnv
- Remove unnecessary disposeInternal method. The disposal of the default environment is managed by C++ so there needs to be no disposeInternal method in Java.
- Introduced a RocksMemEnvTest, which is aligned with the C++ equivalent.

Test Plan:
make rocksdbjava
make jtest

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35619

Summary:
RocksDB offers the possibility to set different compression types
on a per level basis. This shall be also available using RocksJava.

Test Plan:
make rocksdbjava
make jtest

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35577

[RocksJava] 32-Bit adjustments

[RocksJava] Enhanced Logger comment