Summary: implemented a commond line shell to talk with leveldb thrift server, which is based on state pattern and can be easily extended.

Test Plan: build and run

Reviewers: dhruba, zshao, heyongqiang

Differential Revision: https://reviews.facebook.net/D4713

Introduce a new option disableDataSync for opening the database. If this is set to true, then the data written to newly created data files are not sycned to disk, instead depend on the OS to flush dirty data to stable storage. This option is good for bulk

Test Plan:
manual tests

Task ID: #

Blame Rev:

Differential Revision: https://reviews.facebook.net/D4515

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Move thrift api  from thrift/leveldb.thrift to thrift/if/leveldb.thrift

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

bits_per_key is already configurable. It defines how many bloom bits will be used for every key in the database.

My change in this patch is to make the Hash code that is used for blooms to be confgurable. In fact,
one can specify a modified HashCode that inspects only parts of the Key to generate the Hash (used
by booms).

Test Plan: none

Differential Revision: https://reviews.facebook.net/D4059

Summary:
First draft.
Unit tests pass.

Test Plan: unit tests attached

Reviewers: heyongqiang

Reviewed By: heyongqiang

Differential Revision: https://reviews.facebook.net/D3969

Summary: as subject. The flush will flush everything in the db.

Test Plan: new test in db_test.cc

Reviewers: dhruba

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D4029

Summary: add disable WAL option

Test Plan: new testcase in db_test.cc

Reviewers: dhruba

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D4011

Summary: as subject

Test Plan: compile

Reviewers: dhruba

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D3957

Summary: add bzip2 compression

Test Plan: testcases in table_test

Reviewers: dhruba

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D3909

Summary: add zlib compression

Test Plan: Will add more testcases

Reviewers: dhruba

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D3873

Summary:
Make configurable 'targetFileSize', 'targetFileSizeMultiplier',
'maxBytesForLevelBase', 'maxBytesForLevelMultiplier',
'expandedCompactionFactor', 'maxGrandParentOverlapFactor'

Test Plan: N/A

Reviewers: dhruba

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D3801

Test Plan: unit tests.

Differential Revision: https://reviews.facebook.net/D3735

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Summary:

Test Plan:

Reviewers:

CC:

Task ID: #

Blame Rev:

Test Plan: Run benchmark

Differential Revision: https://reviews.facebook.net/D3687

Make Leveldb save data into HDFS files. You have to set USE_HDFS in your environment variable to compile leveldb with HDFS support.

Test Plan: Run benchmark.

Differential Revision: https://reviews.facebook.net/D3549

Summary:
Added option --writes=xxx to specify the number of keys that we want to overwrite in the benchmark.

Task ID: #

Blame Rev:

Test Plan: Revert Plan:

Reviewers: adsharma

CC: sc

Differential Revision: https://reviews.facebook.net/D3465

Summary:
The db_bench test was not using the specified value for the max-file-open. Fixed.

The fs readhead is switched off.

Gather statistics about the table cache and print it out at the end of the tets run.

Test Plan: Revert Plan:

Reviewers: adsharma, sc

Reviewed By: adsharma

Differential Revision: https://reviews.facebook.net/D3441

Support --bufferedio=[0,1] from db_bench. If bufferedio = 0, then the read code path clears the OS page cache after the IO is completed. The default remains as bufferedio=1

Summary:
Task ID: #

Blame Rev:

Test Plan: Revert Plan:

Differential Revision: https://reviews.facebook.net/D3429

Ability to make the benchmark issue a large number of IOs. This is helpful to populate many gigabytes of data for benchmarking at scale.

Summary:
Task ID: #

Blame Rev:

Test Plan: Revert Plan:

Differential Revision: https://reviews.facebook.net/D3333

Summary:
Task ID: #

Blame Rev:

Test Plan: Revert Plan:

Differential Revision: https://reviews.facebook.net/D3309

Add support to specify the number of shards for the Block cache. By default, the block cache is sharded into 16 parts.

Summary:
Task ID: #

Blame Rev:

Test Plan: Revert Plan:

Differential Revision: https://reviews.facebook.net/D3273

Summary:
This speeds up CRC computation significantly on
hardware that supports it. Enabled via -msse4.

Note: the binary won't be usable on older CPUs
that don't support the instruction.

Test Plan: crc32c_test

Reviewers: dhruba

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D3201

Summary:
Task ID: #

Blame Rev:

Test Plan: Revert Plan:

Reviewers: chip

Reviewed By: chip

CC: sc, adsharma

Differential Revision: https://reviews.facebook.net/D3225

Summary:
Some code reorganization in-preparation for replacing with a hardware
instruction.

* Use u64 for some of the key types
* Use an ALIGN macro so code is easier to read

Test Plan: crc32c_test

Reviewers: dhruba

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D3135

Summary:
Task ID: #

Blame Rev:

Test Plan: Revert Plan:

Differential Revision: https://reviews.facebook.net/D3147

Summary:
skiplist doesn't cache the location of the last insert and becomes
CPU bound when the input data has sequential keys.

Notes on thread safety: ::Insert() already requires external
synchronization. So this change is not making it any worse.

Test Plan: skiplist_test

Reviewers: dhruba

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D3129

Summary:
Task ID: #

Blame Rev:

Test Plan: Revert Plan:

Differential Revision: https://reviews.facebook.net/D3105