- 26 11月, 2013 10 次提交
-
-
由 Kai Liu 提交于
Added missing component : gflags in Linux platform. Thank you Kangmo!
-
由 Kangmo Kim 提交于
Added missing component : gflags in Linux platform.
-
由 Igor Canadi 提交于
Summary: Check https://github.com/facebook/rocksdb/pull/15 for context. Apparently [1], we need -fPIC in x64 environments (this is added only in non-fbcode). In fbcode, I removed -fPIC per @dhruba's suggestion, since it introduces perf regression. I'm not sure what would are the implications of doing that, but looks like it works, and when releasing to the third-party, we're disabling -fPIC either way [2]. Would love a suggestion from someone who knows more about this [1] http://eli.thegreenplace.net/2011/11/11/position-independent-code-pic-in-shared-libraries-on-x64/ [2] https://our.intern.facebook.com/intern/wiki/index.php/Database/RocksDB/Third_Party Test Plan: make check works Reviewers: dhruba, emayanke, kailiu Reviewed By: dhruba CC: leveldb, dhruba, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14337
-
由 Dhruba Borthakur 提交于
Summary: Large memory allocations and frees are costly and best done outside the db-mutex. The memtables are already allocated outside the db-mutex but they were being freed while holding the db-mutex. This patch frees obsolete memtables outside the db-mutex. Test Plan: make check db_stress Unit tests pass, I am in the process of running stress tests. Reviewers: haobo, igor, emayanke Reviewed By: haobo CC: reconnect.grayhat, leveldb Differential Revision: https://reviews.facebook.net/D14319
-
由 Igor Canadi 提交于
Summary: We need access to options for BackupableDB Test Plan: make check Reviewers: dhruba Reviewed By: dhruba CC: leveldb, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14331
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: The old regression tests didn't cover memtable part at all. This is an atempt to also measure memtable performance in regression tests. Test Plan: Ran regression_build_test.sh Reviewers: dhruba, haobo, kailiu, emayanke Reviewed By: dhruba CC: leveldb, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14325
-
由 Igor Canadi 提交于
Summary: This is part of https://reviews.facebook.net/D14295 -- smaller diff that is easier to review Test Plan: make asan_check Reviewers: dhruba, haobo, emayanke Reviewed By: emayanke CC: leveldb, kailiu, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14301
-
由 Dhruba Borthakur 提交于
Summary: All filesystem Io should be done outside the dbmutex. There was one place when we have to roll the transaction log that we were creating the new log file while holding the dbmutex. I rearranged this code so that the act of creating the new transaction log file is done without holding the dbmutex. I also allocate the new memtable outside the dbmutex, this is important because creating the memtable could be heavyweight. Test Plan: make check and dbstress Reviewers: haobo, igor Reviewed By: haobo CC: leveldb, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14283
-
由 Haobo Xu 提交于
Summary: liveness of the statistics object is already ensured by the shared pointer in DB options. There's no reason to pass again shared pointer among internal functions. Raw pointer is sufficient and efficient. Test Plan: make check Reviewers: dhruba, MarkCallaghan, igor Reviewed By: dhruba CC: leveldb, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14289
-
- 22 11月, 2013 3 次提交
-
-
由 kailiu 提交于
-
由 Siying Dong 提交于
This reverts commit 3d8ac31d.
-
由 Siying Dong 提交于
Summary: Provide a framework to profile a query in detail to figure out latency bottleneck. Currently, in Get(), Put() and iterators, 2-3 simple timing is used. We can easily add more profile counters to the framework later. Test Plan: Enable this profiling in seveal existing tests. Reviewers: haobo, dhruba, kailiu, emayanke, vamsi, igor CC: leveldb Differential Revision: https://reviews.facebook.net/D14001
-
- 21 11月, 2013 6 次提交
-
-
由 kailiu 提交于
We are having more and more options to specify for this table so it makes sense to have a TableOptions for future extension.
-
由 Kai Liu 提交于
-
由 Kai Liu 提交于
-
由 Kai Liu 提交于
-
由 kailiu 提交于
-
由 Haobo Xu 提交于
Summary: user comparator needs to work if either input is prefix only. Test Plan: ./prefix_test --write_buffer_size=100000 --total_prefixes=10000 --items_per_prefix=10 Reviewers: dhruba, igor Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D14241
-
- 20 11月, 2013 6 次提交
-
-
-
由 kailiu 提交于
Summary: Previously we introduce a `flush_block_policy_factory` in Options, however, that options is strongly releated to Table based tables. It will make more sense to move it to block based table's own factory class. Test Plan: make check to pass existing tests Reviewers: dhruba, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14211
-
由 Igor Canadi 提交于
Summary: These bugs were caught by ASAN crash test. 1. The first one, in table/filter_block.cc is very nasty. We first reference entries_ and store the reference to Slice prev. Then, we call entries_.append(), which can change the reference. The Slice prev now points to junk. 2. The second one is a bug in a test, so it's not very serious. Once we set read_opts.prefix, we never clear it, so some other function might still reference it. Test Plan: asan crash test now runs more than 5 mins. Before, it failed immediately. I will run the full one, but the full one takes quite some time (5 hours) Reviewers: dhruba, haobo, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14223
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Add asan_check rule to Makefile. After we add this, we will create Jenkins run that will check for asan errors! Test Plan: make asan_check Reviewers: dhruba, kailiu, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14205
-
由 kailiu 提交于
Summary: The primary motivation of the changes is to make it easier to figure out the inside of the tables. * rename "table stats" to "table properties" since now we have more than "integers" to store in the property block. * Add filter block size to the basic table properties. * Whenever a table is built, we'll log the table properties (the sample output is in Test Plan). * Make an api to expose deleted keys. Test Plan: Passed all existing test. and the sample output of table stats: ================================================================== Basic Properties ------------------------------------------------------------------ # data blocks: 1 # entries: 1 raw key size: 9 raw average key size: 9 raw value size: 9 raw average value size: 0 data block size: 25 index block size: 27 filter block size: 18 (estimated) table size: 70 filter policy: rocksdb.BuiltinBloomFilter ================================================================== User collected properties: InternalKeyPropertiesCollector ------------------------------------------------------------------ kDeletedKeys: 1 ================================================================== Reviewers: dhruba, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14187
-
- 19 11月, 2013 10 次提交
-
-
由 Igor Canadi 提交于
Summary: We don't need that dependency Test Plan: make check Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14199
-
由 Igor Canadi 提交于
Summary: Sure, let me put 8 bytes in that int32_t. Brought to you by ASAN! Test Plan: ttl_test Reviewers: dhruba, haobo, kailiu, emayanke Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14193
-
由 Igor Canadi 提交于
Summary: We were using ZLIB constants in BZIP code path. This caused some errors, like: https://github.com/facebook/rocksdb/issues/8 Test Plan: make clean; make check Reviewers: dhruba, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14175
-
由 Igor Canadi 提交于
Summary: For some reason, snappy on CentOS 5.2 when compiled with gcc 4.8.1 segfaults on strcmp. (!?) Add an if to compile with gcc4.7.1 if you're compiling on CentOS 5.2. Please update your devservers to CentOS 6. Test Plan: make clean; make check on both my devserver (CentOS 6) and dhruba's (CentOS 5.2) Reviewers: dhruba, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14169
-
由 Igor Canadi 提交于
Summary: We also need to use custom glibc and libgcc includes instead of system ones. Test Plan: 'make clean; make check'. Will also try on @dhruba's dev server. Reviewers: dhruba, kailiu Reviewed By: kailiu CC: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D14157
-
由 Igor Canadi 提交于
Summary: Explained what it takes to compile it on linux. Test Plan: - Reviewers: dhruba, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D14151
-
由 Igor Canadi 提交于
Summary: 1. Moved the compiler back to 4.8.1 and uses Centos 5.2 binaries if OS is Centos 5.2. 2. Fixes this issue: https://github.com/facebook/rocksdb/issues/7 3. We use lot of c++11 features, so we can't pretend we can compile without them. Makes it a first class dependency. 4. Fix blob_store_test, which failes on Ubuntu with "too many files opened" error 5. Removed dependency on port/port_chromium.h, which does not even exist on our system Test Plan: make clean; make check Reviewers: dhruba, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D14145
-
由 kailiu 提交于
Summary: as title. Test Plan: no Reviewers: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14121
-
由 Dhruba Borthakur 提交于
Summary: Add License message to public header files. Test Plan: Reviewers: CC: Task ID: # Blame Rev:
-
由 Igor Canadi 提交于
Summary: Dhruba can't compile on gcc4.8.1 so I'm moving temporarily back to 4.7.1 until we figure out what's wrong with 4.8. on his server. Test Plan: It can compile on my devserver, but please 'arc patch' this diff and try compiling on your machine Reviewers: dhruba, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14139
-
- 18 11月, 2013 3 次提交
-
-
由 Igor Canadi 提交于
Summary: Argumentation here: https://github.com/facebook/rocksdb/issues/9 Even though we include snappy in the distribution, we do not link with it if we don't have snappy installed on the system. Installing snappy is easy nowadays, just type: sudo apt-get install libsnappy-dev Test Plan: compile on ubuntu Reviewers: dhruba, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14133
-
由 Igor Canadi 提交于
Summary: This is the only compile issue in Ubuntu. It might be better to include <unistd.h> only in env_posix and add Truncate function to Env, but since we use truncate only in db_test, I don't think it makes much sense. Test Plan: Rocksdb now compiles on Ubuntu! Reviewers: dhruba, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14127
-
由 Igor Canadi 提交于
Summary: Finally did it - the trick was in using --dynamic-linker option. This is first step to running ASAN. All of our code seems to compile just fine on 4.8.1. However, I still left fbcode.471.sh in the 'build_tools/' just in case. Test Plan: make clean; make Reviewers: dhruba, haobo, kailiu, emayanke, sdong Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14109
-
- 17 11月, 2013 2 次提交
-
-
由 Kai Liu 提交于
Summary: Previously in KeyMayExist(), if DB::Get() returns non-Status::OK(), we assumes key may not exist. However, as if index block is not in block cache, Status::Incomplete() will return. Worse still, if options::filter_delete is enabled, we may falsely ignore the "delete" operation: https://github.com/facebook/rocksdb/blob/master/db/write_batch.cc#L217-L220 This diff fixes this bug and will let crash-test pass. Test Plan: Ran: ./db_stress --test_batches_snapshots=1 --ops_per_thread=1000000 --threads=32 --write_buffer_size=4194304 --destroy_db_initially=1 --reopen=0 --readpercent=5 --prefixpercent=45 --writepercent=35 --delpercent=5 --iterpercent=10 --db=/home/kailiu/local/newer --max_key=100000000 --disable_seek_compaction=0 --mmap_read=0 --block_size=16384 --cache_size=1048576 --open_files=500000 --verify_checksum=1 --sync=0 --disable_wal=0 --disable_data_sync=0 --target_file_size_base=2097152 --target_file_size_multiplier=2 --max_write_buffer_number=3 --max_background_compactions=20 --max_bytes_for_level_base=10485760 --filter_deletes=1 Previously we'll see crash happens very soon. Reviewers: igor, dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14115
-
由 kailiu 提交于
Summary: This diff invoves some more complicated issues in the posix environment. Test Plan: works under mac os. will need to verify dev box. Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14061
-