- 21 11月, 2013 3 次提交
-
-
由 Kai Liu 提交于
-
由 kailiu 提交于
-
由 Haobo Xu 提交于
Summary: user comparator needs to work if either input is prefix only. Test Plan: ./prefix_test --write_buffer_size=100000 --total_prefixes=10000 --items_per_prefix=10 Reviewers: dhruba, igor Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D14241
-
- 20 11月, 2013 6 次提交
-
-
-
由 kailiu 提交于
Summary: Previously we introduce a `flush_block_policy_factory` in Options, however, that options is strongly releated to Table based tables. It will make more sense to move it to block based table's own factory class. Test Plan: make check to pass existing tests Reviewers: dhruba, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14211
-
由 Igor Canadi 提交于
Summary: These bugs were caught by ASAN crash test. 1. The first one, in table/filter_block.cc is very nasty. We first reference entries_ and store the reference to Slice prev. Then, we call entries_.append(), which can change the reference. The Slice prev now points to junk. 2. The second one is a bug in a test, so it's not very serious. Once we set read_opts.prefix, we never clear it, so some other function might still reference it. Test Plan: asan crash test now runs more than 5 mins. Before, it failed immediately. I will run the full one, but the full one takes quite some time (5 hours) Reviewers: dhruba, haobo, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14223
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: Add asan_check rule to Makefile. After we add this, we will create Jenkins run that will check for asan errors! Test Plan: make asan_check Reviewers: dhruba, kailiu, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14205
-
由 kailiu 提交于
Summary: The primary motivation of the changes is to make it easier to figure out the inside of the tables. * rename "table stats" to "table properties" since now we have more than "integers" to store in the property block. * Add filter block size to the basic table properties. * Whenever a table is built, we'll log the table properties (the sample output is in Test Plan). * Make an api to expose deleted keys. Test Plan: Passed all existing test. and the sample output of table stats: ================================================================== Basic Properties ------------------------------------------------------------------ # data blocks: 1 # entries: 1 raw key size: 9 raw average key size: 9 raw value size: 9 raw average value size: 0 data block size: 25 index block size: 27 filter block size: 18 (estimated) table size: 70 filter policy: rocksdb.BuiltinBloomFilter ================================================================== User collected properties: InternalKeyPropertiesCollector ------------------------------------------------------------------ kDeletedKeys: 1 ================================================================== Reviewers: dhruba, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14187
-
- 19 11月, 2013 10 次提交
-
-
由 Igor Canadi 提交于
Summary: We don't need that dependency Test Plan: make check Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14199
-
由 Igor Canadi 提交于
Summary: Sure, let me put 8 bytes in that int32_t. Brought to you by ASAN! Test Plan: ttl_test Reviewers: dhruba, haobo, kailiu, emayanke Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14193
-
由 Igor Canadi 提交于
Summary: We were using ZLIB constants in BZIP code path. This caused some errors, like: https://github.com/facebook/rocksdb/issues/8 Test Plan: make clean; make check Reviewers: dhruba, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14175
-
由 Igor Canadi 提交于
Summary: For some reason, snappy on CentOS 5.2 when compiled with gcc 4.8.1 segfaults on strcmp. (!?) Add an if to compile with gcc4.7.1 if you're compiling on CentOS 5.2. Please update your devservers to CentOS 6. Test Plan: make clean; make check on both my devserver (CentOS 6) and dhruba's (CentOS 5.2) Reviewers: dhruba, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14169
-
由 Igor Canadi 提交于
Summary: We also need to use custom glibc and libgcc includes instead of system ones. Test Plan: 'make clean; make check'. Will also try on @dhruba's dev server. Reviewers: dhruba, kailiu Reviewed By: kailiu CC: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D14157
-
由 Igor Canadi 提交于
Summary: Explained what it takes to compile it on linux. Test Plan: - Reviewers: dhruba, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D14151
-
由 Igor Canadi 提交于
Summary: 1. Moved the compiler back to 4.8.1 and uses Centos 5.2 binaries if OS is Centos 5.2. 2. Fixes this issue: https://github.com/facebook/rocksdb/issues/7 3. We use lot of c++11 features, so we can't pretend we can compile without them. Makes it a first class dependency. 4. Fix blob_store_test, which failes on Ubuntu with "too many files opened" error 5. Removed dependency on port/port_chromium.h, which does not even exist on our system Test Plan: make clean; make check Reviewers: dhruba, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D14145
-
由 kailiu 提交于
Summary: as title. Test Plan: no Reviewers: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14121
-
由 Dhruba Borthakur 提交于
Summary: Add License message to public header files. Test Plan: Reviewers: CC: Task ID: # Blame Rev:
-
由 Igor Canadi 提交于
Summary: Dhruba can't compile on gcc4.8.1 so I'm moving temporarily back to 4.7.1 until we figure out what's wrong with 4.8. on his server. Test Plan: It can compile on my devserver, but please 'arc patch' this diff and try compiling on your machine Reviewers: dhruba, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14139
-
- 18 11月, 2013 3 次提交
-
-
由 Igor Canadi 提交于
Summary: Argumentation here: https://github.com/facebook/rocksdb/issues/9 Even though we include snappy in the distribution, we do not link with it if we don't have snappy installed on the system. Installing snappy is easy nowadays, just type: sudo apt-get install libsnappy-dev Test Plan: compile on ubuntu Reviewers: dhruba, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14133
-
由 Igor Canadi 提交于
Summary: This is the only compile issue in Ubuntu. It might be better to include <unistd.h> only in env_posix and add Truncate function to Env, but since we use truncate only in db_test, I don't think it makes much sense. Test Plan: Rocksdb now compiles on Ubuntu! Reviewers: dhruba, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14127
-
由 Igor Canadi 提交于
Summary: Finally did it - the trick was in using --dynamic-linker option. This is first step to running ASAN. All of our code seems to compile just fine on 4.8.1. However, I still left fbcode.471.sh in the 'build_tools/' just in case. Test Plan: make clean; make Reviewers: dhruba, haobo, kailiu, emayanke, sdong Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14109
-
- 17 11月, 2013 6 次提交
-
-
由 Kai Liu 提交于
Summary: Previously in KeyMayExist(), if DB::Get() returns non-Status::OK(), we assumes key may not exist. However, as if index block is not in block cache, Status::Incomplete() will return. Worse still, if options::filter_delete is enabled, we may falsely ignore the "delete" operation: https://github.com/facebook/rocksdb/blob/master/db/write_batch.cc#L217-L220 This diff fixes this bug and will let crash-test pass. Test Plan: Ran: ./db_stress --test_batches_snapshots=1 --ops_per_thread=1000000 --threads=32 --write_buffer_size=4194304 --destroy_db_initially=1 --reopen=0 --readpercent=5 --prefixpercent=45 --writepercent=35 --delpercent=5 --iterpercent=10 --db=/home/kailiu/local/newer --max_key=100000000 --disable_seek_compaction=0 --mmap_read=0 --block_size=16384 --cache_size=1048576 --open_files=500000 --verify_checksum=1 --sync=0 --disable_wal=0 --disable_data_sync=0 --target_file_size_base=2097152 --target_file_size_multiplier=2 --max_write_buffer_number=3 --max_background_compactions=20 --max_bytes_for_level_base=10485760 --filter_deletes=1 Previously we'll see crash happens very soon. Reviewers: igor, dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14115
-
由 kailiu 提交于
Summary: This diff invoves some more complicated issues in the posix environment. Test Plan: works under mac os. will need to verify dev box. Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14061
-
由 Kai Liu 提交于
Summary: By original design, the regular `block options` and index `block options` in table_builder is mutable. We can use ChangeOptions to change the options directly. However, with my last change, `BlockBuilder` no longer hold the reference to the index_block_options -- as a result, any changes made after the creation of index block builder will be of no effect. But still the code is very error-prone and developers can easily fall into the trap without aware of it. To avoid this problem from happening in the future, I deleted the `ChangeOptions` and the `index_block_options`, as well as many other changes to make it less misleading. Test Plan: make make check make release Reviewers: dhruba, haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D13707
-
由 Kai Liu 提交于
-
由 Igor Canadi 提交于
Fixed typos
-
由 Igor Canadi 提交于
-
- 16 11月, 2013 5 次提交
-
-
由 Pascal Borreli 提交于
-
由 Siying Dong 提交于
Add an option to table_reader_bench to access the table from DB And Iterating non-existing prefix case. Summary: This patch adds an option to table_reader_bench that queries run against DB level (which has one table). It is useful if user wants to see the extra costs DB level introduces. Test Plan: Run the benchmark with and without the new parameter Reviewers: haobo, dhruba, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D13863
-
由 Kai Liu 提交于
-
由 Igor Canadi 提交于
Summary: Remove all the files from the test dir before the test. The test failed when there were some old files still in the directory, since it checks the file counts. This is what caused jenkins' test failures. It was running fine on my machine so it was hard to repro. Test Plan: 1. create an extra 000001.log file in the test directory 2. run a ./deletefile_test - test failes 3. patch ./deletefile_test with this 4. test succeeds Reviewers: haobo, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14097
-
由 Kai Liu 提交于
-
- 15 11月, 2013 4 次提交
-
-
由 Igor Canadi 提交于
Summary: Also changed some comments Test Plan: ./deletefile_test Reviewers: haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14091
-
由 Igor Canadi 提交于
Summary: Created a unittest that verifies that automatic deletion performed by PurgeObsoleteFiles() works correctly. Also, few small fixes on the logic part -- call version_set_->GetObsoleteFiles() in FindObsoleteFiles() instead of on some arbitrary positions. Test Plan: Created a unit test Reviewers: dhruba, haobo, nkg- Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14079
-
由 Vamsi Ponnekanti 提交于
Summary: We iterate until we find a different key than original key. ikey is pointing to next key when we break out of loop. After the loop we apply all merge operands meant for original key on the next key! Test Plan: Need to give a build to Marcin to test out. Revert Plan: OK Task ID: #3181932 Reviewers: haobo, emayanke, dhruba Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14073
-
由 Igor Canadi 提交于
Summary: https://reviews.facebook.net/D13167 broke bloom filters. If filter is not in cache, we want to return true (safe thing). Am I right? Test Plan: when benchmarking https://reviews.facebook.net/D14031 I got different results when using bloom filters vs. when not using them. This fixed the issue. I will also be putting this change to the other diff, but that one will probably be in review for longer time. Reviewers: kailiu, dhruba, haobo Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D14085
-
- 14 11月, 2013 1 次提交
-
-
由 Igor Canadi 提交于
Summary: Log files are stored in wal_dir, not dbname_ Test Plan: deletfile_test Reviewers: nkg- Reviewed By: nkg- CC: leveldb Differential Revision: https://reviews.facebook.net/D14067
-
- 13 11月, 2013 2 次提交
-
-
由 Kai Liu 提交于
-
由 Kai Liu 提交于
Summary: This diff leverage the existing block cache and extend it to cache index/filter block. Test Plan: Added new tests in db_test and table_test The correctness is checked by: 1. make check 2. make valgrind_check Performance is test by: 1. 10 times of build_tools/regression_build_test.sh on two versions of rocksdb before/after the code change. Test results suggests no significant difference between them. For the two key operatons `overwrite` and `readrandom`, the average iops are both 20k and ~260k, with very small variance). 2. db_stress. Reviewers: dhruba Reviewed By: dhruba CC: leveldb, haobo, xjin Differential Revision: https://reviews.facebook.net/D13167
-