- 03 5月, 2016 1 次提交
-
-
由 Warren Falk 提交于
Using explicit 64-bit type in conditional in platforms above 32-bits This appears to be necessary on Mac OSX as std::conditional does not appear to short circuit and evaluates the third template arg Making the third template arg be 64 bits explicitly works around this problem and will work on both 32 bit and 64+ bit platforms.
-
- 30 4月, 2016 6 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: GetCurrentMutableCFOptions() can only be called when DB mutex is held so we cannot call it in CompactionJob::ProcessKeyValueCompaction() since it's not holding the db mutex Test Plan: make check -j64 Reviewers: sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57471
-
由 Dmitri Smirnov 提交于
make preallocation inline with other writable files make sure that we map no more than pre-allocated size.
-
由 Islam AbdelRahman 提交于
Summary: fix clang build Test Plan: USE_CLANG make all -j64 Reviewers: horuff, sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57453
-
由 Reid Horuff 提交于
Summary: Adds three new WriteBatch data types: Prepare(xid), Commit(xid), Rollback(xid). Prepare(xid) should precede the (single) operation to which is applies. There can obviously be multiple Prepare(xid) markers. There should only be one Rollback(xid) or Commit(xid) marker yet not both. None of this logic is currently enforced and will most likely be implemented further up such as in the memtableinserter. All three markers are similar to PutLogData in that they are writebatch meta-data, ie stored but not counted. All three markers differ from PutLogData in that they will actually be written to disk. As for WriteBatchWithIndex, Prepare, Commit, Rollback are all implemented just as PutLogData and none are tested just as PutLogData. Test Plan: single unit test in write_batch_test. Reviewers: hermanlee4, sdong, anthony Subscribers: andrewkr, vasilep, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54093
-
由 Reid Horuff 提交于
-
由 Yi Wu 提交于
Summary: Added EventListener::OnTableFileCreationStarted. EventListener::OnTableFileCreated will be called on failure case. User can check creation status via TableFileCreationInfo::status. Test Plan: unit test. Reviewers: dhruba, yhchiang, ott, sdong Reviewed By: sdong Subscribers: sdong, kradhakrishnan, IslamAbdelRahman, andrewkr, yhchiang, leveldb, ott, dhruba Differential Revision: https://reviews.facebook.net/D56337
-
- 29 4月, 2016 8 次提交
-
-
由 PraveenSinghRao 提交于
This reverts commit b54c3474. Revert async file handle change as it causes failures with appveyor
-
由 sdong 提交于
Summary: Changing several option defaults: options.max_open_files changes from 5000 to -1 options.base_background_compactions changes from max_background_compactions to 1 options.wal_recovery_mode changes from kTolerateCorruptedTailRecords to kTolerateCorruptedTailRecords options.compaction_pri changes from kByCompensatedSize to kByCompensatedSize Test Plan: Write unit tests to see OldDefaults() works as expected. Reviewers: IslamAbdelRahman, yhchiang, igor Reviewed By: igor Subscribers: MarkCallaghan, yiwu, kradhakrishnan, leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56427
-
由 Peter Mattis 提交于
This avoids excessive iteration in tombstone fields.
-
由 sdong 提交于
Summary: Warning is printed out with USE_CLANG=1 when including jemalloc.h. Disable it in that case. Test Plan: Run db_bench with USE_CLANG=1 and not. Make sure they can all build and jemalloc status is printed out in the case where USE_CLANG is not set. Reviewers: andrewkr, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57399
-
由 Islam AbdelRahman 提交于
Summary: crash_test grep for 'fail' string in the output and if found it consider that we failed. Update the output to use something else Test Plan: make crash_test (still running) Reviewers: yhchiang, sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57381
-
由 Andrew Kryczka 提交于
Summary: This test relies on "rocksdb.num-files-at-levelN" property that isn't implemented in rocksdb lite. So we will compile it only for non-lite builds. Test Plan: $ make -j40 check 'OPT=-g -DROCKSDB_LITE' Reviewers: sdong, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57387
-
由 Dmitri Smirnov 提交于
-
由 Andrew Kryczka 提交于
Summary: There was one narrowing conversion in D52287 that only showed up with clang on osx. Test Plan: $ make clean && USE_CLANG=1 DISABLE_JEMALLOC=1 TEST_TMPDIR=/dev/shm/rocksdb OPT=-g make -j32 check Reviewers: sdong, lightmark, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57357
-
- 28 4月, 2016 8 次提交
-
-
由 Li Peng 提交于
fix typos and remove duplicated words
-
由 Islam AbdelRahman 提交于
Summary: Fix typo in build_tools/fbcode_config.sh Test Plan: none Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57363
-
由 Andrew Kryczka 提交于
Summary: There were a few narrowing conversions that clang didn't like. Test Plan: $ make clean && USE_CLANG=1 DISABLE_JEMALLOC=1 TEST_TMPDIR=/dev/shm/rocksdb OPT=-g make -j32 check Reviewers: IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57351
-
由 Islam AbdelRahman 提交于
Summary: It looks like we mistakenly enable JEMALLOC even if it's not available on the machine, that's why travis is failing Test Plan: check on my devserver check on my mac Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57345
-
由 Andrew Kryczka 提交于
Summary: Made compression type and dictionary size configurable via environment variables. Depends on D52287. Test Plan: check these options are passed to the db. $ COMPRESSION_MAX_DICT_BYTES=65536 COMPRESSION_TYPE=LZ4 NUM_KEYS=10000000 DB_DIR=./tmp/ WAL_DIR=./tmp/ ./tools/benchmark.sh filluniquerandom ... $ grep Options.compression tmp/LOG 2016/04/22-19:11:30.397829 7f5f263a2980 Options.compression: LZ4 ... 2016/04/22-19:11:30.397837 7f5f263a2980 Options.compression_opts.max_dict_bytes: 65536 Reviewers: IslamAbdelRahman, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57141
-
由 Andrew Kryczka 提交于
Summary: This adds a new metablock containing a shared dictionary that is used to compress all data blocks in the SST file. The size of the shared dictionary is configurable in CompressionOptions and defaults to 0. It's currently only used for zlib/lz4/lz4hc, but the block will be stored in the SST regardless of the compression type if the user chooses a nonzero dictionary size. During compaction, computes the dictionary by randomly sampling the first output file in each subcompaction. It pre-computes the intervals to sample by assuming the output file will have the maximum allowable length. In case the file is smaller, some of the pre-computed sampling intervals can be beyond end-of-file, in which case we skip over those samples and the dictionary will be a bit smaller. After the dictionary is generated using the first file in a subcompaction, it is loaded into the compression library before writing each block in each subsequent file of that subcompaction. On the read path, gets the dictionary from the metablock, if it exists. Then, loads that dictionary into the compression library before reading each block. Test Plan: new unit test Reviewers: yhchiang, IslamAbdelRahman, cyan, sdong Reviewed By: sdong Subscribers: andrewkr, yoshinorim, kradhakrishnan, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D52287
-
由 Yueh-Hsuan Chiang 提交于
Summary: As db_stress with CompactFiles possibly catches a previous bug currently, temporarily disable CompactFiles in db_stress in its default setting to allows new bug to be detected while investigating the bug in CompactFiles. Test Plan: crash test Reviewers: sdong, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57333
-
由 Sergey Makarenko 提交于
Summary: Introduced option to dump malloc statistics using new option flag. Added new command line option to db_bench tool to enable this funtionality. Also extended build to support environments with/without jemalloc. Test Plan: 1) Build rocksdb using `make` command. Launch the following command `./db_bench --benchmarks=fillrandom --dump_malloc_stats=true --num=10000000` end verified that jemalloc dump is present in LOG file. 2) Build rocksdb using `DISABLE_JEMALLOC=1 make db_bench -j32` and ran the same db_bench tool and found the following message in LOG file: "Please compile with jemalloc to enable malloc dump". 3) Also built rocksdb using `make` command on MacOS to verify behavior in non-FB environment. Also to debug build configuration change temporary changed AM_DEFAULT_VERBOSITY = 1 in Makefile to see compiler and build tools output. For case 1) -DROCKSDB_JEMALLOC was present in compiler command line. For both 2) and 3) this flag was not present. Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57321
-
- 27 4月, 2016 4 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Fix BackupableDBTest.NoDoubleCopy and BackupableDBTest.DifferentEnvs by mocking the db files in db_env instead of backup_env_ Test Plan: make check -j64 Reviewers: sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57273
-
由 sdong 提交于
Summary: CompactedDB skips memtable. So we shouldn't use compacted DB if there is outstanding WAL files. Test Plan: Change to options.max_open_files = -1 perf context test to create a compacted DB, which we shouldn't do. Reviewers: yhchiang, kradhakrishnan, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57057
-
由 Islam AbdelRahman 提交于
Summary: While trying to reuse PinData() / ReleasePinnedData() .. to optimize away some memcpys I realized that there is a significant overhead for using PinData() / ReleasePinnedData if they were called many times. This diff refactor the pinning logic by introducing PinnedIteratorsManager a centralized component that will be created once and will be notified whenever we need to Pin an Iterator. This implementation have much less overhead than the original implementation Test Plan: make check -j64 COMPILE_WITH_ASAN=1 make check -j64 Reviewers: yhchiang, sdong, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56493
-
由 Andrew Kryczka 提交于
Summary: When db_env_ != backup_env_, InsertPathnameToSizeBytes() would use the wrong Env during backup creation. This happened because this function used backup_env_ instead of db_env_ to get WAL/data file sizes. This diff adds an argument to InsertPathnameToSizeBytes() indicating which Env to use. Test Plan: ran @anirbanb's BackupTestTool Reviewers: sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57159
-
- 26 4月, 2016 5 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: The current implementation find the first different byte and try to increment it, if it cannot it return the original key we can improve this by keep going after the first different byte to find the first non 0xFF byte and increment it After trying this patch on some logdevice sst files I see decrease in there index block size by 8.5% Test Plan: existing tests and updated test Reviewers: yhchiang, andrewkr, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D56241
-
由 Islam AbdelRahman 提交于
Summary: In this test some times automatic compactions do everything and Manual compaction become a no-op. Update the test to make sure manual compaction is not a no-op Test Plan: run the test Reviewers: andrewkr, yhchiang, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57189
-
由 sdong 提交于
Summary: Now we collect compaction stats per column family, but report default colum family's stat as compaction stats for DB. Fix it by reporting compaction stats per column family instead. Test Plan: Run db_bench with --num_column_families=4 and see the number fixed. Reviewers: IslamAbdelRahman, yhchiang Reviewed By: yhchiang Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57063
-
https://github.com/facebook/infer由 Dhruba Borthakur 提交于
Test Plan: make check Reviewers: leveldb, sdong Reviewed By: sdong Subscribers: leveldb, andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D57165
-
由 Yueh-Hsuan Chiang 提交于
Summary: In https://reviews.facebook.net/D56271, we fixed an issue where we consider flush as compaction. However, that makes us mistakenly count FLUSH_WRITE_BYTES twice (one in flush_job and one in db_impl.) This patch removes the one incremented in db_impl. Test Plan: db_test Reviewers: yiwu, andrewkr, IslamAbdelRahman, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57111
-
- 23 4月, 2016 7 次提交
-
-
由 dx9 提交于
* Musl libc does not provide adaptive mutex. Added feature test for PTHREAD_MUTEX_ADAPTIVE_NP. * Musl libc does not provide backtrace(3). Added a feature check for backtrace(3). * Fixed compiler error. * Musl libc does not implement backtrace(3). Added platform check for libexecinfo. * Alpine does not appear to support gcc -pg option. By default (gcc has PIE option enabled) it fails with: gcc: error: -pie and -pg|p|profile are incompatible when linking When -fno-PIE and -nopie are used it fails with: /usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find gcrt1.o: No such file or directory Added gcc -pg platform test and output PROFILING_FLAGS accordingly. Replaced pg var in Makefile with PROFILING_FLAGS. * fix segfault when TEST_IOCTL_FRIENDLY_TMPDIR is undefined and default candidates are not suitable * use ASSERT_DOUBLE_EQ instead of ASSERT_EQ * When compiled with ROCKSDB_MALLOC_USABLE_SIZE UniversalCompactionFourPaths and UniversalCompactionSecondPathRatio tests fail due to premature memtable flushes on systems with 16-byte alignment. Arena runs out of block space before GenerateNewFile() completes. Increased options.write_buffer_size.
-
由 Igor Canadi 提交于
-
由 Naitik Shah 提交于
rocksdb_backup_engine_purge_old_backups for C libraries
-
由 PraveenSinghRao 提交于
-
由 Naitik Shah 提交于
-
由 Naitik Shah 提交于
-
由 Naitik Shah 提交于
-
- 22 4月, 2016 1 次提交
-
-
由 Yueh-Hsuan Chiang 提交于
Summary: Fix RocksDB Lite build in db_stress Test Plan: OPT=-DROCKSDB_LITE db_stress Reviewers: IslamAbdelRahman, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57045
-