- 11 12月, 2013 1 次提交
-
-
由 Igor Canadi 提交于
Rename leveldb to rocksdb in C api
-
- 10 12月, 2013 4 次提交
-
-
由 Doğan Çeçen 提交于
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: In this diff I present you BackupableDB v1. You can easily use it to backup your DB and it will do incremental snapshots for you. Let's first describe how you would use BackupableDB. It's inheriting StackableDB interface so you can easily construct it with your DB object -- it will add a method RollTheSnapshot() to the DB object. When you call RollTheSnapshot(), current snapshot of the DB will be stored in the backup dir. To restore, you can just call RestoreDBFromBackup() on a BackupableDB (which is a static method) and it will restore all files from the backup dir. In the next version, it will even support automatic backuping every X minutes. There are multiple things you can configure: 1. backup_env and db_env can be different, which is awesome because then you can easily backup to HDFS or wherever you feel like. 2. sync - if true, it *guarantees* backup consistency on machine reboot 3. number of snapshots to keep - this will keep last N snapshots around if you want, for some reason, be able to restore from an earlier snapshot. All the backuping is done in incremental fashion - if we already have 00010.sst, we will not copy it again. *IMPORTANT* -- This is based on assumption that 00010.sst never changes - two files named 00010.sst from the same DB will always be exactly the same. Is this true? I always copy manifest, current and log files. 4. You can decide if you want to flush the memtables before you backup, or you're fine with backing up the log files -- either way, you get a complete and consistent view of the database at a time of backup. 5. More things you can find in BackupableDBOptions Here is the directory structure I use: backup_dir/CURRENT_SNAPSHOT - just 4 bytes holding the latest snapshot 0, 1, 2, ... - files containing serialized version of each snapshot - containing a list of files files/*.sst - sst files shared between snapshots - if one snapshot references 00010.sst and another one needs to backup it from the DB, it will just reference the same file files/ 0/, 1/, 2/, ... - snapshot directories containing private snapshot files - current, manifest and log files All the files are ref counted and deleted immediatelly when they get out of scope. Some other stuff in this diff: 1. Added GetEnv() method to the DB. Discussed with @haobo and we agreed that it seems right thing to do. 2. Fixed StackableDB interface. The way it was set up before, I was not able to implement BackupableDB. Test Plan: I have a unittest, but please don't look at this yet. I just hacked it up to help me with debugging. I will write a lot of good tests and update the diff. Also, `make asan_check` Reviewers: dhruba, haobo, emayanke Reviewed By: dhruba CC: leveldb, haobo Differential Revision: https://reviews.facebook.net/D14295
-
由 Igor Canadi 提交于
Branch detection did not work in Jenkins. I realized that it set GIT_BRANCH env variable to point to the current branch, so let's try using this for branch detection.
-
- 07 12月, 2013 3 次提交
-
-
由 Igor Canadi 提交于
Summary: This will help me a lot! When we hit an assertion in unittest, we get the whole stack trace now. Also, changed stack trace a bit, we now include actual demangled C++ class::function symbols! Test Plan: Added ASSERT_TRUE(false) to a test, observed a stack trace Reviewers: haobo, dhruba, kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D14499
-
由 Igor Canadi 提交于
Summary: When running regression tests on other branches, this will push values to entity rocksdb_build.$git_branch Test Plan: Ran regression test on regression branch, observed values send to ODS in entity rocksdb_build.regression Reviewers: kailiu Reviewed By: kailiu CC: leveldb Differential Revision: https://reviews.facebook.net/D14493
-
由 Igor Canadi 提交于
Summary: Now DBWithTTL takes DB* and can behave more like StackableDB. This saves us a lot of duplicate work by defining interfaces Test Plan: ttl_test with ASAN - OK Reviewers: emayanke Reviewed By: emayanke CC: leveldb Differential Revision: https://reviews.facebook.net/D14481
-
- 06 12月, 2013 2 次提交
-
-
由 kailiu 提交于
-
由 Mayank Agarwal 提交于
Summary: As title Test Plan: make clean and make Reviewers: igor Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D14469
-
- 05 12月, 2013 2 次提交
-
-
由 Mayank Agarwal 提交于
Summary: This would enable rocksdb users to get the db identity without depending on implementation details(storing that in IDENTITY file) Test Plan: db/db_test (has identity checks) Reviewers: dhruba, haobo, igor, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14463
-
由 Vamsi Ponnekanti 提交于
Summary: $title Test Plan: on my dev box Revert Plan: OK Task ID: # Reviewers: emayanke, dhruba, haobo Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D14451
-
- 04 12月, 2013 4 次提交
-
-
由 Mark Callaghan 提交于
Summary: This adds 2 options for compression to db_bench: * universal_compression_size_percent * compression_level - to set zlib compression level It also logs compression_size_percent at startup in LOG Task ID: # Blame Rev: Test Plan: make check, run db_bench Revert Plan: Database Impact: Memcache Impact: Other Notes: EImportant: - begin *PUBLIC* platform impact section - Bugzilla: # - end platform impact - Reviewers: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14439
-
由 Sajal Jain 提交于
Summary: added counters rocksdb.memtable.hit - for memtable hit rocksdb.memtable.miss - for memtable miss Test Plan: db_bench tests Reviewers: igor, dhruba, haobo Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D14433
-
由 Igor Canadi 提交于
Summary: Let's get rid of TransformRep and it's children. We have confirmed that HashSkipListRep works better with multifeed, so there is no benefit to keeping this around. This diff is mostly just deleting references to obsoleted functions. I also have a diff for fbcode that we'll need to push when we switch to new release. I had to expose HashSkipListRepFactory in the client header files because db_impl.cc needs access to GetTransform() function for SanitizeOptions. Test Plan: make check Reviewers: dhruba, haobo, kailiu, sdong Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14397
-
由 Igor Canadi 提交于
Summary: I went through all remaining shared_ptrs and removed the ones that I found not-necessary. Only GenerateCachePrefix() is called fairly often, so don't expect much perf wins. The ones that are left are accessed infrequently and I think we're fine with keeping them. Test Plan: make asan_check Reviewers: dhruba, haobo Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14427
-
- 03 12月, 2013 2 次提交
-
-
由 lovro 提交于
Summary: Documenting our discussion Test Plan: make Reviewers: dhruba, haobo Reviewed By: dhruba CC: igor Differential Revision: https://reviews.facebook.net/D14403
-
由 Igor Canadi 提交于
-
- 02 12月, 2013 3 次提交
-
-
由 Igor Canadi 提交于
-
由 Dhruba Borthakur 提交于
Summary: Memtables should be deleted appropriately in the unit test. Test Plan: Reviewers: CC: Task ID: # Blame Rev:
-
由 lovro 提交于
Summary: The preprocessor does not follow normal rules of && evaluation, tries to evaluate __GLIBC_PREREQ(2, 12) even though the defined() check fails. This breaks the build if __GLIBC_PREREQ is absent. Test Plan: Try adding #undef __GLIBC_PREREQ above the offending line, build no longer breaks Reviewed By: igor Blame Rev: 4c813836
-
- 30 11月, 2013 1 次提交
-
-
由 Dhruba Borthakur 提交于
Summary: Removed redundant slice_transform.h and memtablerep.h Test Plan: make check Reviewers: CC: Task ID: # Blame Rev:
-
- 29 11月, 2013 1 次提交
-
-
由 Dhruba Borthakur 提交于
Summary: The commit at 27bbef11 had a memory leak that was detected by valgrind. The memtable that has a refcount decrement in MemTableList::InstallMemtableFlushResults was not freed. Test Plan: valgrind ./db_test --leak-check=full Reviewers: igor Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D14391
-
- 28 11月, 2013 5 次提交
-
-
由 Igor Canadi 提交于
Summary: Added readrandom benchmark with 300MB block cache, while database has 1GB of data Test Plan: Ran it Reviewers: dhruba, MarkCallaghan Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14373
-
由 Igor Canadi 提交于
Summary: These tests fail if compression libraries are not installed. Test Plan: Manually disabled snappy, observed tests not ran. Reviewers: dhruba, kailiu Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D14379
-
由 lovro 提交于
Summary: Makes it easier to monitor performance with top Test Plan: ./manual_compaction_test with `top -H` running. Previously was two `manual_compacti`, now one shows `rocksdb:bg0`. Reviewers: igor, dhruba Reviewed By: igor CC: leveldb Differential Revision: https://reviews.facebook.net/D14367
-
由 Igor Canadi 提交于
Fix typo.
-
由 Isamu Arimoto 提交于
-
- 27 11月, 2013 1 次提交
-
-
由 Igor Canadi 提交于
Summary: * Fixed regression test params by @dhruba's suggestion * Added p50, p75 and p99 to regression metrics Test Plan: build_tools/build_regression_test.sh Reviewers: dhruba, emayanke Reviewed By: dhruba CC: leveldb, dhruba, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14355
-
- 26 11月, 2013 11 次提交
-
-
由 Dhruba Borthakur 提交于
Summary: This code path can potentially accumulate multiple important_files for level 0. But for other levels, it should have only one file in the important_files, so it is ok not to reserve excessive space, is it not? Test Plan: make check Reviewers: haobo Reviewed By: haobo CC: reconnect.grayhat, leveldb Differential Revision: https://reviews.facebook.net/D14349
-
由 Kai Liu 提交于
Added missing component : gflags in Linux platform. Thank you Kangmo!
-
由 Kangmo Kim 提交于
Added missing component : gflags in Linux platform.
-
由 Igor Canadi 提交于
Summary: Check https://github.com/facebook/rocksdb/pull/15 for context. Apparently [1], we need -fPIC in x64 environments (this is added only in non-fbcode). In fbcode, I removed -fPIC per @dhruba's suggestion, since it introduces perf regression. I'm not sure what would are the implications of doing that, but looks like it works, and when releasing to the third-party, we're disabling -fPIC either way [2]. Would love a suggestion from someone who knows more about this [1] http://eli.thegreenplace.net/2011/11/11/position-independent-code-pic-in-shared-libraries-on-x64/ [2] https://our.intern.facebook.com/intern/wiki/index.php/Database/RocksDB/Third_Party Test Plan: make check works Reviewers: dhruba, emayanke, kailiu Reviewed By: dhruba CC: leveldb, dhruba, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14337
-
由 Dhruba Borthakur 提交于
Summary: Large memory allocations and frees are costly and best done outside the db-mutex. The memtables are already allocated outside the db-mutex but they were being freed while holding the db-mutex. This patch frees obsolete memtables outside the db-mutex. Test Plan: make check db_stress Unit tests pass, I am in the process of running stress tests. Reviewers: haobo, igor, emayanke Reviewed By: haobo CC: reconnect.grayhat, leveldb Differential Revision: https://reviews.facebook.net/D14319
-
由 Igor Canadi 提交于
Summary: We need access to options for BackupableDB Test Plan: make check Reviewers: dhruba Reviewed By: dhruba CC: leveldb, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14331
-
由 Igor Canadi 提交于
-
由 Igor Canadi 提交于
Summary: The old regression tests didn't cover memtable part at all. This is an atempt to also measure memtable performance in regression tests. Test Plan: Ran regression_build_test.sh Reviewers: dhruba, haobo, kailiu, emayanke Reviewed By: dhruba CC: leveldb, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14325
-
由 Igor Canadi 提交于
Summary: This is part of https://reviews.facebook.net/D14295 -- smaller diff that is easier to review Test Plan: make asan_check Reviewers: dhruba, haobo, emayanke Reviewed By: emayanke CC: leveldb, kailiu, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14301
-
由 Dhruba Borthakur 提交于
Summary: All filesystem Io should be done outside the dbmutex. There was one place when we have to roll the transaction log that we were creating the new log file while holding the dbmutex. I rearranged this code so that the act of creating the new transaction log file is done without holding the dbmutex. I also allocate the new memtable outside the dbmutex, this is important because creating the memtable could be heavyweight. Test Plan: make check and dbstress Reviewers: haobo, igor Reviewed By: haobo CC: leveldb, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14283
-
由 Haobo Xu 提交于
Summary: liveness of the statistics object is already ensured by the shared pointer in DB options. There's no reason to pass again shared pointer among internal functions. Raw pointer is sufficient and efficient. Test Plan: make check Reviewers: dhruba, MarkCallaghan, igor Reviewed By: dhruba CC: leveldb, reconnect.grayhat Differential Revision: https://reviews.facebook.net/D14289
-