1. 11 1月, 2014 1 次提交
  2. 10 1月, 2014 1 次提交
    • I
      Feature requests for BackupableDB · cb37ddf2
      Igor Canadi 提交于
      Summary:
      This diff introduces some features that were requested by two internal customers:
      * Ability for backups not to share table files, because we can't guarantee that equal filename means equal content accross replicas
      * Ability for two threads to call EnableFileDeletions() and DisableFileDeletions()
      * Ability to stop backup from another thread and not slow down the DB close
      * Copy the files to the temporary folder first and then atomically rename
      
      Test Plan: Added some tests to backupable_db_test
      
      Reviewers: dhruba, sanketh, muthu, sdong, haobo
      
      Reviewed By: haobo
      
      CC: leveldb, sanketh, muthu
      
      Differential Revision: https://reviews.facebook.net/D14769
      cb37ddf2
  3. 13 12月, 2013 2 次提交
  4. 11 12月, 2013 2 次提交
  5. 10 12月, 2013 2 次提交
    • I
      Fix unused variable warning · 784e62f9
      Igor Canadi 提交于
      784e62f9
    • I
      [RocksDB] BackupableDB · fb9fce4f
      Igor Canadi 提交于
      Summary:
      In this diff I present you BackupableDB v1. You can easily use it to backup your DB and it will do incremental snapshots for you.
      Let's first describe how you would use BackupableDB. It's inheriting StackableDB interface so you can easily construct it with your DB object -- it will add a method RollTheSnapshot() to the DB object. When you call RollTheSnapshot(), current snapshot of the DB will be stored in the backup dir. To restore, you can just call RestoreDBFromBackup() on a BackupableDB (which is a static method) and it will restore all files from the backup dir. In the next version, it will even support automatic backuping every X minutes.
      
      There are multiple things you can configure:
      1. backup_env and db_env can be different, which is awesome because then you can easily backup to HDFS or wherever you feel like.
      2. sync - if true, it *guarantees* backup consistency on machine reboot
      3. number of snapshots to keep - this will keep last N snapshots around if you want, for some reason, be able to restore from an earlier snapshot. All the backuping is done in incremental fashion - if we already have 00010.sst, we will not copy it again. *IMPORTANT* -- This is based on assumption that 00010.sst never changes - two files named 00010.sst from the same DB will always be exactly the same. Is this true? I always copy manifest, current and log files.
      4. You can decide if you want to flush the memtables before you backup, or you're fine with backing up the log files -- either way, you get a complete and consistent view of the database at a time of backup.
      5. More things you can find in BackupableDBOptions
      
      Here is the directory structure I use:
      
         backup_dir/CURRENT_SNAPSHOT - just 4 bytes holding the latest snapshot
                     0, 1, 2, ... - files containing serialized version of each snapshot - containing a list of files
                     files/*.sst - sst files shared between snapshots - if one snapshot references 00010.sst and another one needs to backup it from the DB, it will just reference the same file
                     files/ 0/, 1/, 2/, ... - snapshot directories containing private snapshot files - current, manifest and log files
      
      All the files are ref counted and deleted immediatelly when they get out of scope.
      
      Some other stuff in this diff:
      1. Added GetEnv() method to the DB. Discussed with @haobo and we agreed that it seems right thing to do.
      2. Fixed StackableDB interface. The way it was set up before, I was not able to implement BackupableDB.
      
      Test Plan:
      I have a unittest, but please don't look at this yet. I just hacked it up to help me with debugging. I will write a lot of good tests and update the diff.
      
      Also, `make asan_check`
      
      Reviewers: dhruba, haobo, emayanke
      
      Reviewed By: dhruba
      
      CC: leveldb, haobo
      
      Differential Revision: https://reviews.facebook.net/D14295
      fb9fce4f