1. 02 6月, 2017 1 次提交
  2. 01 6月, 2017 2 次提交
    • T
      db: avoid `#include`ing malloc and jemalloc simultaneously · 0dc3040d
      Tamir Duberstein 提交于
      Summary:
      This fixes a compilation failure on Linux when the system libc is not
      glibc. jemalloc's configure script incorrectly assumes that glibc is
      always used on Linux systems, producing glibc-style signatures; when
      the system libc is e.g. musl, the following error is observed:
      
      ```
        [  0%] Building CXX object CMakeFiles/rocksdb.dir/db/db_impl.cc.o
        In file included from /go/src/github.com/cockroachdb/cockroach/c-deps/rocksdb.src/table/block.h:19:0,
                         from /go/src/github.com/cockroachdb/cockroach/c-deps/rocksdb.src/db/db_impl.cc:77:
        /x-tools/x86_64-unknown-linux-musl/x86_64-unknown-linux-musl/sysroot/usr/include/malloc.h:19:8: error: declaration of 'size_t malloc_usable_size(void*)' has a different exception specifier
         size_t malloc_usable_size(void *);
                ^~~~~~~~~~~~~~~~~~
        In file included from /go/src/github.com/cockroachdb/cockroach/c-deps/rocksdb.src/db/db_impl.cc:20:0:
        /go/native/x86_64-unknown-linux-musl/jemalloc/include/jemalloc/jemalloc.h:78:33: note: from previous declaration 'size_t malloc_usable_size(void*) throw ()'
         #  define je_malloc_usable_size malloc_usable_size
                                         ^
        /go/native/x86_64-unknown-linux-musl/jemalloc/include/jemalloc/jemalloc.h:239:41: note: in expansion of macro 'je_malloc_usable_size'
         JEMALLOC_EXPORT size_t JEMALLOC_NOTHROW je_malloc_usable_size(
                                                 ^~~~~~~~~~~~~~~~~~~~~
        CMakeFiles/rocksdb.dir/build.make:350: recipe for target 'CMakeFiles/rocksdb.dir/db/db_impl.cc.o' failed
      ```
      
      This works around the issue by rearranging the sources such that
      jemalloc's headers are never in the same scope as the system's malloc
      header. The jemalloc issue has been reported as well, see:
      https://github.com/jemalloc/jemalloc/issues/778.
      
      cc tschottdorf
      Closes https://github.com/facebook/rocksdb/pull/2188
      
      Differential Revision: D5163048
      
      Pulled By: siying
      
      fbshipit-source-id: c553125458892def175c1be5682b0330d80b2a0d
      0dc3040d
    • Y
      Fixing blob db sequence number handling · ad19eb86
      Yi Wu 提交于
      Summary:
      Blob db rely on base db returning sequence number through write batch after DB::Write(). However after recent changes to the write path, DB::Writ()e no longer return sequence number in some cases. Fixing it by have WriteBatchInternal::InsertInto() always encode sequence number into write batch.
      
      Stacking on #2375.
      Closes https://github.com/facebook/rocksdb/pull/2385
      
      Differential Revision: D5148358
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: 8bda0aa07b9334ed03ed381548b39d167dc20c33
      ad19eb86
  3. 31 5月, 2017 1 次提交
    • Y
      update blob_db_test · 345878a7
      Yi Wu 提交于
      Summary:
      Re-enable blob_db_test with some update:
      * Commented out delay at the end of GC tests. Will update the logic later with sync point to properly trigger GC.
      * Added some helper functions.
      
      Also update make files to include blob_dump tool.
      Closes https://github.com/facebook/rocksdb/pull/2375
      
      Differential Revision: D5133793
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: 95470b26d0c1f9592ba4b7637e027fdd263f425c
      345878a7
  4. 26 5月, 2017 1 次提交
  5. 25 5月, 2017 1 次提交
  6. 16 5月, 2017 2 次提交
    • N
      cross-platform compatibility improvements · 11c5d474
      Nikhil Benesch 提交于
      Summary:
      We've had a couple CockroachDB users fail to build RocksDB on exotic platforms, so I figured I'd try my hand at solving these issues upstream. The problems stem from a) `USE_SSE=1` being too aggressive about turning on SSE4.2, even on toolchains that don't support SSE4.2 and b) RocksDB attempting to detect support for thread-local storage based on OS, even though it can vary by compiler on the same OS.
      
      See the individual commit messages for details. Regarding SSE support, this PR should change virtually nothing for non-CMake based builds. `make`, `PORTABLE=1 make`, `USE_SSE=1 make`, and `PORTABLE=1 USE_SSE=1 make` function exactly as before, except that SSE support will be automatically disabled when a simple SSE4.2-using test program fails to compile, as it does on OpenBSD. (OpenBSD's ports GCC supports SSE4.2, but its binutils do not, so `__SSE_4_2__` is defined but an SSE4.2-using program will fail to assemble.) A warning is emitted in this case. The CMake build is modified to support the same set of options, except that `USE_SSE` is spelled `FORCE_SSE42` because `USE_SSE` is rather useless now that we can automatically detect SSE support, and I figure changing options in the CMake build is less disruptive than changing the non-CMake build.
      
      I've tested these changes on all the platforms I can get my hands on (macOS, Windows MSVC, Windows MinGW, and OpenBSD) and it all works splendidly. Let me know if there's anything you object to—I obviously don't mean to break any of your build pipelines in the process of fixing ours downstream.
      Closes https://github.com/facebook/rocksdb/pull/2199
      
      Differential Revision: D5054042
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: 938e1fc665c049c02ae15698e1409155b8e72171
      11c5d474
    • Y
      Fix build error with blob DB. · 86d54925
      Yi Wu 提交于
      Summary:
      snprintf is in <stdio.h> and not in namespace std.
      Closes https://github.com/facebook/rocksdb/pull/2287
      
      Reviewed By: anirbanr-fb
      
      Differential Revision: D5054752
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: 356807ec38f3c7d95951cdb41f31a3d3ae0714d4
      86d54925
  7. 13 5月, 2017 1 次提交
    • A
      Add GetAllKeyVersions API · 3fa9a39c
      Andrew Kryczka 提交于
      Summary:
      - Introduced an include/ file dedicated to db-related debug functions to avoid making db.h more complex
      - Added debugging function, `GetAllKeyVersions()`, to return a listing of internal data for a range of user keys. The new `struct KeyVersion` exposes data similar to internal key without exposing any internal type.
      - Migrated the "ldb idump" subcommand to use this function
      - The API takes an inclusive-exclusive range to match behavior of "ldb idump". This will be quite annoying for users who want to query a single user key's versions :(.
      Closes https://github.com/facebook/rocksdb/pull/2232
      
      Differential Revision: D4976007
      
      Pulled By: ajkr
      
      fbshipit-source-id: cab375da53a7595d6575af2b7e3b776aa3ad793e
      3fa9a39c
  8. 11 5月, 2017 3 次提交
    • A
      update TARGETS · 93949667
      Andrew Kryczka 提交于
      Summary:
      address siying's comment in #2272.
      Closes https://github.com/facebook/rocksdb/pull/2274
      
      Differential Revision: D5039489
      
      Pulled By: ajkr
      
      fbshipit-source-id: 3e2d957d3469c13d0e33ededa59320c4c3f24ef6
      93949667
    • A
      Blob storage pr · d85ff495
      Anirban Rahut 提交于
      Summary:
      The final pull request for Blob Storage.
      Closes https://github.com/facebook/rocksdb/pull/2269
      
      Differential Revision: D5033189
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: 6356b683ccd58cbf38a1dc55e2ea400feecd5d06
      d85ff495
    • A
      portable sched_getcpu calls · be421b0b
      Andrew Kryczka 提交于
      Summary:
      - added a feature test in build_detect_platform to check whether sched_getcpu() is available. glibc offers it only on some platforms (e.g., linux but not mac); this way should be easier than maintaining a list of platforms on which it's available.
      - refactored PhysicalCoreID() to be simpler / less repetitive. ordered the conditional compilation clauses from most-to-least preferred
      Closes https://github.com/facebook/rocksdb/pull/2272
      
      Differential Revision: D5038093
      
      Pulled By: ajkr
      
      fbshipit-source-id: 81d7db3cc620250de220bdeb3194b2b3d7673de7
      be421b0b
  9. 27 4月, 2017 1 次提交
  10. 25 4月, 2017 1 次提交
    • A
      Reunite checkpoint and backup core logic · e5e545a0
      Andrew Kryczka 提交于
      Summary:
      These code paths forked when checkpoint was introduced by copy/pasting the core backup logic. Over time they diverged and bug fixes were sometimes applied to one but not the other (like fix to include all relevant WALs for 2PC), or it required extra effort to fix both (like fix to forge CURRENT file). This diff reunites the code paths by extracting the core logic into a function, CreateCustomCheckpoint(), that is customizable via callbacks to implement both checkpoint and backup.
      
      Related changes:
      
      - flush_before_backup is now forcibly enabled when 2PC is enabled
      - Extracted CheckpointImpl class definition into a header file. This is so the function, CreateCustomCheckpoint(), can be called by internal rocksdb code but not exposed to users.
      - Implemented more functions in DummyDB/DummyLogFile (in backupable_db_test.cc) that are used by CreateCustomCheckpoint().
      Closes https://github.com/facebook/rocksdb/pull/1932
      
      Differential Revision: D4622986
      
      Pulled By: ajkr
      
      fbshipit-source-id: 157723884236ee3999a682673b64f7457a7a0d87
      e5e545a0
  11. 12 4月, 2017 1 次提交
  12. 05 4月, 2017 1 次提交