- 14 4月, 2017 1 次提交
-
-
由 Aaron Gao 提交于
Summary: add the missing space Closes https://github.com/facebook/rocksdb/pull/2150 Differential Revision: D4880696 Pulled By: lightmark fbshipit-source-id: a4e0ad6a8ea45d6469d3f6c8514fdeb4cf10aaf5
-
- 12 4月, 2017 1 次提交
-
-
由 Aaron Gao 提交于
Summary: filter the warning out and only print it once. Closes https://github.com/facebook/rocksdb/pull/2137 Differential Revision: D4870925 Pulled By: lightmark fbshipit-source-id: 91b363ce7f70bce88b0780337f408fc4649139b8
-
- 07 4月, 2017 1 次提交
-
-
由 Aaron Gao 提交于
Summary: In RocksDB, we sometimes preallocate the estimated space for a file to have better perf with fallocate (if supported). Usually it is a little bit bigger than the real resulting file size. At this time, we have to let the Filesystem reclaim the space not used. Ideally, calling ftruncate to truncate the file to its real size should be enough. HOWEVER, it isn't on tmpfs, which we witness in our case, with some buggy kernel version. ftruncate a file with preallocated space doesn't change number of the blocks used by the file, which means the space not used by the file is not returned to the filesystems. So in this case we need fallocate with FALLOC_FL_PUNCH_HOLE to explicitly reclaim the used blocks. It is a hack to cope with the kernel bug and usually we should not need it. Closes https://github.com/facebook/rocksdb/pull/2102 Differential Revision: D4848934 Pulled By: lightmark fbshipit-source-id: f1b40b5
-
- 06 4月, 2017 2 次提交
-
-
由 Siying Dong 提交于
Summary: Move some files under util/ to new directories env/, monitoring/ options/ and cache/ Closes https://github.com/facebook/rocksdb/pull/2090 Differential Revision: D4833681 Pulled By: siying fbshipit-source-id: 2fd8bef
-
由 Aaron Gao 提交于
Summary: This reverts commit 0fd57492. It breaks tmpfs on kernel 4.0 or earlier. We will wait for the fix before remove this part Closes https://github.com/facebook/rocksdb/pull/2096 Differential Revision: D4839661 Pulled By: lightmark fbshipit-source-id: 574a51f
-
- 29 3月, 2017 1 次提交
-
-
由 Aaron Gao 提交于
Summary: As discuss in this thread: https://www.facebook.com/groups/rocksdb.dev/permalink/1218043868294125/ We remove fallocate with FALLOC_FL_PUNCH_HOLE because the recent bug on xfs in kernel 4.x+ that align file size to page size even with FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE. Closes https://github.com/facebook/rocksdb/pull/2038 Differential Revision: D4779974 Pulled By: siying fbshipit-source-id: 5f54625
-
- 24 2月, 2017 1 次提交
-
-
由 Aaron Gao 提交于
Summary: querying logical sector size from the device instead of hardcoding it for linux platform. Closes https://github.com/facebook/rocksdb/pull/1875 Differential Revision: D4591946 Pulled By: ajkr fbshipit-source-id: 4e9805c
-
- 23 2月, 2017 1 次提交
-
-
由 Aaron Gao 提交于
Summary: we occasionally missing this call so the file size will be wrong Closes https://github.com/facebook/rocksdb/pull/1894 Differential Revision: D4598446 Pulled By: lightmark fbshipit-source-id: 42b6ef5
-
- 03 2月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: The recent update about fallocate failed OSX build. Fix it. Closes https://github.com/facebook/rocksdb/pull/1830 Differential Revision: D4500235 Pulled By: siying fbshipit-source-id: a5f2b40
-
- 02 2月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/1824 Differential Revision: D4493862 Pulled By: siying fbshipit-source-id: c168446
-
- 19 1月, 2017 1 次提交
-
-
由 Yi Wu 提交于
Summary: Remove the logic since we don't use buffer cache with direct IO. Resolve read regression we currently have. Closes https://github.com/facebook/rocksdb/pull/1782 Differential Revision: D4430408 Pulled By: yiwu-arbug fbshipit-source-id: 5557bba
-
- 14 1月, 2017 1 次提交
-
-
由 Aaron Gao 提交于
Summary: also change variable name `direct_io_` to `use_direct_io_` in WritableFile to make it consistent with read path. Closes https://github.com/facebook/rocksdb/pull/1770 Differential Revision: D4416435 Pulled By: lightmark fbshipit-source-id: 4143c53
-
- 13 1月, 2017 1 次提交
-
-
由 Aaron Gao 提交于
Summary: add build guard Closes https://github.com/facebook/rocksdb/pull/1771 Differential Revision: D4410779 Pulled By: siying fbshipit-source-id: 3796c30
-
- 12 1月, 2017 2 次提交
-
-
由 Aaron Gao 提交于
Summary: direct IO reads refactoring remove unnecessary classes and unified interfaces tested with db_bench need more change for options and ON/OFF for different files. Since disabled is default, it should be fine now Closes https://github.com/facebook/rocksdb/pull/1636 Differential Revision: D4307189 Pulled By: lightmark fbshipit-source-id: 6991e22
-
由 Anirban Rahut 提交于
Summary: …perly on travis There is some old code in PosixWritableFile::Close(), which truncates the file to the measured size and then does an extra fallocate with KEEP_SIZE. This is commented as a failsafe because in some cases ftruncate doesn't do the right job (I don't know of an instance of this btw). However doing an fallocate with KEEP_SIZE should not increase the file size. However on Travis Worker which is Docker (likely AUFS ) its not working. There are comments on web that show that the AUFS author had initially not implemented fallocate, and then did it later. So not sure what is the quality of the implementation. Closes https://github.com/facebook/rocksdb/pull/1765 Differential Revision: D4401340 Pulled By: anirbanr-fb fbshipit-source-id: e2d8100
-
- 23 12月, 2016 1 次提交
-
-
由 Aaron Gao 提交于
Summary: rocksdb direct io support ``` [gzh@dev11575.prn2 ~/rocksdb] ./db_bench -benchmarks=fillseq --num=1000000 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags RocksDB: version 5.0 Date: Wed Nov 23 13:17:43 2016 CPU: 40 * Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz CPUCache: 25600 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Compression: Snappy Memtablerep: skip_list Perf Level: 1 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags DB path: [/tmp/rocksdbtest-112628/dbbench] fillseq : 4.393 micros/op 227639 ops/sec; 25.2 MB/s [gzh@dev11575.prn2 ~/roc Closes https://github.com/facebook/rocksdb/pull/1564 Differential Revision: D4241093 Pulled By: lightmark fbshipit-source-id: 98c29e3
-
- 19 11月, 2016 2 次提交
-
-
由 Siying Dong 提交于
Summary: Currently, in the Direct I/O read mode, the last sector of the file, if not full, is not handled correctly. If the return value of pread is not multiplier of kSectorSize, we still go ahead and continue reading, even if the buffer is not aligned. With the commit, if the return value is not multiplier of kSectorSize, and all but the last sector has been read, we simply return. Closes https://github.com/facebook/rocksdb/pull/1550 Differential Revision: D4209609 Pulled By: lightmark fbshipit-source-id: cb0b439
-
由 Maysam Yabandeh 提交于
Summary: This patch clarifies the contract of PositionedAppend with some unit tests and also implements it for PosixWritableFile. (Tasks: 14524071) Closes https://github.com/facebook/rocksdb/pull/1514 Differential Revision: D4204907 Pulled By: maysamyabandeh fbshipit-source-id: 06eabd2
-
- 14 9月, 2016 1 次提交
-
-
由 Islam AbdelRahman 提交于
Summary: Add Env::RandomRWFile in env.h and implement it for POSIX RandomRWFile is a file that allow us to read from / write to random offsets in the file I will implement it for other Envs later after finishing the whole task for AddFile() Test Plan: unit tests Reviewers: andrewkr, kradhakrishnan, sdong Reviewed By: sdong Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D62433
-
- 02 8月, 2016 1 次提交
-
-
由 krad 提交于
Summary: The patch is a continuation of part 5. It glues the abstraction for file layout and metadata, and flush out the implementation of the API. It adds unit tests for the implementation. Test Plan: Run unit tests Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57549
-
- 24 5月, 2016 1 次提交
-
-
由 krad 提交于
Summary: This patch adds direct IO capability to RocksDB Env. The direct IO capability is required for persistent cache since NVM is best accessed as 4K direct IO. SSDs can leverage direct IO for reading. Direct IO requires the offset and size be sector size aligned, and memory to be kernel page aligned. Since neither RocksDB/Persistent read cache data layout is aligned to sector size, the code can accommodate reading unaligned IO size (or unaligned memory) at the cost of an alloc/copy. The write code path expects the size and memory to be aligned. Test Plan: Run RocksDB unit tests Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D57393
-
- 20 5月, 2016 1 次提交
-
-
由 krad 提交于
Summary: Persistent read cache relies on the accuracy of the GetUniqueIdFromFile to generate a unique key for a given block of data. Currently we don't have an implementation for Mac. This patch adds an implementation. Test Plan: Run unit tests Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D58413
-
- 16 5月, 2016 1 次提交
-
-
由 krad 提交于
Summary: Added a new abstraction to cache page to RocksDB designed for the read cache use. RocksDB current block cache is more of an object cache. For the persistent read cache project, what we need is a page cache equivalent. This changes adds a cache abstraction to RocksDB to cache pages called PersistentCache. PersistentCache can cache uncompressed pages or raw pages (content as in filesystem). The user can choose to operate PersistentCache either in COMPRESSED or UNCOMPRESSED mode. Blame Rev: Test Plan: Run unit tests Reviewers: sdong Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D55707
-
- 10 2月, 2016 1 次提交
-
-
由 Baraa Hamodi 提交于
-
- 11 11月, 2015 2 次提交
-
-
由 Dmitri Smirnov 提交于
in 64-bit. Currently, a signed off_t type is being used for the following interfaces for both offset and the length in bytes: * `Allocate` * `RangeSync` On Linux `off_t` is automatically either 32 or 64-bit depending on the platform. On Windows it is always a 32-bit signed long which limits file access and in particular space pre-allocation to effectively 2 Gb. Proposal is to replace off_t with uint64_t as a portable type always access files with 64-bit interfaces. May need to modify posix code but lack resources to test it.
-
由 Dmitri Smirnov 提交于
in 64-bit. Currently, a signed off_t type is being used for the following interfaces for both offset and the length in bytes: * `Allocate` * `RangeSync` On Linux `off_t` is automatically either 32 or 64-bit depending on the platform. On Windows it is always a 32-bit signed long which limits file access and in particular space pre-allocation to effectively 2 Gb. Proposal is to replace off_t with uint64_t as a portable type always access files with 64-bit interfaces. May need to modify posix code but lack resources to test it.
-
- 29 10月, 2015 2 次提交
-
-
由 sdong 提交于
Summary: include/posix/io_posix.h is not a public API. Although include/posix/ is not a public header directory, it is confusing to put non-public headers to under include/. Move it to util/ to be clearer. Test Plan: Run all tests Reviewers: rven, IslamAbdelRahman, anthony, kradhakrishnan, yhchiang, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49611
-
- 28 10月, 2015 2 次提交
-
-
由 sdong 提交于
Summary: include/posix/io_posix.h should not depend on ROCKSDB_FALLOCATE_PRESENT. Remove it. Test Plan: Build it with both of ROCKSDB_FALLOCATE_PRESENT defined and not defined. Reviewers: rven, yhchiang, anthony, kradhakrishnan, IslamAbdelRahman, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49563
-
由 sdong 提交于
Summary: Mac build breaks as include/posix/io_posix.h doesn't include errno. Move the exact function declaration to io_posix.cc Test Plan: Run all test. Will run on Mac Reviewers: rven, anthony, yhchiang, IslamAbdelRahman, igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49551
-
- 27 10月, 2015 1 次提交
-
-
由 sdong 提交于
Summary: IO Posix depends on too many .h files. Move most of them to .cc files. Test Plan: make all Reviewers: anthony, rven, IslamAbdelRahman, yhchiang, kradhakrishnan, igor Reviewed By: igor Subscribers: leveldb, dhruba Differential Revision: https://reviews.facebook.net/D49479
-
- 22 10月, 2015 1 次提交
-
-
由 Javier González 提交于
Summary: This patch splits the posix storage backend into Env and the actual *File implementations. The motivation is to allow other Envs to use posix as a library. This enables a storage backend different from posix to split its secondary storage between a normal file system partition managed by posix, and it own media. Test Plan: No new functionality is added to posix Env or the library, thus the current tests should suffice.
-