- 18 4月, 2020 1 次提交
-
-
由 Yanqin Jin 提交于
Summary: IsDirectory() is a common API to check whether a path is a regular file or directory. POSIX: call stat() and use S_ISDIR(st_mode) Windows: PathIsDirectoryA() and PathIsDirectoryW() HDFS: FileSystem.IsDirectory() Java: File.IsDirectory() ... Pull Request resolved: https://github.com/facebook/rocksdb/pull/6711 Test Plan: make check Reviewed By: anand1976 Differential Revision: D21053520 Pulled By: riversand963 fbshipit-source-id: 680aadfd8ce982b63689190cf31b3145d5a89e27
-
- 21 3月, 2020 1 次提交
-
-
由 Cheng Chang 提交于
Summary: By supporting direct IO in RandomAccessFileReader::MultiRead, the benefits of parallel IO (IO uring) and direct IO can be combined. In direct IO mode, read requests are aligned and merged together before being issued to RandomAccessFile::MultiRead, so blocks in the original requests might share the same underlying buffer, the shared buffers are returned in `aligned_bufs`, which is a new parameter of the `MultiRead` API. For example, suppose alignment requirement for direct IO is 4KB, one request is (offset: 1KB, len: 1KB), another request is (offset: 3KB, len: 1KB), then since they all belong to page (offset: 0, len: 4KB), `MultiRead` only reads the page with direct IO into a buffer on heap, and returns 2 Slices referencing regions in that same buffer. See `random_access_file_reader_test.cc` for more examples. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6446 Test Plan: Added a new test `random_access_file_reader_test.cc`. Reviewed By: anand1976 Differential Revision: D20097518 Pulled By: cheng-chang fbshipit-source-id: ca48a8faf9c3af146465c102ef6b266a363e78d1
-
- 21 2月, 2020 1 次提交
-
-
由 sdong 提交于
Summary: When dynamically linking two binaries together, different builds of RocksDB from two sources might cause errors. To provide a tool for user to solve the problem, the RocksDB namespace is changed to a flag which can be overridden in build time. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6433 Test Plan: Build release, all and jtest. Try to build with ROCKSDB_NAMESPACE with another flag. Differential Revision: D19977691 fbshipit-source-id: aa7f2d0972e1c31d75339ac48478f34f6cfcfb3e
-
- 08 1月, 2020 1 次提交
-
-
由 Huisheng Liu 提交于
Summary: A new interface method Env::GetFreeSpace was added in https://github.com/facebook/rocksdb/issues/4164. It needs to be implemented for Windows port. Some error_handler_test cases fail on Windows because recovery cannot succeed without free space being reported. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6265 Differential Revision: D19303065 fbshipit-source-id: 1f1a83e53f334284781cf61feabc996e87b945ca
-
- 21 9月, 2019 1 次提交
-
-
由 sdong 提交于
Summary: Further apply formatter to more recent commits. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5830 Test Plan: Run all existing tests. Differential Revision: D17488031 fbshipit-source-id: 137458fd94d56dd271b8b40c522b03036943a2ab
-
- 16 7月, 2019 1 次提交
-
-
由 Tomas Kolda 提交于
Summary: Fixes https://github.com/facebook/rocksdb/issues/5552 Pull Request resolved: https://github.com/facebook/rocksdb/pull/5557 Differential Revision: D16266329 fbshipit-source-id: a8f6b50298a6f7c8d6c7e172bb26dd7eb6bd8a4d
-
- 05 4月, 2019 1 次提交
-
-
由 datonli 提交于
#5145 , rename port/dirent.h to port/port_dirent.h to avoid compile err when use port dir as header dir output (#5152) Summary: mv port/dirent.h to port/port_dirent.h to avoid compile err when use port dir as header dir output Pull Request resolved: https://github.com/facebook/rocksdb/pull/5152 Differential Revision: D14779409 Pulled By: siying fbshipit-source-id: d4162c47c979c6e8cc6a9e601802864ab3768ecb
-
- 28 3月, 2019 1 次提交
-
-
由 Burton Li 提交于
Summary: The existing code for env_win src and header file doesn't fully followed the recommended code style (https://google.github.io/styleguide/cppguide.html#Functions). Fix it for better readability. anand1976 siying Pull Request resolved: https://github.com/facebook/rocksdb/pull/5096 Differential Revision: D14585358 Pulled By: anand1976 fbshipit-source-id: 7ce35ffe9e922f5c1421b0bbaa5fce7abad57617
-
- 22 3月, 2019 1 次提交
-
-
由 Burton Li 提交于
Summary: The original implementation of WinEnvIO::NowNanos() has a constant data overflow by: li.QuadPart *= std::nano::den; As a result, the api provides a incorrect result. e.g.: li.QuadPart=13477844301545 std::nano::den=1e9 The fix uses pre-computed nano_seconds_per_period_ to present the nano seconds per performance counter period, in the case if nano::den is divisible by perf_counter_frequency_. Otherwise it falls back to use high_resolution_clock. siying ajkr Pull Request resolved: https://github.com/facebook/rocksdb/pull/5062 Differential Revision: D14426842 Pulled By: anand1976 fbshipit-source-id: 127f1daf423dd4b30edd0dcf8ea0466f468bec12
-
- 12 10月, 2018 1 次提交
-
-
由 Wilfried Goesgens 提交于
Summary: The default behaviour of rocksdb is to use the `*A(` windows API functions. These accept filenames in the currently configured system encoding, be it Latin 1, utf8 or whatever. If the Application intends to completely work with utf8 strings internally, converting these to that codepage properly isn't even always possible. Thus this patch adds a switch to use the `*W(` functions, which accept UTF-16 filenames, and uses C++11 features to translate the UTF8 containing std::string to an UTF16 containing std::wstring. This feature is a compile time options, that can be enabled by setting `WITH_WINDOWS_UTF8_FILENAMES` to true. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4469 Differential Revision: D10356011 Pulled By: yiwu-arbug fbshipit-source-id: 27b6ae9171f209085894cdf80069e8a896642044
-
- 15 9月, 2018 1 次提交
-
-
由 Constantin Belyaev 提交于
Summary: Now port/win_env.cc do check error for cross device link creation. Fixes #4364 Pull Request resolved: https://github.com/facebook/rocksdb/pull/4365 Differential Revision: D9833144 Pulled By: ajkr fbshipit-source-id: be7555e510f4b8d2196d843841606a6cfada7644
-
- 06 9月, 2018 1 次提交
-
-
由 cngzhnp 提交于
Summary: As you know, almost all compilers support "pragma once" keyword instead of using include guards. To be keep consistency between header files, all header files are edited. Besides this, try to fix some warnings about loss of data. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4339 Differential Revision: D9654990 Pulled By: ajkr fbshipit-source-id: c2cf3d2d03a599847684bed81378c401920ca848
-
- 24 8月, 2018 1 次提交
-
-
由 Yanqin Jin 提交于
Summary: We want to sample the file I/O issued by RocksDB and report the function calls. This requires us to include the file paths otherwise it's hard to tell what has been going on. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4039 Differential Revision: D8670178 Pulled By: riversand963 fbshipit-source-id: 97ee806d1c583a2983e28e213ee764dc6ac28f7a
-
- 10 8月, 2018 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: Although delete scheduler implementation allows for the interface not to be supported, the delete_scheduler_test does not allow for that. Address compiler warnings Make sst_dump_test use test directory structure as the current execution directory may not be writiable. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4221 Differential Revision: D9210152 Pulled By: siying fbshipit-source-id: 381a74511e969ecb8089d5c4b4df87dc30c8df63
-
- 13 6月, 2018 1 次提交
-
-
由 shpala 提交于
Summary: Fix a crash in `WinEnvIO::GetSectorSize` that happens on old Windows systems (e.g Windows 7). On old Windows systems that don't support querying StorageAccessAlignmentProperty using IOCTL_STORAGE_QUERY_PROPERTY, the flow calls a different DeviceIoControl with nullptr as lpBytesReturned. When the code reaches this point, we get an access violation. Closes https://github.com/facebook/rocksdb/pull/3975 Differential Revision: D8385186 Pulled By: ajkr fbshipit-source-id: fae4c9b4b0a52c8a10182e1b35bcaa30dc393bbb
-
- 25 5月, 2018 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: Catch up with Posix features NewWritableRWFile must fail when file does not exists Implement Env::Truncate() Adjust Env options optimization functions Implement MemoryMappedBuffer on Windows. Closes https://github.com/facebook/rocksdb/pull/3857 Differential Revision: D8053610 Pulled By: ajkr fbshipit-source-id: ccd0d46c29648a9f6f496873bc1c9d6c5547487e
-
- 04 5月, 2018 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: Delete archive directory before WAL folder since archive may be contained as a subfolder. Also improve loop readability. Closes https://github.com/facebook/rocksdb/pull/3797 Differential Revision: D7866378 Pulled By: riversand963 fbshipit-source-id: 0c45d97677ce6fbefa3f8d602ef5e2a2a925e6f5
-
- 07 3月, 2018 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: This patch addressed several issues. Portability including db_test std::thread -> port::Thread Cc: @ and %z to ROCKSDB portable macro. Cc: maysamyabandeh Implement Env::AreFilesSame Make the implementation of file unique number more robust Get rid of C-runtime and go directly to Windows API when dealing with file primitives. Implement GetSectorSize() and aling unbuffered read on the value if available. Adjust Windows Logger for the new interface, implement CloseImpl() Cc: anand1976 Fix test running script issue where $status var was of incorrect scope so the failures were swallowed and not reported. DestroyDB() creates a logger and opens a LOG file in the directory being cleaned up. This holds a lock on the folder and the cleanup is prevented. This fails one of the checkpoin tests. We observe the same in production. We close the log file in this change. Fix DBTest2.ReadAmpBitmapLiveInCacheAfterDBClose failure where the test attempts to open a directory with NewRandomAccessFile which does not work on Windows. Fix DBTest.SoftLimit as it is dependent on thread timing. CC: yiwu-arbug Closes https://github.com/facebook/rocksdb/pull/3552 Differential Revision: D7156304 Pulled By: siying fbshipit-source-id: 43db0a757f1dfceffeb2b7988043156639173f5b
-
- 03 2月, 2018 1 次提交
-
-
由 Tamir Duberstein 提交于
Summary: - Use `__unused__` everywhere - Suppress unused warnings in Release mode + This currently affects non-MSVC builds (e.g. mingw64). Closes https://github.com/facebook/rocksdb/pull/3448 Differential Revision: D6885496 Pulled By: miasantreble fbshipit-source-id: f2f6adacec940cc3851a9eee328fafbf61aad211
-
- 22 12月, 2017 1 次提交
-
-
由 burtonli 提交于
Summary: FILE_FLAG_WRITE_THROUGH is for disabling device on-board cache in windows API, which should be disabled if user doesn't need system cache. There was a perf issue related with this, we found during memtable flush, the high percentile latency jumps significantly. During profiling, we found those high latency (P99.9) read requests got queue-jumped by write requests from memtable flush and takes 80ms or even more time to wait, even when SSD overall IO throughput is relatively low. After enabling FILE_FLAG_WRITE_THROUGH, we rerun the test found high percentile latency drops a lot without observable impact on writes. Scenario 1: 40MB/s + 40MB/s R/W compaction throughput Original | FILE_FLAG_WRITE_THROUGH | Percentage reduction --------------------------------------------------------------- P99.9 | 56.897 ms | 35.593 ms | -37.4% P99 | 3.905 ms | 3.896 ms | -2.8% Scenario 2: 14MB/s + 14MB/s R/W compaction throughput, cohosted with 100+ other rocksdb instances have manually triggered memtable flush operations (memtable is tiny), creating a lot of randomized the small file writes operations during test. Original | FILE_FLAG_WRITE_THROUGH | Percentage reduction --------------------------------------------------------------- P99.9 | 86.227 ms | 50.436 ms | -41.5% P99 | 8.415 ms | 3.356 ms | -60.1% Closes https://github.com/facebook/rocksdb/pull/3225 Differential Revision: D6624174 Pulled By: miasantreble fbshipit-source-id: 321b86aee9d74470840c70e5d0d4fa9880660a91
-
- 01 11月, 2017 1 次提交
-
-
由 Shaohua Li 提交于
Summary: The DBOptions::writable_file_max_buffer_size can be changed dynamically. Closes https://github.com/facebook/rocksdb/pull/3053 Differential Revision: D6152720 Pulled By: shligit fbshipit-source-id: aa0c0cfcfae6a54eb17faadb148d904797c68681
-
- 24 10月, 2017 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: MSVC does not support unused attribute at this time. A separate assignment line fixes the issue probably by being counted as usage for MSVC and it no longer complains about unused var. Closes https://github.com/facebook/rocksdb/pull/3048 Differential Revision: D6126272 Pulled By: maysamyabandeh fbshipit-source-id: 4907865db45fd75a39a15725c0695aaa17509c1f
-
- 09 8月, 2017 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2702 Differential Revision: D5585389 Pulled By: ajkr fbshipit-source-id: cb54041eb481d0d759c440f82a8a2c5b34534173
-
- 16 7月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2589 Differential Revision: D5431502 Pulled By: siying fbshipit-source-id: 8ebf8c87883daa9daa54b2303d11ce01ab1f6f75
-
- 21 6月, 2017 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: Make default impl return NoSupported so the db_blob tests exist in a meaningful manner. Replace std::thread to port::Thread Closes https://github.com/facebook/rocksdb/pull/2465 Differential Revision: D5275563 Pulled By: yiwu-arbug fbshipit-source-id: cedf1a18a2c05e20d768c1308b3f3224dbd70ab6
-
- 13 6月, 2017 1 次提交
-
-
由 Orgad Shaneh 提交于
Summary: cc tamird sagar0 Closes https://github.com/facebook/rocksdb/pull/2325 Differential Revision: D5098302 Pulled By: sagar0 fbshipit-source-id: 297c5506b5d9b2ed1d7719c8caf0b96cffe503b8
-
- 24 5月, 2017 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: Previously users could set `max_background_flushes=0` to force rocksdb to use a single thread pool for both background flushes and compactions. That'll no longer be possible since I'm going to deprecate `max_background_flushes` and `max_background_compactions` in favor of a single option. This diff introduces a new way to force a single thread pool: when high-pri pool has zero threads, all background jobs will be submitted to low-pri pool. Note the majority of the code change is adding `Env::GetBackgroundThreads()`, which is necessary to check whether the user has provided a zero-sized thread pool. Closes https://github.com/facebook/rocksdb/pull/2204 Differential Revision: D4936256 Pulled By: ajkr fbshipit-source-id: 929a07a0c0705f7766f5339cd013ff74e90d6e01
-
- 18 5月, 2017 1 次提交
-
-
由 Tamir Duberstein 提交于
Summary: This was exposed by a48a62d5, which made NDEBUG the default for cmake builds. Closes https://github.com/facebook/rocksdb/pull/2315 Differential Revision: D5079583 Pulled By: sagar0 fbshipit-source-id: c614e96a40df016a834a62b6236852265e7ee4db
-
- 06 5月, 2017 1 次提交
-
-
由 Tamir Duberstein 提交于
Summary: - downcase includes for case-sensitive filesystems - give targets the same name (librocksdb) on all platforms With this patch it is possible to cross-compile RocksDB for Windows from a Linux host using mingw. cc yuslepukhin orgads Closes https://github.com/facebook/rocksdb/pull/2107 Differential Revision: D4849784 Pulled By: siying fbshipit-source-id: ad26ed6b4d393851aa6551e6aa4201faba82ef60
-
- 28 4月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: Closes https://github.com/facebook/rocksdb/pull/2226 Differential Revision: D4967547 Pulled By: siying fbshipit-source-id: dd3b58ae1e7a106ab6bb6f37ab5c88575b125ab4
-
- 06 4月, 2017 1 次提交
-
-
由 Siying Dong 提交于
Summary: Move some files under util/ to new directories env/, monitoring/ options/ and cache/ Closes https://github.com/facebook/rocksdb/pull/2090 Differential Revision: D4833681 Pulled By: siying fbshipit-source-id: 2fd8bef
-
- 31 3月, 2017 1 次提交
-
-
由 Orgad Shaneh 提交于
Summary: There still are many warnings (most of them about invalid printf format for long long), but it builds if FAIL_ON_WARNINGS is disabled. Closes https://github.com/facebook/rocksdb/pull/2052 Differential Revision: D4807355 Pulled By: siying fbshipit-source-id: ef03786
-
- 07 2月, 2017 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: introduce new methods into a public threadpool interface, - allow submission of std::functions as they allow greater flexibility. - add Joining methods to the implementation to join scheduled and submitted jobs with an option to cancel jobs that did not start executing. - Remove ugly `#ifdefs` between pthread and std implementation, make it uniform. - introduce pimpl for a drop in replacement of the implementation - Introduce rocksdb::port::Thread typedef which is a replacement for std::thread. On Posix Thread defaults as before std::thread. - Implement WindowsThread that allocates memory in a more controllable manner than windows std::thread with a replaceable implementation. - should be no functionality changes. Closes https://github.com/facebook/rocksdb/pull/1823 Differential Revision: D4492902 Pulled By: siying fbshipit-source-id: c74cb11
-
- 16 1月, 2017 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: Windows follow up for https://github.com/facebook/rocksdb/commit/dc2584eea0d9965f97d38d4d462a7b57750144d3 Differential Revision: D4420337 Pulled By: IslamAbdelRahman fbshipit-source-id: fedc5b5
-
- 10 1月, 2017 1 次提交
-
-
由 Dmitri Smirnov 提交于
Summary: Enable directIO on WritableFileImpl::Append with offset being current length of the file. Enable UniqueID tests on Windows, disable others but leeting them to compile. Unique tests are valuable to detect failures on different filesystems and upcoming ReFS. Clear output in WinEnv Getchildren.This is different from previous strategy, do not touch output on failure. Make sure DBTest.OpenWhenOpen works with windows error message Closes https://github.com/facebook/rocksdb/pull/1746 Differential Revision: D4385681 Pulled By: IslamAbdelRahman fbshipit-source-id: c07b702
-
- 23 12月, 2016 1 次提交
-
-
由 Aaron Gao 提交于
Summary: rocksdb direct io support ``` [gzh@dev11575.prn2 ~/rocksdb] ./db_bench -benchmarks=fillseq --num=1000000 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags RocksDB: version 5.0 Date: Wed Nov 23 13:17:43 2016 CPU: 40 * Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz CPUCache: 25600 KB Keys: 16 bytes each Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Compression: Snappy Memtablerep: skip_list Perf Level: 1 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags DB path: [/tmp/rocksdbtest-112628/dbbench] fillseq : 4.393 micros/op 227639 ops/sec; 25.2 MB/s [gzh@dev11575.prn2 ~/roc Closes https://github.com/facebook/rocksdb/pull/1564 Differential Revision: D4241093 Pulled By: lightmark fbshipit-source-id: 98c29e3
-
- 13 12月, 2016 1 次提交
-
-
由 Andrew Kryczka 提交于
Summary: It'd be nice to use the error status type to distinguish between user error and system error. For example, GetChildren can fail listing a backup directory's contents either because a bad path was provided (user error) or because an operation failed, e.g., a remote storage service call failed (system error). In the former case, we want to continue and treat the backup directory as empty; in the latter case, we want to immediately propagate the error to the caller. This diff uses NotFound to indicate user error and IOError to indicate system error. Previously IOError indicated both. Closes https://github.com/facebook/rocksdb/pull/1644 Differential Revision: D4312157 Pulled By: ajkr fbshipit-source-id: 51b4f24
-
- 14 10月, 2016 1 次提交
-
-
由 Dmitri Smirnov 提交于
-
- 20 5月, 2016 1 次提交
-
-
由 Dmitri Smirnov 提交于
For ease of reuse and customization as a library without wrapping. WinEnvThreads is a class for replacement. WintEnvIO is a class for reuse and behavior override. Added private virtual functions for custom override of fallocate pread for io classes.
-
- 17 5月, 2016 1 次提交
-
-
由 Dmitri Smirnov 提交于
Implement GetUniqueIdFromFile to support new tests and the feature.
-