提交 9da87480 编写于 作者: M Mark Callaghan

Get benchmark.sh loads to run faster

Summary:
This changes loads to use vector memtable and disable the WAL. This also
increases the chance we will see IO bottlenecks during loads which is good to stress
test HW. But I also think it is a good way to load data quickly as this is a bulk
operation and the WAL isn't needed.

The two numbers below are the MB/sec rates for fillseq, bulkload using a skiplist
or vector memtable and the WAL enabled or disabled. There is a big benefit from
using the vector memtable and WAL disabled. Alas there is also a perf bug in
the use of std::sort for ordered input when the vector is flushed. Task is open
for that.
  112, 66 - skiplist with wal
  250, 116 - skiplist without wal
  110, 108 - vector with wal
  232, 370 - vector without wal

Task ID: #

Blame Rev:

Test Plan:
Revert Plan:

Database Impact:

Memcache Impact:

Other Notes:

EImportant:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D36957
上级 9b983bef
...@@ -124,6 +124,8 @@ function summarize_result { ...@@ -124,6 +124,8 @@ function summarize_result {
} }
function run_bulkload { function run_bulkload {
# This runs with a vector memtable and the WAL disabled to load faster. It is still crash safe and the
# client can discover where to restart a load after a crash. I think this is a good way to load.
echo "Bulk loading $num_keys random keys" echo "Bulk loading $num_keys random keys"
cmd="./db_bench --benchmarks=fillrandom \ cmd="./db_bench --benchmarks=fillrandom \
--use_existing_db=0 \ --use_existing_db=0 \
...@@ -131,6 +133,8 @@ function run_bulkload { ...@@ -131,6 +133,8 @@ function run_bulkload {
--sync=0 \ --sync=0 \
$params_bulkload \ $params_bulkload \
--threads=1 \ --threads=1 \
--memtablerep=vector \
--disable_wal=1 \
2>&1 | tee -a $output_dir/benchmark_bulkload_fillrandom.log" 2>&1 | tee -a $output_dir/benchmark_bulkload_fillrandom.log"
echo $cmd | tee $output_dir/benchmark_bulkload_fillrandom.log echo $cmd | tee $output_dir/benchmark_bulkload_fillrandom.log
eval $cmd eval $cmd
...@@ -148,6 +152,8 @@ function run_bulkload { ...@@ -148,6 +152,8 @@ function run_bulkload {
} }
function run_fillseq { function run_fillseq {
# This runs with a vector memtable and the WAL disabled to load faster. It is still crash safe and the
# client can discover where to restart a load after a crash. I think this is a good way to load.
echo "Loading $num_keys keys sequentially" echo "Loading $num_keys keys sequentially"
cmd="./db_bench --benchmarks=fillseq \ cmd="./db_bench --benchmarks=fillseq \
--use_existing_db=0 \ --use_existing_db=0 \
...@@ -155,6 +161,8 @@ function run_fillseq { ...@@ -155,6 +161,8 @@ function run_fillseq {
$params_w \ $params_w \
--min_level_to_compress=0 \ --min_level_to_compress=0 \
--threads=1 \ --threads=1 \
--memtablerep=vector \
--disable_wal=1 \
2>&1 | tee -a $output_dir/benchmark_fillseq.log" 2>&1 | tee -a $output_dir/benchmark_fillseq.log"
echo $cmd | tee $output_dir/benchmark_fillseq.log echo $cmd | tee $output_dir/benchmark_fillseq.log
eval $cmd eval $cmd
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册