diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index ebc4c1c1c2d0518291bfab5341c2969ddeaaccc2..040c9ecae55c59a6ba7049f5602b88af4295db3a 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -3,6 +3,11 @@ You can view information and options set for each of the mounted NFS file systems by running `nfsstat -m` and `cat /etc/fstab`. +NOTE: **Note:** Filesystem performance has a big impact on overall GitLab +performance, especially for actions that read or write to Git repositories. See +[Filesystem Performance Benchmarking](../operations/filesystem_benchmarking.md) +for steps to test filesystem performance. + ## NFS Server features ### Required features diff --git a/doc/administration/operations/filesystem_benchmarking.md b/doc/administration/operations/filesystem_benchmarking.md new file mode 100644 index 0000000000000000000000000000000000000000..44018e966e037fee5de8356fe98c6052ad435076 --- /dev/null +++ b/doc/administration/operations/filesystem_benchmarking.md @@ -0,0 +1,55 @@ +# Filesystem Performance Benchmarking + +Filesystem performance has a big impact on overall GitLab performance, +especially for actions that read or write to Git repositories. This information +will help benchmark filesystem performance against known good and bad real-world +systems. + +Normally when talking about filesystem performance the biggest concern is +with Network Filesystems (NFS). However, even some local disks can have slow +IO. The information on this page can be used for either scenario. + +## Write Performance + +The following one-line command is a quick benchmark for filesystem write +performance. This will write 1,000 small files to the directory in which it is +executed. + +1. Change into the root of the appropriate + [repository storage path](../repository_storage_paths.md). +1. Create a temporary directory for the test so it's easy to remove the files later: + + ```sh + mkdir test; cd test + ``` +1. Run the command: + + ```sh + time for i in {0..1000}; do echo 'test' > "test${i}.txt"; done + ``` +1. Remove the test files: + + ```sh + cd ../; rm -rf test + ``` + +The output of the `time for ...` command will look similar to the following. The +important metric is the `real` time. + +```sh +$ time for i in {0..1000}; do echo 'test' > "test${i}.txt"; done + +real 0m0.116s +user 0m0.025s +sys 0m0.091s +``` + +From experience with multiple customers, the following are ranges that indicate +whether your filesystem performance is satisfactory or less than ideal: + +| Rating | Benchmark result | +|:----------|:------------------------| +| Best | Less than 10 seconds | +| OK | 10-18 seconds | +| Poor | 18-25 seconds | +| Very poor | Greater than 25 seconds | diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md index dea98cb8197892c6517717f6f9ccc143d81c3a3f..a16fc7ae74f483fd519c5debff3c7678adfb43a3 100644 --- a/doc/administration/operations/index.md +++ b/doc/administration/operations/index.md @@ -16,3 +16,7 @@ to restart Sidekiq. indexed lookup to the GitLab database](fast_ssh_key_lookup.md), and/or by [doing away with user SSH keys stored on GitLab entirely in favor of SSH certificates](ssh_certificates.md). +- [Filesystem Performance Benchmarking](filesystem_benchmarking.md): Filesystem +performance can have a big impact on GitLab performance, especially for actions +that read or write Git repositories. This information will help benchmark +filesystem performance against known good and bad real-world systems.