From be4b94be261a9978795dcb0e60aee4cf112412b3 Mon Sep 17 00:00:00 2001 From: Arash Ashari Date: Thu, 10 Sep 2020 00:39:47 -0700 Subject: [PATCH] Sparse attention: updating code tag in documentation (#394) Co-authored-by: Jeff Rasley --- docs/_pages/features.md | 6 +++--- docs/_tutorials/sparse-attention.md | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/_pages/features.md b/docs/_pages/features.md index 5eb341b7..344c7222 100755 --- a/docs/_pages/features.md +++ b/docs/_pages/features.md @@ -240,13 +240,13 @@ Please see the [core API doc](https://deepspeed.readthedocs.io/) for more detail } ``` ## Sparse Attention -DeepSpeed offers sparse attention to support long sequences. Please refer to the [Sparse Attention](/tutorials/sparse_attention/) tutorial. +DeepSpeed offers sparse attention to support long sequences. Please refer to the [Sparse Attention](/tutorials/sparse-attention/) tutorial. -```python +```bash --deepspeed_sparse_attention ``` -```python +```json "sparse_attention": { "mode": "fixed", "block": 16, diff --git a/docs/_tutorials/sparse-attention.md b/docs/_tutorials/sparse-attention.md index 5e402715..6279fe7c 100644 --- a/docs/_tutorials/sparse-attention.md +++ b/docs/_tutorials/sparse-attention.md @@ -115,7 +115,7 @@ if self.sparse_attention_config is not None and pad_len > 0: * **Enable sparse attention*: To use DeepSpeed Sparse Attention, you need to enable it in the launcher script through `deepspeed_sparse_attention` argument: -```python +```bash --deepspeed_sparse_attention ``` @@ -123,7 +123,7 @@ Please check [our bing_bert runner script](https://github.com/microsoft/DeepSpee * **Add sparsity config**: The sparsity config can be set through the [DeepSpeed JSON config file](https://github.com/microsoft/DeepSpeedExamples/blob/master/bing_bert/deepspeed_bsz64k_lamb_config_seq128.json). In this example, we have used `fixed` sparsity mode that will be described in [How to config sparsity structures](/tutorials/sparse-attention/#how-to-config-sparsity-structures) section. -```python +```json "sparse_attention": { "mode": "fixed", "block": 16, -- GitLab