# 使用`torchtext`的文本分类 > 原文： 本教程说明如何使用`torchtext`中的文本分类数据集，包括 ```py - AG_NEWS, - SogouNews, - DBpedia, - YelpReviewPolarity, - YelpReviewFull, - YahooAnswers, - AmazonReviewPolarity, - AmazonReviewFull ``` 此示例显示了如何使用这些`TextClassification`数据集之一训练用于分类的监督学习算法。 ## 使用 N 元组加载数据 一袋 N 元组特征用于捕获有关本地单词顺序的一些部分信息。 在实践中，应用二元语法或三元语法作为单词组比仅一个单词提供更多的好处。 一个例子： ```py "load data with ngrams" Bi-grams results: "load data", "data with", "with ngrams" Tri-grams results: "load data with", "data with ngrams" ``` `TextClassification`数据集支持`ngrams`方法。 通过将`ngrams`设置为 2，数据集中的示例文本将是一个单字加二元组字符串的列表。 ```py import torch import torchtext from torchtext.datasets import text_classification NGRAMS = 2 import os if not os.path.isdir('./.data'): os.mkdir('./.data') train_dataset, test_dataset = text_classification.DATASETS['AG_NEWS']( root='./.data', ngrams=NGRAMS, vocab=None) BATCH_SIZE = 16 device = torch.device("cuda" if torch.cuda.is_available() else "cpu") ``` ## 定义模型 该模型由[`EmbeddingBag`](https://pytorch.org/docs/stable/nn.html?highlight=embeddingbag#torch.nn.EmbeddingBag)层和线性层组成（请参见下图）。 `nn.EmbeddingBag`计算嵌入“袋”的平均值。 此处的文本条目具有不同的长度。 `nn.EmbeddingBag`此处不需要填充，因为文本长度以偏移量保存。 另外，由于`nn.EmbeddingBag`会动态累积嵌入中的平均值，因此`nn.EmbeddingBag`可以提高性能和存储效率，以处理张量序列。 ![../_img/text_sentiment_ngrams_model.png](img/30f766e7717c0e45a583a4f58ebc322a.png) ```py import torch.nn as nn import torch.nn.functional as F class TextSentiment(nn.Module): def __init__(self, vocab_size, embed_dim, num_class): super().__init__() self.embedding = nn.EmbeddingBag(vocab_size, embed_dim, sparse=True) self.fc = nn.Linear(embed_dim, num_class) self.init_weights() def init_weights(self): initrange = 0.5 self.embedding.weight.data.uniform_(-initrange, initrange) self.fc.weight.data.uniform_(-initrange, initrange) self.fc.bias.data.zero_() def forward(self, text, offsets): embedded = self.embedding(text, offsets) return self.fc(embedded) ``` ## 启动实例 `AG_NEWS`数据集具有四个标签，因此类别数是四个。 ```py 1 : World 2 : Sports 3 : Business 4 : Sci/Tec ``` 词汇的大小等于词汇的长度（包括单个单词和 N 元组）。 类的数量等于标签的数量，在`AG_NEWS`情况下为 4。 ```py VOCAB_SIZE = len(train_dataset.get_vocab()) EMBED_DIM = 32 NUN_CLASS = len(train_dataset.get_labels()) model = TextSentiment(VOCAB_SIZE, EMBED_DIM, NUN_CLASS).to(device) ``` ## 用于生成批量的函数 由于文本条目的长度不同，因此使用自定义函数`generate_batch()`生成数据批和偏移量。 该函数被传递到`torch.utils.data.DataLoader`中的`collate_fn`。 `collate_fn`的输入是张量列表，其大小为`batch_size`，`collate_fn`函数将它们打包成一个小批量。 请注意此处，并确保将`collate_fn`声明为顶级`def`。 这样可以确保该函数在每个工作程序中均可用。 原始数据批量输入中的文本条目打包到一个列表中，并作为单个张量级联，作为`nn.EmbeddingBag`的输入。 偏移量是定界符的张量，表示文本张量中各个序列的起始索引。 `Label`是一个张量，用于保存单个文本条目的标签。 ```py def generate_batch(batch): label = torch.tensor([entry[0] for entry in batch]) text = [entry[1] for entry in batch] offsets = [0] + [len(entry) for entry in text] # torch.Tensor.cumsum returns the cumulative sum # of elements in the dimension dim. # torch.Tensor([1.0, 2.0, 3.0]).cumsum(dim=0) offsets = torch.tensor(offsets[:-1]).cumsum(dim=0) text = torch.cat(text) return text, offsets, label ``` ## 定义函数来训练模型并评估结果 建议 PyTorch 用户使用[`torch.utils.data.DataLoader`](https://pytorch.org/docs/stable/data.html?highlight=dataloader#torch.utils.data.DataLoader)，它可以轻松地并行加载数据（[教程在这里](https://pytorch.org/tutorials/beginner/data_loading_tutorial.html)）。 我们在此处使用`DataLoader`加载`AG_NEWS`数据集，并将其发送到模型以进行训练/验证。 ```py from torch.utils.data import DataLoader def train_func(sub_train_): # Train the model train_loss = 0 train_acc = 0 data = DataLoader(sub_train_, batch_size=BATCH_SIZE, shuffle=True, collate_fn=generate_batch) for i, (text, offsets, cls) in enumerate(data): optimizer.zero_grad() text, offsets, cls = text.to(device), offsets.to(device), cls.to(device) output = model(text, offsets) loss = criterion(output, cls) train_loss += loss.item() loss.backward() optimizer.step() train_acc += (output.argmax(1) == cls).sum().item() # Adjust the learning rate scheduler.step() return train_loss / len(sub_train_), train_acc / len(sub_train_) def test(data_): loss = 0 acc = 0 data = DataLoader(data_, batch_size=BATCH_SIZE, collate_fn=generate_batch) for text, offsets, cls in data: text, offsets, cls = text.to(device), offsets.to(device), cls.to(device) with torch.no_grad(): output = model(text, offsets) loss = criterion(output, cls) loss += loss.item() acc += (output.argmax(1) == cls).sum().item() return loss / len(data_), acc / len(data_) ``` ## 分割数据集并运行模型 由于原始的`AG_NEWS`没有有效的数据集，因此我们将训练数据集分为训练/有效集，其分割比率为 0.95（训练）和 0.05（有效）。 在这里，我们在 PyTorch 核心库中使用[`torch.utils.data.dataset.random_split`](https://pytorch.org/docs/stable/data.html?highlight=random_split#torch.utils.data.random_split)函数。 [`CrossEntropyLoss`](https://pytorch.org/docs/stable/nn.html?highlight=crossentropyloss#torch.nn.CrossEntropyLoss)标准将`nn.LogSoftmax()`和`nn.NLLLoss()`合并到一个类中。 在训练带有`C`类的分类问题时很有用。 [`SGD`](https://pytorch.org/docs/stable/_modules/torch/optim/sgd.html)实现了随机梯度下降方法作为优化程序。 初始学习率设置为 4.0。 [`StepLR`](https://pytorch.org/docs/master/_modules/torch/optim/lr_scheduler.html#StepLR)在此处用于通过历时调整学习率。 ```py import time from torch.utils.data.dataset import random_split N_EPOCHS = 5 min_valid_loss = float('inf') criterion = torch.nn.CrossEntropyLoss().to(device) optimizer = torch.optim.SGD(model.parameters(), lr=4.0) scheduler = torch.optim.lr_scheduler.StepLR(optimizer, 1, gamma=0.9) train_len = int(len(train_dataset) * 0.95) sub_train_, sub_valid_ = \ random_split(train_dataset, [train_len, len(train_dataset) - train_len]) for epoch in range(N_EPOCHS): start_time = time.time() train_loss, train_acc = train_func(sub_train_) valid_loss, valid_acc = test(sub_valid_) secs = int(time.time() - start_time) mins = secs / 60 secs = secs % 60 print('Epoch: %d' %(epoch + 1), " | time in %d minutes, %d seconds" %(mins, secs)) print(f'\tLoss: {train_loss:.4f}(train)\t|\tAcc: {train_acc * 100:.1f}%(train)') print(f'\tLoss: {valid_loss:.4f}(valid)\t|\tAcc: {valid_acc * 100:.1f}%(valid)') ``` 出： ```py Epoch: 1 | time in 0 minutes, 11 seconds Loss: 0.0262(train) | Acc: 84.7%(train) Loss: 0.0002(valid) | Acc: 89.3%(valid) Epoch: 2 | time in 0 minutes, 11 seconds Loss: 0.0119(train) | Acc: 93.6%(train) Loss: 0.0002(valid) | Acc: 89.6%(valid) Epoch: 3 | time in 0 minutes, 11 seconds Loss: 0.0069(train) | Acc: 96.3%(train) Loss: 0.0000(valid) | Acc: 91.8%(valid) Epoch: 4 | time in 0 minutes, 11 seconds Loss: 0.0038(train) | Acc: 98.1%(train) Loss: 0.0000(valid) | Acc: 91.5%(valid) Epoch: 5 | time in 0 minutes, 11 seconds Loss: 0.0022(train) | Acc: 99.0%(train) Loss: 0.0000(valid) | Acc: 91.4%(valid) ``` 使用以下信息在 GPU 上运行模型： 周期：1 | 时间在 0 分 11 秒内 ```py Loss: 0.0263(train) | Acc: 84.5%(train) Loss: 0.0001(valid) | Acc: 89.0%(valid) ``` 周期：2 | 时间在 0 分钟 10 秒内 ```py Loss: 0.0119(train) | Acc: 93.6%(train) Loss: 0.0000(valid) | Acc: 89.6%(valid) ``` 周期：3 | 时间在 0 分钟 9 秒内 ```py Loss: 0.0069(train) | Acc: 96.4%(train) Loss: 0.0000(valid) | Acc: 90.5%(valid) ``` 周期：4 | 时间在 0 分 11 秒内 ```py Loss: 0.0038(train) | Acc: 98.2%(train) Loss: 0.0000(valid) | Acc: 90.4%(valid) ``` 周期：5 | 时间在 0 分 11 秒内 ```py Loss: 0.0022(train) | Acc: 99.0%(train) Loss: 0.0000(valid) | Acc: 91.0%(valid) ``` ## 使用测试数据集评估模型 ```py print('Checking the results of test dataset...') test_loss, test_acc = test(test_dataset) print(f'\tLoss: {test_loss:.4f}(test)\t|\tAcc: {test_acc * 100:.1f}%(test)') ``` 出： ```py Checking the results of test dataset... Loss: 0.0002(test) | Acc: 90.9%(test) ``` 正在检查测试数据集的结果… ```py Loss: 0.0237(test) | Acc: 90.5%(test) ``` ## 测试随机新闻 使用到目前为止最好的模型并测试高尔夫新闻。 标签信息在[这里](https://pytorch.org/text/datasets.html?highlight=ag_news#torchtext.datasets.AG_NEWS)。 ```py import re from torchtext.data.utils import ngrams_iterator from torchtext.data.utils import get_tokenizer ag_news_label = {1 : "World", 2 : "Sports", 3 : "Business", 4 : "Sci/Tec"} def predict(text, model, vocab, ngrams): tokenizer = get_tokenizer("basic_english") with torch.no_grad(): text = torch.tensor([vocab[token] for token in ngrams_iterator(tokenizer(text), ngrams)]) output = model(text, torch.tensor([0])) return output.argmax(1).item() + 1 ex_text_str = "MEMPHIS, Tenn. – Four days ago, Jon Rahm was \ enduring the season's worst weather conditions on Sunday at The \ Open on his way to a closing 75 at Royal Portrush, which \ considering the wind and the rain was a respectable showing. \ Thursday's first round at the WGC-FedEx St. Jude Invitational \ was another story. With temperatures in the mid-80s and hardly any \ wind, the Spaniard was 13 strokes better in a flawless round. \ Thanks to his best putting performance on the PGA Tour, Rahm \ finished with an 8-under 62 for a three-stroke lead, which \ was even more impressive considering he'd never played the \ front nine at TPC Southwind." vocab = train_dataset.get_vocab() model = model.to("cpu") print("This is a %s news" %ag_news_label[predict(ex_text_str, model, vocab, 2)]) ``` 出： ```py This is a Sports news ``` 这是体育新闻 [您可以在此处找到本说明中显示的代码示例](https://github.com/pytorch/text/tree/master/examples/text_classification)。 **脚本的总运行时间**：（1 分 38.483 秒） [下载 Python 源码：`text_sentiment_ngrams_tutorial.py`](../_downloads/1824f32965271d21829e1739cc434729/text_sentiment_ngrams_tutorial.py) [下载 Jupyter 笔记本：`text_sentiment_ngrams_tutorial.ipynb`](../_downloads/27bd42079e7f46673b53e90153168529/text_sentiment_ngrams_tutorial.ipynb) [由 Sphinx 画廊](https://sphinx-gallery.readthedocs.io)生成的画廊