-[175b_samples.jsonl](175b_samples.jsonl) - Unconditional, unfiltered 2048 token samples from GPT-3 with p=.85, t=1.　
**CONTENT WARNING:** GPT-3 was trained on arbitrary data from the web, so may contain offensive content and language.
-[data](data) - Synthetic datasets for word scramble and arithmetic tasks described in the paper.
-[dataset_statistics](dataset_statistics) - Statistics for all languages included in the training dataset mix.
-[overlap_frequency.md](overlap_frequency.md) - Samples of 13-gram overlaps between our training data and benchmarks, selected by frequency in the training set.
We provide statistics about the relative and absolute prevalence of different languages in the dataset mix used during training of GPT-3.
The concepts of "characters" and "words" can have different meanings in different languages, so any effort to count is imperfect, but our hope is that this provides helpful information to our readers nonetheless. To help support a wide variety of downstream analyses, we provide language-level summary counts broken down at the unicode-character level, whitespace-delineated word level, and document level.
-[Languages by character count](languages_by_character_count.csv)
-[Languages by word count](languages_by_word_count.csv)
-[Languages by document count](languages_by_document_count.csv)
