Susan Zhang @suchenzang

books3 was absolutely necessary for llama to happen back in the day. you can even find my pathetic cleanup code for it here: https:// github.com/facebookresear ch/metaseq/tree/main/preprocessing/books3 … it was also noted to be a critical datasource for mmlu in the llama1 report, though by now we know mmlu is useless and high