Due to inlining it is possible to visit the same module multiple times
during `<Cache as DocFolder>::fold_crate`, so we keep track of the
modules we've already visited.