During metadata loading, the AdtDefs for every ADT in the universe need
to be loaded (for example, for coherence of builtin traits). For that,
the attributes of the AdtDef need to be loaded too.

The attributes of a struct are duplicated between 2 def ids - the
constructor def-id, and the "type" def id. Loading attributes for both
def-ids, which was done in #53721, slowed the compilation of small
crates by 2-3%. This PR makes sure we only load the attributes for the
"type" def-id, avoiding the slowdown.