Always preserve `None`-delimited groups in a captured `TokenStream`
Previously, we would silently remove any `None`-delimiters when capturing a `TokenStream`, 'flattenting' them to their inner tokens. This was not normally visible, since we usually have `TokenKind::Interpolated` (which gets converted to a `None`-delimited group during macro invocation) instead of an actual `None`-delimited group. However, there are a couple of cases where this becomes visible to proc-macros: 1. A cross-crate `macro_rules!` macro has a `None`-delimited group stored in its body (as a result of being produced by another `macro_rules!` macro). The cross-crate `macro_rules!` invocation can then expand to an attribute macro invocation, which needs to be able to see the `None`-delimited group. 2. A proc-macro can invoke an attribute proc-macro with its re-collected input. If there are any nonterminals present in the input, they will get re-collected to `None`-delimited groups, which will then get captured as part of the attribute macro invocation. Both of these cases are incredibly obscure, so there hopefully won't be any breakage. This change will allow more agressive 'flattenting' of nonterminals in #82608 without losing `None`-delimited groups.
Showing
想要评论请 注册 或 登录