Improve Markdown code block tokens (#17591)

* Improve Markdown code block tokens

Code blocks without a language weren't tokenized.
Code blocks didn't have their ending ``` punctuation tokenized.
Code blocks used to only have one token. Now each block has the following tokens available for syntax highlighters:
- Starting and ending ``` punctuations
- Code block's language setting
- Code snippet

* Variable whitespace for MD code block ``` token

Allow for variable amount of whitespacing before ``` code blocks

* Reorder raw blocks

Raw blocks were preventing tokenizing as languaged blocks. Putting them on bottom resolves this.

* Fix MD block detection when following paragraph

Used to require a new line inbetween ``` code blocks and preceding paragraph text.

* Prevent broken language grammar leaks in MD fences

Prevents leaks in MD code fences while also capturing the closing fence punctuations.

* Update Markdown tokenizer test file

extensions/markdown/test/colorize-results/test_md.json

@@ -1827,7 +1827,7 @@
 	},
 	{
 		"c": "~~~",
-		"t": "markdown.meta.paragraph",
+		"t": "block.definition.fenced_code.markdown.markup.punctuation",
 		"r": {
 			"dark_plus": ".vs-dark .token rgb(212, 212, 212)",
 			"light_plus": ".vs .token rgb(0, 0, 0)",
@@ -1838,7 +1838,7 @@
 	},
 	{
 		"c": "// Markdown extra adds un-indented code blocks too",
-		"t": "markdown.meta.paragraph",
+		"t": "block.fenced_code.markdown.markup",
 		"r": {
 			"dark_plus": ".vs-dark .token rgb(212, 212, 212)",
 			"light_plus": ".vs .token rgb(0, 0, 0)",
@@ -1848,74 +1848,8 @@
 		}
 	},
 	{
-		"c": "if (this",
-		"t": "markdown.meta.paragraph",
-		"r": {
-			"dark_plus": ".vs-dark .token rgb(212, 212, 212)",
-			"light_plus": ".vs .token rgb(0, 0, 0)",
-			"dark_vs": ".vs-dark .token rgb(212, 212, 212)",
-			"light_vs": ".vs .token rgb(0, 0, 0)",
-			"hc_black": ".hc-black .token rgb(255, 255, 255)"
-		}
-	},
-	{
-		"c": "_",
-		"t": "definition.italic.markdown.markup.meta.paragraph.punctuation",
-		"r": {
-			"dark_plus": ".vs-dark.vscode-theme-defaults-themes-dark_plus-json .token.markup.italic rgb(212, 212, 212)",
-			"light_plus": ".vs.vscode-theme-defaults-themes-light_plus-json .token.markup.italic rgb(0, 0, 0)",
-			"dark_vs": ".vs-dark.vscode-theme-defaults-themes-dark_vs-json .token.markup.italic rgb(212, 212, 212)",
-			"light_vs": ".vs.vscode-theme-defaults-themes-light_vs-json .token.markup.italic rgb(0, 0, 0)",
-			"hc_black": ".hc-black.vscode-theme-defaults-themes-hc_black-json .token.markup.italic rgb(255, 255, 255)"
-		}
-	},
-	{
-		"c": "is",
-		"t": "italic.markdown.markup.meta.paragraph",
-		"r": {
-			"dark_plus": ".vs-dark.vscode-theme-defaults-themes-dark_plus-json .token.markup.italic rgb(212, 212, 212)",
-			"light_plus": ".vs.vscode-theme-defaults-themes-light_plus-json .token.markup.italic rgb(0, 0, 0)",
-			"dark_vs": ".vs-dark.vscode-theme-defaults-themes-dark_vs-json .token.markup.italic rgb(212, 212, 212)",
-			"light_vs": ".vs.vscode-theme-defaults-themes-light_vs-json .token.markup.italic rgb(0, 0, 0)",
-			"hc_black": ".hc-black.vscode-theme-defaults-themes-hc_black-json .token.markup.italic rgb(255, 255, 255)"
-		}
-	},
-	{
-		"c": "_",
-		"t": "definition.italic.markdown.markup.meta.paragraph.punctuation",
-		"r": {
-			"dark_plus": ".vs-dark.vscode-theme-defaults-themes-dark_plus-json .token.markup.italic rgb(212, 212, 212)",
-			"light_plus": ".vs.vscode-theme-defaults-themes-light_plus-json .token.markup.italic rgb(0, 0, 0)",
-			"dark_vs": ".vs-dark.vscode-theme-defaults-themes-dark_vs-json .token.markup.italic rgb(212, 212, 212)",
-			"light_vs": ".vs.vscode-theme-defaults-themes-light_vs-json .token.markup.italic rgb(0, 0, 0)",
-			"hc_black": ".hc-black.vscode-theme-defaults-themes-hc_black-json .token.markup.italic rgb(255, 255, 255)"
-		}
-	},
-	{
-		"c": "more_code == true ",
-		"t": "markdown.meta.paragraph",
-		"r": {
-			"dark_plus": ".vs-dark .token rgb(212, 212, 212)",
-			"light_plus": ".vs .token rgb(0, 0, 0)",
-			"dark_vs": ".vs-dark .token rgb(212, 212, 212)",
-			"light_vs": ".vs .token rgb(0, 0, 0)",
-			"hc_black": ".hc-black .token rgb(255, 255, 255)"
-		}
-	},
-	{
-		"c": "&&",
-		"t": "markdown.meta.other.paragraph.valid-ampersand",
-		"r": {
-			"dark_plus": ".vs-dark .token rgb(212, 212, 212)",
-			"light_plus": ".vs .token rgb(0, 0, 0)",
-			"dark_vs": ".vs-dark .token rgb(212, 212, 212)",
-			"light_vs": ".vs .token rgb(0, 0, 0)",
-			"hc_black": ".hc-black .token rgb(255, 255, 255)"
-		}
-	},
-	{
-		"c": " !indented) {",
-		"t": "markdown.meta.paragraph",
+		"c": "if (this_is_more_code == true && !indented) {",
+		"t": "block.fenced_code.markdown.markup",
 		"r": {
 			"dark_plus": ".vs-dark .token rgb(212, 212, 212)",
 			"light_plus": ".vs .token rgb(0, 0, 0)",
@@ -1926,7 +1860,7 @@
 	},
 	{
 		"c": "    // tild wrapped code blocks, also not indented",
-		"t": "markdown.meta.paragraph",
+		"t": "block.fenced_code.markdown.markup",
 		"r": {
 			"dark_plus": ".vs-dark .token rgb(212, 212, 212)",
 			"light_plus": ".vs .token rgb(0, 0, 0)",
@@ -1937,7 +1871,7 @@
 	},
 	{
 		"c": "}",
-		"t": "markdown.meta.paragraph",
+		"t": "block.fenced_code.markdown.markup",
 		"r": {
 			"dark_plus": ".vs-dark .token rgb(212, 212, 212)",
 			"light_plus": ".vs .token rgb(0, 0, 0)",
@@ -1948,7 +1882,7 @@
 	},
 	{
 		"c": "~~~",
-		"t": "markdown.meta.paragraph",
+		"t": "block.definition.fenced_code.markdown.markup.punctuation",
 		"r": {
 			"dark_plus": ".vs-dark .token rgb(212, 212, 212)",
 			"light_plus": ".vs .token rgb(0, 0, 0)",
@@ -2595,4 +2529,4 @@
 			"hc_black": ".hc-black .token rgb(255, 255, 255)"
 		}
 	}
-]
\ No newline at end of file
+]