Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
lwm1986
roslyn
提交
4badaf18
R
roslyn
项目概览
lwm1986
/
roslyn
与 Fork 源项目一致
从无法访问的项目Fork
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
R
roslyn
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
4badaf18
编写于
5月 12, 2015
作者:
E
Evan Hauck
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Allow underscores in VB numeric literals
上级
dd6e0e35
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
107 addition
and
15 deletion
+107
-15
src/Compilers/VisualBasic/Portable/Scanner/Scanner.vb
src/Compilers/VisualBasic/Portable/Scanner/Scanner.vb
+41
-15
src/Compilers/VisualBasic/Test/Syntax/Scanner/ScannerTests.vb
...Compilers/VisualBasic/Test/Syntax/Scanner/ScannerTests.vb
+66
-0
未找到文件。
src/Compilers/VisualBasic/Portable/Scanner/Scanner.vb
浏览文件 @
4badaf18
...
@@ -1554,6 +1554,7 @@ FullWidthRepeat:
...
@@ -1554,6 +1554,7 @@ FullWidthRepeat:
Dim
Here
As
Integer
=
0
Dim
Here
As
Integer
=
0
Dim
IntegerLiteralStart
As
Integer
Dim
IntegerLiteralStart
As
Integer
Dim
UnderscoreInWrongPlace
As
Boolean
Dim
Base
As
LiteralBase
=
LiteralBase
.
Decimal
Dim
Base
As
LiteralBase
=
LiteralBase
.
Decimal
Dim
literalKind
As
NumericLiteralKind
=
NumericLiteralKind
.
Integral
Dim
literalKind
As
NumericLiteralKind
=
NumericLiteralKind
.
Integral
...
@@ -1576,39 +1577,45 @@ FullWidthRepeat:
...
@@ -1576,39 +1577,45 @@ FullWidthRepeat:
IntegerLiteralStart
=
Here
IntegerLiteralStart
=
Here
Base
=
LiteralBase
.
Hexadecimal
Base
=
LiteralBase
.
Hexadecimal
UnderscoreInWrongPlace
=
Peek
(
Here
)
=
"_"
c
While
CanGet
(
Here
)
While
CanGet
(
Here
)
ch
=
Peek
(
Here
)
ch
=
Peek
(
Here
)
If
Not
IsHexDigit
(
ch
)
Then
If
Not
IsHexDigit
(
ch
)
AndAlso
ch
<>
"_"
c
Then
Exit
While
Exit
While
End
If
End
If
Here
+=
1
Here
+=
1
End
While
End
While
UnderscoreInWrongPlace
=
UnderscoreInWrongPlace
Or
(
Peek
(
Here
-
1
)
=
"_"
c
)
Case
"B"
c
,
"b"
c
Case
"B"
c
,
"b"
c
Here
+=
1
Here
+=
1
IntegerLiteralStart
=
Here
IntegerLiteralStart
=
Here
Base
=
LiteralBase
.
Binary
Base
=
LiteralBase
.
Binary
UnderscoreInWrongPlace
=
Peek
(
Here
)
=
"_"
c
While
CanGet
(
Here
)
While
CanGet
(
Here
)
ch
=
Peek
(
Here
)
ch
=
Peek
(
Here
)
If
Not
IsBinaryDigit
(
ch
)
Then
If
Not
IsBinaryDigit
(
ch
)
AndAlso
ch
<>
"_"
c
Then
Exit
While
Exit
While
End
If
End
If
Here
+=
1
Here
+=
1
End
While
End
While
UnderscoreInWrongPlace
=
UnderscoreInWrongPlace
Or
(
Peek
(
Here
-
1
)
=
"_"
c
)
Case
"O"
c
,
"o"
c
Case
"O"
c
,
"o"
c
Here
+=
1
Here
+=
1
IntegerLiteralStart
=
Here
IntegerLiteralStart
=
Here
Base
=
LiteralBase
.
Octal
Base
=
LiteralBase
.
Octal
UnderscoreInWrongPlace
=
Peek
(
Here
)
=
"_"
c
While
CanGet
(
Here
)
While
CanGet
(
Here
)
ch
=
Peek
(
Here
)
ch
=
Peek
(
Here
)
If
Not
IsOctalDigit
(
ch
)
Then
If
Not
IsOctalDigit
(
ch
)
AndAlso
ch
<>
"_"
c
Then
Exit
While
Exit
While
End
If
End
If
Here
+=
1
Here
+=
1
End
While
End
While
UnderscoreInWrongPlace
=
UnderscoreInWrongPlace
Or
(
Peek
(
Here
-
1
)
=
"_"
c
)
Case
Else
Case
Else
If
IsFullWidth
(
ch
)
Then
If
IsFullWidth
(
ch
)
Then
...
@@ -1621,13 +1628,17 @@ FullWidthRepeat:
...
@@ -1621,13 +1628,17 @@ FullWidthRepeat:
Else
Else
' no base specifier - just go through decimal digits.
' no base specifier - just go through decimal digits.
IntegerLiteralStart
=
Here
IntegerLiteralStart
=
Here
UnderscoreInWrongPlace
=
Peek
(
Here
)
=
"_"
c
While
CanGet
(
Here
)
While
CanGet
(
Here
)
ch
=
Peek
(
Here
)
ch
=
Peek
(
Here
)
If
Not
IsDecimalDigit
(
ch
)
Then
If
Not
IsDecimalDigit
(
ch
)
AndAlso
ch
<>
"_"
c
Then
Exit
While
Exit
While
End
If
End
If
Here
+=
1
Here
+=
1
End
While
End
While
If
Here
<>
IntegerLiteralStart
Then
UnderscoreInWrongPlace
=
UnderscoreInWrongPlace
Or
(
Peek
(
Here
-
1
)
=
"_"
c
)
End
If
End
If
End
If
' we may have a dot, and then it is a float, but if this is an integral, then we have seen it all.
' we may have a dot, and then it is a float, but if this is an integral, then we have seen it all.
...
@@ -1647,11 +1658,12 @@ FullWidthRepeat:
...
@@ -1647,11 +1658,12 @@ FullWidthRepeat:
' all following decimal digits belong to the literal (fractional part)
' all following decimal digits belong to the literal (fractional part)
While
CanGet
(
Here
)
While
CanGet
(
Here
)
ch
=
Peek
(
Here
)
ch
=
Peek
(
Here
)
If
Not
IsDecimalDigit
(
ch
)
Then
If
Not
IsDecimalDigit
(
ch
)
AndAlso
ch
<>
"_"
c
Then
Exit
While
Exit
While
End
If
End
If
Here
+=
1
Here
+=
1
End
While
End
While
UnderscoreInWrongPlace
=
UnderscoreInWrongPlace
Or
(
Peek
(
Here
-
1
)
=
"_"
c
)
literalKind
=
NumericLiteralKind
.
Float
literalKind
=
NumericLiteralKind
.
Float
End
If
End
If
...
@@ -1672,11 +1684,12 @@ FullWidthRepeat:
...
@@ -1672,11 +1684,12 @@ FullWidthRepeat:
Here
+=
1
Here
+=
1
While
CanGet
(
Here
)
While
CanGet
(
Here
)
ch
=
Peek
(
Here
)
ch
=
Peek
(
Here
)
If
Not
IsDecimalDigit
(
ch
)
Then
If
Not
IsDecimalDigit
(
ch
)
AndAlso
ch
<>
"_"
c
Then
Exit
While
Exit
While
End
If
End
If
Here
+=
1
Here
+=
1
End
While
End
While
UnderscoreInWrongPlace
=
UnderscoreInWrongPlace
Or
(
Peek
(
Here
-
1
)
=
"_"
c
)
Else
Else
Return
MakeBadToken
(
precedingTrivia
,
Here
,
ERRID
.
ERR_InvalidLiteralExponent
)
Return
MakeBadToken
(
precedingTrivia
,
Here
,
ERRID
.
ERR_InvalidLiteralExponent
)
End
If
End
If
...
@@ -1819,12 +1832,16 @@ FullWidthRepeat2:
...
@@ -1819,12 +1832,16 @@ FullWidthRepeat2:
If
IntegerLiteralStart
=
IntegerLiteralEnd
Then
If
IntegerLiteralStart
=
IntegerLiteralEnd
Then
Return
MakeBadToken
(
precedingTrivia
,
Here
,
ERRID
.
ERR_Syntax
)
Return
MakeBadToken
(
precedingTrivia
,
Here
,
ERRID
.
ERR_Syntax
)
Else
Else
IntegralValue
=
IntegralLiteralCharacterValue
(
Peek
(
IntegerLiteralStart
))
IntegralValue
=
0
If
Base
=
LiteralBase
.
Decimal
Then
If
Base
=
LiteralBase
.
Decimal
Then
' Init For loop
' Init For loop
For
LiteralCharacter
As
Integer
=
IntegerLiteralStart
+
1
To
IntegerLiteralEnd
-
1
For
LiteralCharacter
As
Integer
=
IntegerLiteralStart
To
IntegerLiteralEnd
-
1
Dim
NextCharacterValue
As
UInteger
=
IntegralLiteralCharacterValue
(
Peek
(
LiteralCharacter
))
Dim
LiteralCharacterValue
As
Char
=
Peek
(
LiteralCharacter
)
If
LiteralCharacterValue
=
"_"
c
Then
Continue
For
End
If
Dim
NextCharacterValue
As
UInteger
=
IntegralLiteralCharacterValue
(
LiteralCharacterValue
)
If
IntegralValue
<
1844674407370955161UL
OrElse
If
IntegralValue
<
1844674407370955161UL
OrElse
(
IntegralValue
=
1844674407370955161UL
AndAlso
NextCharacterValue
<=
5UI
)
Then
(
IntegralValue
=
1844674407370955161UL
AndAlso
NextCharacterValue
<=
5UI
)
Then
...
@@ -1844,12 +1861,17 @@ FullWidthRepeat2:
...
@@ -1844,12 +1861,17 @@ FullWidthRepeat2:
Dim
OverflowMask
As
UInt64
=
If
(
Base
=
LiteralBase
.
Hexadecimal
,
&
HF000000000000000UL
,
If
(
Base
=
LiteralBase
.
Octal
,
&
HE000000000000000UL
,
&
H8000000000000000UL
))
Dim
OverflowMask
As
UInt64
=
If
(
Base
=
LiteralBase
.
Hexadecimal
,
&
HF000000000000000UL
,
If
(
Base
=
LiteralBase
.
Octal
,
&
HE000000000000000UL
,
&
H8000000000000000UL
))
' Init For loop
' Init For loop
For
LiteralCharacter
As
Integer
=
IntegerLiteralStart
+
1
To
IntegerLiteralEnd
-
1
For
LiteralCharacter
As
Integer
=
IntegerLiteralStart
To
IntegerLiteralEnd
-
1
Dim
LiteralCharacterValue
As
Char
=
Peek
(
LiteralCharacter
)
If
LiteralCharacterValue
=
"_"
c
Then
Continue
For
End
If
If
(
IntegralValue
And
OverflowMask
)
<>
0
Then
If
(
IntegralValue
And
OverflowMask
)
<>
0
Then
Overflows
=
True
Overflows
=
True
End
If
End
If
IntegralValue
=
(
IntegralValue
<<
Shift
)
+
IntegralLiteralCharacterValue
(
Peek
(
LiteralCharacter
)
)
IntegralValue
=
(
IntegralValue
<<
Shift
)
+
IntegralLiteralCharacterValue
(
LiteralCharacterValue
)
Next
Next
End
If
End
If
...
@@ -1892,7 +1914,9 @@ FullWidthRepeat2:
...
@@ -1892,7 +1914,9 @@ FullWidthRepeat2:
Dim
scratch
=
GetScratch
()
Dim
scratch
=
GetScratch
()
For
i
=
0
To
literalWithoutTypeChar
-
1
For
i
=
0
To
literalWithoutTypeChar
-
1
Dim
curCh
=
Peek
(
i
)
Dim
curCh
=
Peek
(
i
)
scratch
.
Append
(
If
(
IsFullWidth
(
curCh
),
MakeHalfWidth
(
curCh
),
curCh
))
If
curCh
<>
"_"
c
Then
scratch
.
Append
(
If
(
IsFullWidth
(
curCh
),
MakeHalfWidth
(
curCh
),
curCh
))
End
If
Next
Next
Dim
LiteralSpelling
=
GetScratchTextInterned
(
scratch
)
Dim
LiteralSpelling
=
GetScratchTextInterned
(
scratch
)
...
@@ -1920,17 +1944,19 @@ FullWidthRepeat2:
...
@@ -1920,17 +1944,19 @@ FullWidthRepeat2:
Dim
result
As
SyntaxToken
Dim
result
As
SyntaxToken
Select
Case
literalKind
Select
Case
literalKind
Case
NumericLiteralKind
.
Integral
Case
NumericLiteralKind
.
Integral
result
=
MakeIntegerLiteralToken
(
precedingTrivia
,
Base
,
TypeCharacter
,
If
(
Overflows
,
0UL
,
IntegralValue
),
Here
)
result
=
MakeIntegerLiteralToken
(
precedingTrivia
,
Base
,
TypeCharacter
,
If
(
Overflows
Or
UnderscoreInWrongPlace
,
0UL
,
IntegralValue
),
Here
)
Case
NumericLiteralKind
.
Float
Case
NumericLiteralKind
.
Float
result
=
MakeFloatingLiteralToken
(
precedingTrivia
,
TypeCharacter
,
If
(
Overflows
,
0.0
F
,
FloatingValue
),
Here
)
result
=
MakeFloatingLiteralToken
(
precedingTrivia
,
TypeCharacter
,
If
(
Overflows
Or
UnderscoreInWrongPlace
,
0.0
F
,
FloatingValue
),
Here
)
Case
NumericLiteralKind
.
Decimal
Case
NumericLiteralKind
.
Decimal
result
=
MakeDecimalLiteralToken
(
precedingTrivia
,
TypeCharacter
,
If
(
Overflows
,
0D
,
DecimalValue
),
Here
)
result
=
MakeDecimalLiteralToken
(
precedingTrivia
,
TypeCharacter
,
If
(
Overflows
Or
UnderscoreInWrongPlace
,
0D
,
DecimalValue
),
Here
)
Case
Else
Case
Else
Throw
ExceptionUtilities
.
UnexpectedValue
(
literalKind
)
Throw
ExceptionUtilities
.
UnexpectedValue
(
literalKind
)
End
Select
End
Select
If
Overflows
Then
If
Overflows
Then
result
=
DirectCast
(
result
.
AddError
(
ErrorFactory
.
ErrorInfo
(
ERRID
.
ERR_Overflow
)),
SyntaxToken
)
result
=
DirectCast
(
result
.
AddError
(
ErrorFactory
.
ErrorInfo
(
ERRID
.
ERR_Overflow
)),
SyntaxToken
)
ElseIf
UnderscoreInWrongPlace
Then
result
=
DirectCast
(
result
.
AddError
(
ErrorFactory
.
ErrorInfo
(
ERRID
.
ERR_Syntax
)),
SyntaxToken
)
End
If
End
If
Return
result
Return
result
...
...
src/Compilers/VisualBasic/Test/Syntax/Scanner/ScannerTests.vb
浏览文件 @
4badaf18
...
@@ -694,6 +694,13 @@ End If]]>.Value,
...
@@ -694,6 +694,13 @@ End If]]>.Value,
Assert.Equal(42, tk.Value)
Assert.Equal(42, tk.Value)
Assert.Equal("
42
", tk.ToFullString())
Assert.Equal("
42
", tk.ToFullString())
Str = "
4
_2
"
tk = ScanOnce(Str)
Assert.Equal(SyntaxKind.IntegerLiteralToken, tk.Kind)
Assert.Equal(LiteralBase.Decimal, tk.GetBase())
Assert.Equal(42, tk.Value)
Assert.Equal("
4
_2
", tk.ToFullString())
Str = "
&
H42L
"
Str = "
&
H42L
"
tk = ScanOnce(Str)
tk = ScanOnce(Str)
Assert.Equal(SyntaxKind.IntegerLiteralToken, tk.Kind)
Assert.Equal(SyntaxKind.IntegerLiteralToken, tk.Kind)
...
@@ -701,6 +708,13 @@ End If]]>.Value,
...
@@ -701,6 +708,13 @@ End If]]>.Value,
Assert.Equal(&H42L, tk.Value)
Assert.Equal(&H42L, tk.Value)
Assert.Equal("
&
H42L
", tk.ToFullString())
Assert.Equal("
&
H42L
", tk.ToFullString())
Str = "
&
H4_2L
"
tk = ScanOnce(Str)
Assert.Equal(SyntaxKind.IntegerLiteralToken, tk.Kind)
Assert.Equal(LiteralBase.Hexadecimal, tk.GetBase())
Assert.Equal(&H42L, tk.Value)
Assert.Equal("
&
H4_2L
", tk.ToFullString())
Str = "
&
H42L
&
H42
&
"
Str = "
&
H42L
&
H42
&
"
Dim tks = ScanAllCheckDw(Str)
Dim tks = ScanAllCheckDw(Str)
Assert.Equal(SyntaxKind.IntegerLiteralToken, tks(0).Kind)
Assert.Equal(SyntaxKind.IntegerLiteralToken, tks(0).Kind)
...
@@ -714,6 +728,13 @@ End If]]>.Value,
...
@@ -714,6 +728,13 @@ End If]]>.Value,
Assert.Equal(LiteralBase.Binary, tk.GetBase())
Assert.Equal(LiteralBase.Binary, tk.GetBase())
Assert.Equal(&HAL, tk.Value)
Assert.Equal(&HAL, tk.Value)
Assert.Equal("
&
B1010L
", tk.ToFullString())
Assert.Equal("
&
B1010L
", tk.ToFullString())
Str = "
&
B1_0_1_0L
"
tk = ScanOnce(Str)
Assert.Equal(SyntaxKind.IntegerLiteralToken, tk.Kind)
Assert.Equal(LiteralBase.Binary, tk.GetBase())
Assert.Equal(&HAL, tk.Value)
Assert.Equal("
&
B1_0_1_0L
", tk.ToFullString())
End Sub
End Sub
<Fact>
<Fact>
...
@@ -730,6 +751,13 @@ End If]]>.Value,
...
@@ -730,6 +751,13 @@ End If]]>.Value,
Assert.IsType(Of Double)(tk.Value)
Assert.IsType(Of Double)(tk.Value)
Assert.Equal("
0.42
", tk.ToFullString())
Assert.Equal("
0.42
", tk.ToFullString())
Str = "
0
_0
.
4
_2
"
tk = ScanOnce(Str)
Assert.Equal(SyntaxKind.FloatingLiteralToken, tk.Kind)
Assert.Equal(0.42, tk.Value)
Assert.IsType(Of Double)(tk.Value)
Assert.Equal("
0
_0
.
4
_2
", tk.ToFullString())
Str = "
0.42
#
"
Str = "
0.42
#
"
tk = ScanOnce(Str)
tk = ScanOnce(Str)
Assert.Equal(SyntaxKind.FloatingLiteralToken, tk.Kind)
Assert.Equal(SyntaxKind.FloatingLiteralToken, tk.Kind)
...
@@ -858,6 +886,44 @@ End If]]>.Value,
...
@@ -858,6 +886,44 @@ End If]]>.Value,
Assert
.
Equal
(
0.0
F
,
tk
.
Value
)
Assert
.
Equal
(
0.0
F
,
tk
.
Value
)
End
Sub
End
Sub
<
Fact
>
Public
Sub
Scanner_UnderscoreWrongLocation
()
Dim
Str
=
"_1"
Dim
tk
=
ScanOnce
(
Str
)
Assert
.
Equal
(
SyntaxKind
.
IdentifierToken
,
tk
.
Kind
)
Assert
.
Equal
(
0
,
tk
.
GetSyntaxErrorsNoTree
().
Count
())
Str
=
"1_"
tk
=
ScanOnce
(
Str
)
Assert
.
Equal
(
SyntaxKind
.
IntegerLiteralToken
,
tk
.
Kind
)
Assert
.
Equal
(
30035
,
tk
.
GetSyntaxErrorsNoTree
()(
0
).
Code
)
Assert
.
Equal
(
0
,
CInt
(
tk
.
Value
))
Str
=
"&H_1"
tk
=
ScanOnce
(
Str
)
Assert
.
Equal
(
SyntaxKind
.
IntegerLiteralToken
,
tk
.
Kind
)
Assert
.
Equal
(
30035
,
tk
.
GetSyntaxErrorsNoTree
()(
0
).
Code
)
Assert
.
Equal
(
0
,
CInt
(
tk
.
Value
))
Str
=
"&H1_"
tk
=
ScanOnce
(
Str
)
Assert
.
Equal
(
SyntaxKind
.
IntegerLiteralToken
,
tk
.
Kind
)
Assert
.
Equal
(
30035
,
tk
.
GetSyntaxErrorsNoTree
()(
0
).
Code
)
Assert
.
Equal
(
0
,
CInt
(
tk
.
Value
))
Str
=
"1_.1"
tk
=
ScanOnce
(
Str
)
Assert
.
Equal
(
SyntaxKind
.
FloatingLiteralToken
,
tk
.
Kind
)
Assert
.
Equal
(
30035
,
tk
.
GetSyntaxErrorsNoTree
()(
0
).
Code
)
Assert
.
Equal
(
0
,
CInt
(
tk
.
Value
))
Str
=
"1.1_"
tk
=
ScanOnce
(
Str
)
Assert
.
Equal
(
SyntaxKind
.
FloatingLiteralToken
,
tk
.
Kind
)
Assert
.
Equal
(
30035
,
tk
.
GetSyntaxErrorsNoTree
()(
0
).
Code
)
Assert
.
Equal
(
0
,
CInt
(
tk
.
Value
))
End
Sub
<
Fact
>
<
Fact
>
Public
Sub
Scanner_DateLiteralToken
()
Public
Sub
Scanner_DateLiteralToken
()
Dim
Str
=
"#10/10/2010#"
Dim
Str
=
"#10/10/2010#"
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录