Enable command line compiler to produce XML doc comment file and embed it as a manifest resource at the same time. Fixes #444. (changeset 1396823)

Bugfix# 1082026 This is a formatting case with Initializer Syntax that went unhandled. Testcase added previously ascertained the regression. Now modifying the testcase to handle the regression.

Also a CR comment (changeset 1396819)

This is necessary to correctly reject invalid assembly references and to match the native compiler. (changeset 1396814)

Replace FixAllContext.GetDiagnosticsAsync(Project) API, which currently returns only non-source diagnostics for the given project with 2 APIs:
(a) GetProjectDiagnosticsAsync(Project): Returns only non-source diagnostics for the given project.
(b) GetAllDiagnosticsAsync(Project): Returns both non-source diagnostics for the given project and document diagnostics for all documents in the given project.

This also fixes this API to stop returning document diagnostics with removed locations. (changeset 1396750)

This changes our projects to consume xUnit through NuGet instead of our checked in binaries in Closed\Tools\xUnit.  This is another step in removing project scrubber.  (changeset 1396548)

Rollback the previous rollback of the native client fixes and skip running in CI for now, due to a tricky chicken/egg problem.
***NO_CI***
 (changeset 1396330)

Moved SyntaxGenerator and editors into Editting namespace.  CodeGeneration namespace is now all internal, and will eventually be removed. (changeset 1396303)

Test helpers:  Enable lookup of names with dots (explicit interface implementation) and generate more useful messages when member lookup fails. (changeset 1396244)

This change adds an analyzer that raises errors when modifying the public APIs.

The idea here is that each project contains a file, PublicAPI.txt, that lists the full names of the types and members that make up the public API of that assembly. When the diagnostic encounters a public symbol, it checks if the name is in the list. If not, it creates a diagnostic. While this does not prevent changes to the public API, it will make them very obvious in code reviews and source control history as the only way to get rid of the error is to update PublicAPI.txt.

At the end of compilation, we also check for symbols that were declared in PublicAPI.txt but were not found in the project, or were not public, and report errors about those as well.

The bulk of the change is actually in the fixer, as I choose to implement a custom FixAllProvider in order to test out that feature. (changeset 1396119)

This change reduces confusion on the intended use of this workspace which is primarily for testing or doing analysis outside of a host environment without requirement to round-trip changes to disk. (changeset 1396090)

Implement IEquatable<ISuggestedActions> for all Roslyn ISuggestedActions. The implementation treats any SuggestedActions that have the same provider as well as CodeActions with equal non-null 'Id's as equivalent.

Also override 'Id 'for
CodeActions of the following code fixes to reduce the occurence of duplicate fixes in the lightbulb menu - Add Imports, Generate Type and  Generate Property / Field.

Note: Addressing the above three code fixes takes care of the most common cases where users currently see duplicate fixes in the light bulb menu. We can apply similar de-duping for several other code fixes but we will address them separately later.

Also fixing a couple of other problems discovered while testing this fix -

1. The 'implement interface' code fix provider was using same Id for the simple 'implement interface' code action as well as the 'implement dispose pattern' code action causing the latter to be filtered out. This was also causing problems during 'fix all occurrences'. Fixed this by including the type name of the code action in its Id.

2. When invoking SuggestedActionSet constructor, we were passing in linq queries that would construct new SuggestedActions in a couple of places. The platform implementation for SuggestedActionSet would store this query directly in a field and end up returning a new set of SuggestedActions each time the field was accssed. This ended up causing problems when the platform code tried to de-dupe the fixes (because different fixes would be returned each time). I have changed Roslyn-side code to pass in ImmutableArrays instead - but platform (Oleg) is also fixing the platfrom code to be more robust for this issue. (changeset 1396084)

Currently, analyzers can access the additional files passed in to the compiler through the AnalyzerOptions.AdditionalStreams property, which returns an ImmutableArray<AdditionalStream>. AdditionalStream just wraps a Stream, and a path to the file.

Exposing these files as Streams is problematic for the diagnostic author. We always meant for these additional files to be additional *text* files, and not binary files. As such, Stream is too general a type to represent them. This makes it harder for diagnostic authors to consume additional files. They need to handle opening the stream, supplying the correct encoding, (possibly) splitting up the lines, and handling any errors that might arise while doing so.

This breaking change replaces AdditionalStream with AdditionalText in the portable part of the compiler layer. The derived types AdditionalFileStream (compiler desktop) and AdditionalDocumentStream (workspace layer) have become AdditionalTextFile and AdditionalTextDocument, respectively. The compiler layer now handles reading the additional file contents into SourceTexts, as well as any errors that result.
 (changeset 1396073)

Fix Bug 1102783: Opening quotation mark in VB interpolated strings should not be restricted to ASCII double quote only (changeset 1396066)

This reduces confusion for workspace authors on the intended use of these functions, and eliminates naming conflicts with host workspace API's. (changeset 1396064)

***NO_CI***
 (changeset 1395705)

Prevent memory starvation or a compiler crash due to an OutOfMemoryException caused by constant folding for deeply nested constant string concatenation.

To prevent memory starvation, especially inside a 64  bit process, we keep track of lengths of all (to a certain degree) constant string values stored in the bound expression tree that we are building and report an error as soon as the combined length of all of them is about to be greater than Int32.MaxValue. The length of the resulting constant value might still be shorter than Int32.MaxValue.

***NO_CI***
 (changeset 1395692)

 Change Description:

     This change moves Roslyn off of our custom version of xUnit to an official release (xUnit 1.9.2):

     - Deletes all of the custom xUnit binaries from Closed\Tools\xUnit
     - Put a subset of the official 1.9.2 binaries with no name mangling into Closed\Tools\xUnit.
     - Rebuilt our custom xUnit GUI (named xunit.gui.custom) against 1.9.2 and put it in Closed\Tools\xUnit
     - Updated the targets file, RunTests.ps1 and Test.exe to refer to the updated binary names.
     - Changed the Roslyn unit tests to work on standard xUnit.

     The biggest part of the effort was the last part, getting the tests to run on standard xUnit.  The main issue here was many assumed they would be executed on a single STA thread and as a result cached many thread affinitized values into static fields.  Fixing this involved mostly finding the values and annotating them with [ThreadStatic].

     This does mean that MEF composition will happen several times during an xUnit run where before it happened only once.  That is unfortunate but doesn't seem to regress the suites in a significant way on my machine.  Using a multithreaded MEF container isn't an option here because the services end up caching thread affinitized WPF values.

     ForegroudThreadAffinitizedObject was the other largest change.  It's use of a static ctor to establish the foreground thread is dangerous and should be correctly addressed in a future change.  For now I added a method for tests to reset the identified foreground thread.

     More changes will be coming here for our use of xUnit.  But they should be smaller and more targetted (deleting the extra sources, a better build story for the GUI, etc ...).

     Note: Why 1.9.2 instead of 2.0?  We have several uses of custom xUnit extensibility that is not easily portable to 2.0.  Best path forward is to delete that extensibility since it isn't really adding much and move us directly onto 2.0 in a later check in.

 (changeset 1395675)

The failing test SuppressSyntaxDiagnosticsOnRaiseEventAccessorBasic for bug 1103442 blows up intermittently on build queues with an exception thrown for trying to enqueue an event to the CompilationEventQueue after the queue has been completed. I was able to get a consistent repro by running the test in a Parallel.For for about 10k iterations.
The failing call stack shows that the faulty symbol declared event is being generated for a SourceEventSymbol. Currently we generate the symbol declared event for event symbols within ExplicitInterfaceImplementations property. However, this property is accessible from the symbol constructor, as is the case for this failure.
Generating the SymbolDeclaredEvent from a symbol constructor is inherently racy, as multiple threads might be trying to compute members of it's containing symbol and hence might instantiate the member symbols, even though just one thread will succeed in storing the computed members onto the field.
This change moves the SymbolDeclaredEvent for events and properties out of  ExplicitInterfaceImplementations and into GenerateDeclarationErrors, guarded by a special state flag. The modified failing test passes consistently on my machine after this fix. (changeset 1395533)

Remove unnecessary data and implementation details from XML produced by PdbToXml. Update PDB tests accordingly.
The removed data include sizes and versions of CDI blobs, counts of elements (redundant since the elements themselves are included in the XML), hidden sequence point line/columns, etc.

This change only affects tests. (changeset 1395517)

Added BoundPseudoVariable type with GetAddress and GetValue methods called by the emitter (changeset 1395465)

ByteSequenceComparer has been updated as follows:
- The implementation of IEqualityComparer<IEnumerable<byte>> has been removed
- The implementations of IEqualityComparer<{byte[], ImmutableArray<byte>}> are now explicit
- The various overloads of EqualsValue and GetHashCodeValue have been renamed to Equals and GetHasCode, respectively.
***NO_CI***
 (changeset 1395450)

Public types CommonMemberResolutionKind, CommonMemberResolutionResult, CommonOverloadResolutionResult are not used nor returned by any Roslyn public API. Remove them.
 (changeset 1395377)

1) Assigns each member a "member ordinal" - the index in the members array of the containing type. The ordinal is propagated thru lowering rewriters which uses it to make generated names unique.
     2) We no longer calculate "overload ordinal" for names of state machine types, we use the member ordinal of the async/iterator method instead.
     3) Removes GenerateTempNumber from TypeCompilationState. Combination of method ordinal and other indices is used to make generated names unique:
     - Display classes use ordinal of the method containing the lambda and closure scope ordinal (unique within a method). Static display class is shared across all non-generic methods of a containing class and thus doesn't need a unique number in name.
     - Lambda methods include containing method ordinal (unless emitted to a display class whose name already includes it) and lambda ordinal (unique within a method).
     - The same for fields caching lambdas.
     - Delegate relaxation stubs and their closures have a separate numbering and are distinguished from regular lambdas/closures, since they are immutable and EnC doesn't need to map them across generations.
     - Late-bound AddressOf operator oddly creates real closure and is thus not considered a delegate relaxation for the purposes of synthesized naming.
     - Properties generated to override inherited WithEvent properties are assigned ids in the source order of Handles clauses.

     In order to guarantee uniqueness of the names added during EnC include generation ordinal in synthesized names  wherever we use method ordinal. Generation ordinal 0 (regular build) is omitted - the additional number is only added to names of synthesized members created during EnC compilation. (changeset 1395267)

This change removes all uses of Assert.DoesNotThrow. This is no longer a part of the xUnit public API and continuing to use this in our code base is preventing us from moving to a standard version of xUnit.

Here is the issue which tracks removing this API and the justification for doing so.

https://github.com/xunit/xunit/issues/188

In summary: it's a redundant call. If the code throws the test fails with or without this method.  (changeset 1395207)

The change basically moves the stack guard from ParseExpression down to ParseSubExpression
Fixes #446
 (changeset 1395172)

Changeset shamelessly stolen from angocke (changeset 1395147)

Adding an analyzer to the CodeAnalysis analyzers package that issues an error if someone tries to implement an interface marked with an attribute.

Adding a internal type (System.Runtime.CompilerServices.InternalImplementationOnlyAttribute) to CodeAnalysis.dll and applying it to ISymbol so that we can prevent folks from implementing ISymbol (in order for us to be able to make changes to it in the future). Chose the System namespace because the LDM wants this to be a compiler feature in the future. The analyzer just checks by name and so the type can be internal and if a different dll needs it, it can define the attribute by itself. (changeset 1394725)

Use a compatible MethodInfo when generating MethodCallExpressions for virtual methods on value types.

When generating a call to a virtual method on a value type, the least overridden method virtual method is only used in special circumstances. These circumstances are exactly the checks performed by MayUseCallForStructMethod during IL emission when deciding whether or not to call a method with a value type receiver directly). This change applies the same rules when generating MethodCallExpressions for value type receivers in expression lambdas, bringing the generated code back in line with Dev12.
***NO_CI***
 (changeset 1394691)

Add a CodeAnalysis diagnostic for detecting memory leaks in implementation of analyzers (DoNotStorePerCompilationDataOntoFields): Instance of a diagnostic analyzer might outlive the lifetime of compilation (when executed in the IDE). Hence, storing per-compilation data, such as symbols, into the fields of a diagnostic analyzer might cause stale compilations to stay alive and cause memory leaks. Instead, one should store this data on a separate type instantiatied in a compilation start action, registered using AnaylsisContext.RegisterCompilationStartActionName API. An instance of this type will be created per-compilation and it won't outlive compilation's lifetime, avoiding memory leaks.

Analyzer implementation: Analyzer tracks all field declarations within analyzer types (marked with DiagnosticAnalyzerAttribute) and flags if it's variable type declaration has any type syntax descendant that binds to a type implementing ISymbol or has Compilation in its type chain.

I have also added a few positive and negative tests for the diagnostic. (changeset 1394660)