Background:

     As discussed in the dev team all hands all code under the GitHub dotnet foundation is using a single process for contribution, API review, infrastructure and coding style. The idea is to present a unified view to our customer and give them a single story for contributing to any project under the dotnet foundation.

     https://github.com/dotnet/corefx/wiki/Contributing#c-coding-style

     The coding style transition is automated using a Roslyn based rewrite tool:

     https://github.com/dotnet/codeformatter

     This will be applied in stages across our developer tree. Right now the focus is on the Open directory as this is what is being presented on github. Code owners will be contacted before the transition happens.

     Note: this is a soft style requirement. There are no build errors that come from this change.
 (changeset 1408227)

     recent perf regression shows that we are allocating too much memory on one particular spot in the tree serialization.

     this change is to remove that bottleneck. basically, rather than creating temporary array just to pass it in to the serializer, it will iterate through elements itself and serialize each element. (changeset 1408094)

This change reduces confusion on the intended use of this workspace which is primarily for testing or doing analysis outside of a host environment without requirement to round-trip changes to disk. (changeset 1396090)

Use nameof for ArgumentNullExceptions in this file while I'm here. (changeset 1379338)

The current cost-based cache has no domain knowledge of whether items will be needed again or are safe to collect.  It is therefore usually over-sized. This change implements an explicit cache that must be enabled by consumers. The cache will hold trees and compilations as needed and release them when clients are done with them.

Currently, we enable explicit caching of the active document. Rename tracking and the solution crawler also request caching for each project they process. (changeset 1372494)

Moved default host services logic from MSBuildWorkspace to new type DesktopMefHostServices type.

Changed CommandLineProject to make workspace parameter optional. You only need to specify workspace if want to override behavior of some of the services (beyond default).

Added WorkspaceServices.GetRequiredService and LanguageServices.GetRequiredService API that throws InvalidOperationException if service is not available.

Changed Workspace.AddDocument and Workspace.AddAdditionalDocument to take DocumentInfo instead of multiple parameters (for future proofing signature.)

Added ApplyChangesKind enum values for analyzers and additional documents

Changed Workspace TryApplyChanges to check for all unsupported changes before starting to apply.
Changed separate *apply* methods to default to calling equivalent OnXXX method instead of throwing.

Changed CustomWorkspace.CanApplyChanges to default to true for all kinds.
Added error messages for each ApplyChangesKind

Added missing WithXXX methods from DocumentInfo (changeset 1354869)

Move a few types back to the portable layer to allow a CustomWorkspace to be produced without needing the desktop layer.

As part of this, I split apart ITemporaryStorageService's ITemporaryStorage into two separate types, so it's clear whether you're storing text or storing binary streams. The merged implementation allowed some leakage of implementation details. (changeset 1353513)

This change splits the Workspaces layer into two parts, mirroring the Portable/Desktop split that the compilers did. The core parts of the Workspaces layer (managing documents and projects, formatting, some refactoring, code fixes) is kept in the portable subset, with a few non-portable pieces remaining, notably MSBuild support.

This change has a major impact on how MEF now works in Roslyn. Traditional MEF (“MEFv1”) is not portable, and so we must move the Workspaces layer over to using the Microsoft.Composition NuGet package (“MEFv2”) which is. The APIs are distinct in that each has its own namespace, but the concepts are more or less identical. It requires some care though: the workspaces layer is simple in that it only references MEFv2, but higher layers have to reference both versions to use metadata attributes. Exports using metadata attributes from the editor (say, ContentTypeAttribute) must use MEFv1 attributes to export, whereas exports for the workspaces layer must use MEFv2 attributes. The rule is subtle and yet simple, and so a diagnostic is provided which catches any offenses to prevent confusion.

This also has some impact in how we create MEF hosts: if you wish to host just the base workspaces layer, we can use MEFv2 to compose everything. The HostServices implementation (MefV1HostServices) that consumes a MEFv1 ExportProvider. If you’re in Visual Studio, you can use this implementation to get the full set of host services provided in Visual Studio.

Otherwise, most of the changes in here are minor: we react to some APIs that have been moved/renamed in the portable subset we are targeting, and also align our various exception helper utilities with the compiler’s precedent. (changeset 1349276)

This shelveset implements following changes to metadata reference compiler API in order to remove duplication in the public surface and prevent users from unexpected metadata lifetime issues:

     1) MetadataImageReference and MetadataFileReference overlap
     Currently MetadataImageReference can be constructed from a Stream. MetadataImageReference supports metadata prefetch (reading the blob into memory and close the underlying stream) as well deferred reading (no content is read until the reference is consumed during compilation). MetadataFileReference only supports deferred reading.

     Lifetime of MetadataFileReference is non-deterministic, the underlying file is locked until no references exist and we GC the metadata.  On the other hand, it is possible to construct MetadataImageReference in such a way that allows controlling the underlying resources deterministically.

     Remove MetadataFileReference, use MetadataImageReference instead.

     2) Lifetime management
     AssemblyMetadata and ModuleMetadata objects hold on resources and implement IDisposable. When using MetadataFileReference constructors and some MetadataImageReference constructors to create references the underlying metadata objects are created implicitly and the user doesn’t have a way to explicitly dispose them.

     Make MetadataImageReference constructors internal and instead add factory method GetReference on AssemblyMetadata/ModuleMetadata. The usage pattern is:

     using (var metadata = AssemblyMetadata.CreateFromXxx(…))
     {
         var compilation = CSharpCompilation.Create(syntaxTrees, new[] { metadata.GetReference() });
         …
     }

     In addition the shelveset makes MetadataImageReference internal and adds the following convenience APIs, that are not the most efficient but are very convenient, easy to discover and safe to use for customers that don’t wanna explicitly manage the lifetime of metadata objects. (changeset 1345987)

Fix bug 925931: Report error BC36394 (The 'Main' method cannot be marked 'Async') at declaration stage. Many tests were updated that incorrectly targeted a compilation to executable, but did not provide Main method.
 (changeset 1339789)

Replace IDiagnosticAnalyzer with DiagnosticAnalyzer, and move to an action-based diagnostic analysis model. (changeset 1337816)

Fix for bug 1022125: diagnostic analyzer service shouldn't load diagnostic analyzer before it is actually used

This change refactors the IDE diagnostic service to track per-language workspace analyzers (VSIX based analyzers). Each analyzer assembly reference created for every analyzer asset is now queried for analyzers in context of a language, hence avoiding instantiating analyzers targeting a different language. Additionally, now that we use the MetadataReader instead of Reflection based Assembly load to find diagnostic analyzer types within an analyzer assembly, we will delay loading of language specific analyzer assemblies until it is queried for analyzers for that language. (changeset 1332021)

This property controls what happens when referenced projects are not recognized as supported project types during solution or project open. If set to false, an exception is thrown the first time an unrecognized project is encountered.  If set to false (the default) it raises an WorkspaceFailed event and continues, omitting the project from the resulting solution and leaving dangling project references.  Of course, if the project type is unrecognized but its metadata can be identified, the project reference is turned into a metadata references (as per existing behavior).

Added tests for all cases of unrecognized projects; bad path, non-existing project file, unrecognized file extensions, unrecognized project type and unrecognized language.

Updated HostWorkspaceServices to allow for searching for language services without forcing the language to be loaded.  Added ProjectFileExtensionAttribute and ProjectTypeGuidAttribute so ProjectFileLoader's can declare via MEF metadata information used to find them.

Removed Project GUID from ProjectFileReference.  Project GUID's cannot be trusted. :-)
 (changeset 1279722)

Move serialization from CompilationOptions and ParseOptions to dedicated SerializableCompilationOptions and SerializableParseOptions wrappers. (changeset 1274332)

Stores Encoding on SourceText so that we can always calculate checksum of the underlying binary data, even if the SourceText wasn’t created from a Stream.

Adds WithRootAndOptions and WithFilePath methods to SyntaxTree and implements them for all subclasses. I decided to merge WithRoot and WithOptions to a single method WithRootAndOptions since setting one without the other doesn't make much sense (the options should correspond to the root node).

Propagates the Encoding throughout workspaces and services.
Refactors recoverable syntax trees so that they can implement With* methods.

Allow encoding on SourceText to be unspecified (null) and report an emit diagnostic when attempting to emit debug information for a syntax tree that is backed by encoding-less source text. If user creates a syntax tree from a string they previously read from a file and then try to emit debuggable compilation with such tree the original encoding of the file might not match the default UTF8 (they might different by presence of BOM, which is quite subtle). To avoid confusion we don't set UTF8 encoding by default. Instead we report the diagnostic and the user needs to explicitly provide an encoding to the syntax tree factory method.

Avoid leaking DecoderFallbackException out of EncodedStringText.Create. Throw InvalidDataException instead to reduce the amount of exceptions that the caller needs to deal with.

Also moves HasSameText extension method from Services to SourceText and optimizes the comparison for the case we the SHA1 checksums are available.
 (changeset 1260412)

FeaturePacks have changed to become HostServices.
IWorkspaceServiceProvider has become HostWorkspaceServices
ILanguageServiceProvider has become HostLanguageServices

HostWorkspaceServices can be accessed directly from Workspace.Services.
HostLanguageServices can be accessed directly from Project.LanguageServices.

Some of the existing workspace and language services have been made public. (changeset 1226729)

for now, it only contains Formatting Tests, but in future, we should split mixed lang tests in service test to each own one. (changeset 1223390)

Expose AnalyzerReferences instead of the actual Analyzers at workspace layer: After taking feedback from Matt and Neal, we have decided to expose the analyzer reference assemblies from the workspace, instead of the actual diagnostic analyzers. This change adds the following public types (analogous to MetadataReference hierarchy) and changes the workspace APIs to work with the base AnalyzerReference type:
1) AnalyzerReference: Base type representing all analyzer references.
2) AnalyzerFileReference: Analyzer reference based on actual assembly file on disk. Analyzers are loaded lazily and weakly cached.
3) AnalyzerImageReference: In-memory or pseudo analyzer reference which is just a wrapper over an array of analyzers. This should enable clients to add custom analyzers directly to a project, without requiring a file reference.
4) UnresolvedAnalyzerReference: Internal type for unresolved analyzer references on command line. (changeset 1213934)

Remove use of obsolete type Xunit.Sdk.TraceAssertException added by my previous checkin. (changeset 1211338)

User story 900556 (Workspace support for analyzers\rulesets): Add Workspace APIs for Per-Project DiagnosticAnalyzers:
1) Property to get analyzers: Analyzers
2) Methods to update analyzers: AddAnalyzer(s), RemoveAnalyzer, WithAnalyzers
Also updated the IDE diagnostic service to now work for per-project analyzers.

Tests: I have added a test for new workspace APIs. I am working on adding tests for the diagnostic service per-project analyzer support, using the added workspace APIs for populating per-project analyzers. (changeset 1211161)