未验证 提交 d8f85275 编写于 作者: R Radek Doulik 提交者: GitHub

[wasm] Emscripten 3.1.1 bump (#63894)

* Initial changes for emscripten 2.0.34

* Use emcc-link.rsp in build targets

* Use updated docker images

* Fix compiler warnings
Co-authored-by: NZoltan Varga <vargaz@gmail.com>

* Put `--profiling-funcs` to `_EmccLinkFlags`

* Fix build

    src/mono/mono/mini/mini-runtime.c:3407:25: error: ‘invoke’ undeclared (first use in this function); did you mean ‘revoke’?
       3407 |                         invoke = mono_marshal_get_runtime_invoke_dynamic ();

* Add shell to the environment

Environment setting https://github.com/emscripten-core/emscripten/blob/2.0.34/src/settings.js#L616-L641

From emscripten 2.0.25 release notes

    - Support for the 'shell' environment is now disabled by default.  Running under
      `d8`, `js`, or `jsc` is not something that most emscripten users ever want to
      do, so including the support code is, more often than not, unnecessary.  Users
      who want shell support can enable it by including 'shell' in `-s ENVIRONMENT`
      (#14535).

Example of the the size increase for bench sample:

    -a---          12/10/2021  3:35 PM         382113 dotnet.js
    -a---          12/13/2021 10:37 AM         383589 dotnet.js

* Add emcc-link.rsp to PlatformManifestFileEntry

* Feedback

https://github.com/emscripten-core/emscripten/blob/2fda25eea756c78c8cb024aa5b6c2b188bf7990f/src/settings.js#L1173-L1176

-s EXPORT_ES6 is link option

* Bump emscripten version

* Bump llvm package version and use its libclang

* Use newer docker images with emscripten 3.1.1

* Remove unused variable

* Add note about icu repo

* Add comment to improve code readability

* Bump the ICU and emsdk versions

* disable non-wasm builds

* disable wasm jobs in runtime, and runtime-staging, as we'll be running runtime-wasm manually

* [wasm] Re-enable tests that were disabled due to the earlier emsdk bump

System.Linq.Tests.csproj: https://github.com/dotnet/runtime/issues/64774

System.Text.Json.Tests.csproj: https://github.com/dotnet/runtime/issues/64775

System.Text.RegularExpressions.Tests.csproj: https://github.com/dotnet/runtime/issues/64769

System.Threading.Channels.Tests.csproj: https://github.com/dotnet/runtime/issues/65012

System.Collections.Immutable.Tests.csproj: https://github.com/dotnet/runtime/issues/65335

* [wasm] Disable nodejs tests

They are still unstable, and fail with:

```
[01:27:29] fail: /datadisks/disk1/work/A3DA08F7/w/B1B30976/e/test-main.js:97
[01:27:29] fail: if (typeof globalThis.crypto === 'undefined') {
[01:27:29] fail: ^
[01:27:29] fail:
[01:27:29] fail: ReferenceError: globalThis is not defined
[01:27:29] fail:     at Object.<anonymous> (/datadisks/disk1/work/A3DA08F7/w/B1B30976/e/test-main.js:97:1)
[01:27:29] fail:     at Module._compile (internal/modules/cjs/loader.js:778:30)
[01:27:29] fail:     at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
[01:27:29] fail:     at Module.load (internal/modules/cjs/loader.js:653:32)
[01:27:29] fail:     at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
[01:27:29] fail:     at Function.Module._load (internal/modules/cjs/loader.js:585:3)
[01:27:29] fail:     at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
[01:27:29] fail:     at startup (internal/bootstrap/node.js:283:19)
[01:27:29] fail:     at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3)
```

Issue: https://github.com/dotnet/runtime/issues/65485

* fix wbt build

* Update path to node in helix commands.

* disable non-wasm builds

* [wasm] Make emscripten version mismatch an error on CI

* Bump emscripten packages used to preview3, which has emsdk 3.1.1

* Update emscripten package version in Versions.props tooo

* Re-enable wasm jobs

* enable non-wasm jobs

* Add more missing exports

* __dl_seterr shouldn't be needed

* Fix MicrosoftNETCoreRuntimeICUTransportVersion

* Put back the 3.1.1 docker images after merge

* Revert "fix wbt build"

This reverts commit f7c77d995e36130118dd9d98c2c66809aaf42fe2.

* Revert "[wasm] Disable nodejs tests"

This reverts commit 7b90965638b7e90bf4d35c69696f2f361a04ef20.

* Use newer ICU package, last that still had 3.1.1

* Try different helix queue parameter

* Revert "Try different helix queue parameter"

This reverts commit a34230f7fe04760df30c5e63e14826575f1a7545.

* Fix helix queue with docker tag

* Try to set PYTHONPATH for emscripten

* [wasm] Do not set _WasmDevel=true for blazor projects

* Increase timeout

The AOT tests are canceled after timeout

* Fix WBT tests
Co-authored-by: NAnkit Jain <radical@gmail.com>

* [wasm] Export more functions

`htons,ntohs,_get_daylight,_get_timezone,_get_tzname`

.. to fix:

```
EXEC : warning : undefined symbol: htons (referenced by $inetPton6__deps: ['htons','$jstoi_q'], referenced by $writeSockaddr__deps: ['$Sockets','$inetPton4','$inetPton6','$zeroMemory'], referenced by __syscall_accept4__deps: ['$getSocketFromFD','$writeSockaddr','$DNS','$SYSCALLS'], referenced by top-level compiled C/C++ code) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
EXEC : warning : undefined symbol: ntohs (referenced by $inetNtop6__deps: ['$inetNtop4','ntohs'], referenced by $readSockaddr__deps: ['$Sockets','$inetNtop4','$inetNtop6','ntohs'], referenced by $getSocketAddress__deps: ['$readSockaddr','$FS','$DNS'], referenced by __syscall_bind__deps: ['$getSocketFromFD','$getSocketAddress','$SYSCALLS'], referenced by top-level compiled C/C++ code) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
EXEC : warning : undefined symbol: _get_daylight (referenced by tzset_impl__deps: ['_get_daylight','_get_timezone','_get_tzname'], referenced by tzset__deps: ['tzset_impl'], referenced by localtime_r__deps: ['tzset'], referenced by top-level compiled C/C++ code) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
EXEC : warning : undefined symbol: _get_timezone (referenced by tzset_impl__deps: ['_get_daylight','_get_timezone','_get_tzname'], referenced by tzset__deps: ['tzset_impl'], referenced by localtime_r__deps: ['tzset'], referenced by top-level compiled C/C++ code) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
EXEC : warning : undefined symbol: _get_tzname (referenced by tzset_impl__deps: ['_get_daylight','_get_timezone','_get_tzname'], referenced by tzset__deps: ['tzset_impl'], referenced by localtime_r__deps: ['tzset'], referenced by top-level compiled C/C++ code) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
```

```
Starting:    Microsoft.Extensions.Logging.Tests.dll
Message1
:
testKey3:testValue, testKey2:, testKey1:
missing function: ntohs
Error
    at Object.onAbort (test-main.js:176:60)
    at abort (./dotnet.js:1316:24)
    at _ntohs (./dotnet.js:4990:35)
    at readSockaddr (./dotnet.js:5091:18)
    ...
```

* fix exported function names
Co-authored-by: NRadek Doulik <radekdoulik@google.com>
Co-authored-by: NZoltan Varga <vargaz@gmail.com>
Co-authored-by: NLarry Ewing <lewing@microsoft.com>
Co-authored-by: NAnkit Jain <radical@gmail.com>
Co-authored-by: NMarek Fišera <mara@neptuo.com>
上级 5a2b39b0
<Dependencies>
<ProductDependencies>
<Dependency Name="Microsoft.NETCore.Runtime.ICU.Transport" Version="7.0.0-preview.3.22157.1">
<Dependency Name="Microsoft.NETCore.Runtime.ICU.Transport" Version="7.0.0-preview.3.22121.1">
<Uri>https://github.com/dotnet/icu</Uri>
<Sha>5416401e2e54d6ce9bef7f0c4a5fe25088c42652</Sha>
<Sha>d3c9d26f8e787253536d38504d3ac9e307573bfb</Sha>
</Dependency>
<Dependency Name="System.Net.MsQuic.Transport" Version="7.0.0-alpha.1.22160.2">
<Uri>https://github.com/dotnet/msquic</Uri>
<Sha>4fe73fcad4b0af91cc49fd1f5576f0111f15f6fd</Sha>
</Dependency>
<Dependency Name="Microsoft.NET.Workload.Emscripten.Manifest-7.0.100" Version="7.0.0-preview.3.22167.1">
<Dependency Name="Microsoft.NET.Workload.Emscripten.Manifest-7.0.100" Version="7.0.0-preview.3.22121.1">
<Uri>https://github.com/dotnet/emsdk</Uri>
<Sha>d2be7d0bf644d9608bf8180d706483238d85f13a</Sha>
<Sha>b8e71431b3eaa024afad7886699f8ac6bca2ba01</Sha>
</Dependency>
<Dependency Name="System.ServiceModel.Primitives" Version="4.9.0-rc2.21473.1">
<Uri>https://github.com/dotnet/wcf</Uri>
......
......@@ -171,7 +171,7 @@
<MicrosoftNETILLinkTasksVersion>7.0.100-1.22168.6</MicrosoftNETILLinkTasksVersion>
<MicrosoftNETILLinkAnalyzerPackageVersion>$(MicrosoftNETILLinkTasksVersion)</MicrosoftNETILLinkAnalyzerPackageVersion>
<!-- ICU -->
<MicrosoftNETCoreRuntimeICUTransportVersion>7.0.0-preview.3.22157.1</MicrosoftNETCoreRuntimeICUTransportVersion>
<MicrosoftNETCoreRuntimeICUTransportVersion>7.0.0-preview.3.22121.1</MicrosoftNETCoreRuntimeICUTransportVersion>
<!-- MsQuic -->
<SystemNetMsQuicTransportVersion>7.0.0-alpha.1.22160.2</SystemNetMsQuicTransportVersion>
<!-- Mono LLVM -->
......@@ -184,7 +184,7 @@
<runtimeosx1012x64MicrosoftNETCoreRuntimeMonoLLVMSdkVersion>11.1.0-alpha.1.22164.2</runtimeosx1012x64MicrosoftNETCoreRuntimeMonoLLVMSdkVersion>
<runtimeosx1012x64MicrosoftNETCoreRuntimeMonoLLVMToolsVersion>11.1.0-alpha.1.22164.2</runtimeosx1012x64MicrosoftNETCoreRuntimeMonoLLVMToolsVersion>
<!-- emscripten / Node -->
<MicrosoftNETWorkloadEmscriptenManifest70100Version>7.0.0-preview.3.22167.1</MicrosoftNETWorkloadEmscriptenManifest70100Version>
<MicrosoftNETWorkloadEmscriptenManifest70100Version>7.0.0-preview.3.22121.1</MicrosoftNETWorkloadEmscriptenManifest70100Version>
<MicrosoftNETRuntimeEmscriptenVersion>$(MicrosoftNETWorkloadEmscriptenManifest70100Version)</MicrosoftNETRuntimeEmscriptenVersion>
<!-- workloads -->
<SwixPackageVersion>1.1.87-gba258badda</SwixPackageVersion>
......
......@@ -303,7 +303,7 @@ jobs:
targetRid: browser-wasm
platform: Browser_wasm
container:
image: ubuntu-18.04-webassembly-20210707133424-12f133e
image: ubuntu-18.04-webassembly-20220118141522-180197c
registry: mcr
jobParameters:
hostedOs: Linux
......
......@@ -35,7 +35,7 @@ jobs:
testGroup: innerloop
nameSuffix: LibraryTests${{ parameters.nameSuffix }}
buildArgs: -s mono+libs+host+packs+libs.tests -c $(_BuildConfig) /p:ArchiveTests=true /p:BrowserHost=$(_hostedOs) ${{ parameters.runSmokeOnlyArg }} ${{ parameters.extraBuildArgs }}
timeoutInMinutes: 180
timeoutInMinutes: 240
# always run for runtime-wasm builds (triggered manually)
# Always run for rolling builds
# Else run on path changes
......
......@@ -182,6 +182,6 @@ jobs:
# WebAssembly windows
- ${{ if eq(parameters.platform, 'Browser_wasm_win') }}:
- (Windows.Server.Core.1909.Amd64.Open)windows.10.amd64.server20h2.open@mcr.microsoft.com/dotnet-buildtools/prereqs:windowsservercore-2004-helix-webassembly-amd64-20210702131541-6837048
- (Windows.Amd64.Server2022.Open)windows.amd64.server2022.open@mcr.microsoft.com/dotnet-buildtools/prereqs:windowsservercore-ltsc2022-helix-webassembly-20220118164817-dc6e6f1
${{ insert }}: ${{ parameters.jobParameters }}
......@@ -414,6 +414,7 @@ static gboolean buffer_replies;
DebuggerTlsData *tls; \
tls = (DebuggerTlsData *)mono_native_tls_get_value (debugger_tls_id);
#else
/* the thread argument is omitted on wasm, to avoid compiler warning */
#define GET_TLS_DATA_FROM_THREAD(...) \
DebuggerTlsData *tls; \
tls = &debugger_wasm_thread;
......
......@@ -188,6 +188,7 @@ Bumping Emscripten version involves these steps:
* update https://github.com/dotnet/runtime/blob/main/src/mono/wasm/emscripten-version.txt
* bump emscripten versions in docker images in https://github.com/dotnet/dotnet-buildtools-prereqs-docker
* bump emscripten in https://github.com/dotnet/emsdk
* bump docker images in https://github.com/dotnet/icu, update emscripten files in eng/patches/
* update version number in docs
* update `Microsoft.NET.Runtime.Emscripten.<emscripten version>.Node.win-x64` package name, version and sha hash in https://github.com/dotnet/runtime/blob/main/eng/Version.Details.xml and in https://github.com/dotnet/runtime/blob/main/eng/Versions.props. the sha is the commit hash in https://github.com/dotnet/emsdk and the package version can be found at https://dev.azure.com/dnceng/public/_packaging?_a=feed&feed=dotnet6
* update packages in the workload manifest https://github.com/dotnet/runtime/blob/main/src/mono/nuget/Microsoft.NET.Workload.Mono.Toolchain.Manifest/WorkloadManifest.json.in
......
......@@ -137,7 +137,7 @@ them for the new task assembly.
If encountering build performance issues, you can use the rollup `--perf` option and the typescript compiler `--generateCpuProfile` option to get build profile data, like so:
```../emsdk/node/14.15.5_64bit/bin/npm run rollup --perf -- --perf --environment Configuration:Release,NativeBinDir:./rollup-test-data,ProductVersion:12.3.4```
```../emsdk/node/14.18.2_64bit/bin/npm run rollup --perf -- --perf --environment Configuration:Release,NativeBinDir:./rollup-test-data,ProductVersion:12.3.4```
```node node_modules/typescript/lib/tsc.js --generateCpuProfile dotnet-tsc.cpuprofile -p tsconfig.json ```
......
......@@ -255,6 +255,7 @@
<_WasmNativeFileForLinking Include="@(NativeFileReference)" />
<EmscriptenEnvVars Include="EMSDK_PYTHON=$(EmscriptenPythonToolsPath)python.exe" Condition="'$(OS)' == 'Windows_NT'" />
<EmscriptenEnvVars Include="PYTHONPATH=$(EmscriptenPythonToolsPath)" Condition="'$(OS)' == 'Windows_NT'" />
<EmscriptenEnvVars Include="EM_CACHE=$(WasmCachePath)" Condition="'$(WasmCachePath)' != ''" />
</ItemGroup>
......
......@@ -119,7 +119,7 @@
<_EmccLinkFlags Include="-s FORCE_FILESYSTEM=1" />
<_EmccLinkFlags Include="-s EXPORTED_RUNTIME_METHODS=&quot;['FS','print','ccall','cwrap','setValue','getValue','UTF8ToString','UTF8ArrayToString','FS_createPath','FS_createDataFile','removeRunDependency','addRunDependency', 'FS_readFile']&quot;" />
<!-- _htons,_ntohs,__get_daylight,__get_timezone,__get_tzname are exported temporarily, until the issue is fixed in emscripten, https://github.com/dotnet/runtime/issues/64724 -->
<_EmccLinkFlags Include="-s EXPORTED_FUNCTIONS=_free,_malloc,_memalign,_memset" Condition="$([MSBuild]::VersionGreaterThan('$(_EmccVersion)', '3.0'))" />
<_EmccLinkFlags Include="-s EXPORTED_FUNCTIONS=_free,_malloc,_memalign,_memset,_htons,_ntohs,__get_daylight,__get_timezone,__get_tzname" Condition="$([MSBuild]::VersionGreaterThan('$(_EmccVersion)', '3.0'))" />
<_EmccLinkFlags Include="-s EXPORTED_FUNCTIONS=_free,_malloc,_htons,_ntohs,__get_daylight,__get_timezone,__get_tzname,_memalign" Condition="$([MSBuild]::VersionLessThan('$(_EmccVersion)', '3.0'))" />
<_EmccLinkFlags Include="--source-map-base http://example.com" />
<_EmccLinkFlags Include="-s STRICT_JS=1" />
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册