* Pass config.experimental.cpus to export during build

Currently, there is no way of specifying the number of worker threads of
`next export` when run as part of `next build`.

I suggest a sane default should be to just use the same amount of
workers that were used during the build process which currently seems to
be configured through `config.experimental.cpus`.

This setting is already respected in the two other places where
jest-workers are in use: The TerserPlugin and the staticCheckWorkers in
`next build`.

* Only enable worker threads if there is more than 1 worker

Multiple worker threads can cause problems when certain dependencies are
being used, see e.g. https://github.com/zeit/next.js/issues/7894

This patch allows disabling of worker threads by setting
`config.experimental.cpus = 1`.

The benefit of spawning 1 worker thread, if there is any at all, should
very limited anyways, so the workload can just as well be processed in
the main thread.

* Disable parallel build for firebase authentication example

* Add integration test to cover #7894

* Rename test suite and add worker_threads config

* Disable worker_threads by default

* Update index.test.js

* Use workerThreads config for TerserPlugin

* Update to use workerThreads config in
TerserPlugin for consistency

* Disable node 12 specific test

* Add tracking src dir usage to telemetry

* Move isSrcDir back to eventVersion

* Move spinner back

* Add test for isSrcDir telemetry

* Add test for dev mode

* Do Not Rely on Module State

* Update storage.ts

* Add buildId to SPR data routes

* Update buildId replace in serverless loader

* Use new RegExp and add comment

* Apply suggestions from code review
Co-Authored-By: NJoe Haddad <joe.haddad@zeit.co>

* Test to ensure client transition and handle / data route

* Remove experimental.publicDirectory config

* Error when public is set as an output dir

* Remove experimental.publicDirectory checks

* Update publicFiles checking in next build

This pull request converts Next.js' export CLI command to use TypeScript. This is useful for upcoming changes.

* Use Better Telemetry Directory
So, as it turns out, storing in `node_modules` is a bad idea.

Both npm and Yarn will remove additional files when you run `npm install` or `yarn install`.

Instead, we'll store this inside of Next.js' `distDir`. This should also be cached by users, if it's not, it probably won't be any worse as compared to `node_modules`.

* Fix directory name

* Fix build setup

* Record when export session is started

* Move more into branch

* Add revalidate header and error when invalid
object from getStaticProps

* Make sprCache no-op in dev mode

* Update packages/next/next-server/server/next-server.ts

* Update tests for fixes

* Update getStaticProps error and add test

* Update manifest test

* Add validating revalidate value

* Apply suggestions from code review
Co-Authored-By: NJoe Haddad <joe.haddad@zeit.co>

* Change from error to warning high revalidate value

* initial commit for SPRv2

* Add initial SPR cache handling

* update SPR handling

* Implement SPR handling in render

* Update tests, handle caching with serverless next
start, add TODOs, and update manifest generating

* Handle no prerender-manifest from not being used

* Fix url.parse error

* Apply suggestions from code review
Co-Authored-By: NJoe Haddad <joe.haddad@zeit.co>

* Replace set with constants in next-page-config

* simplify sprStatus.used

* Add error if getStaticProps is used with getInitialProps

* Remove stale TODO

* Update revalidate values in SPR cache for non-seeded routes

* Apply suggestions from code review

* Remove concurrency type

* Rename variable for clarity

* Add copying prerender files during export

* Add comment for clarity

* Fix exporting

* Update comment

* Add additional note

* Rename variable

* Update to not re-export SPR pages from build

* Hard navigate when fetching data fails

* Remove default extension

* Add brackets

* Add checking output files to prerender tests

* Adjust export move logic

* Clarify behavior of export aggregation

* Update variable names for clarity

* Update tests

* Add comment

* s/an oxymoron/contradictory/

* rename

* Extract error case

* Add tests for exporting SPR pages and update
/_next/data endpoint to end with .json

* Relocate variable

* Adjust route building

* Rename to unstable

* Rename unstable_getStaticParams

* Fix linting

* Only add this when a data request

* Update prerender data tests

* s/isServerless/isLikeServerless/

* Don't rely on query for `next start` in serverless mode

* Rename var

* Update renderedDuringBuild check

* Add test for dynamic param with bracket

* Fix serverless next start handling

* remove todo

* Adjust comment

* Update calculateRevalidate

* Remove cache logic from render.tsx

* Remove extra imports

* Move SPR cache logic to next-server

* Remove old isDynamic prop

* Add calling App getInitialProps for SPR pages

* Update revalidate logic

* Add isStale to SprCacheValue

* Update headers for SPR

* add awaiting pendingRevalidation

* Dont return null for revalidation render

* Adjust logic

* Be sure to remove coalesced render

* Fix data for serverless

* Create a method coalescing utility

* Remove TODO

* Extract send payload helper

* Wrap in-line

* Move around some code

* Add tests for de-duping and revalidating

* Update prerender manifest test

* Add progress for analyzing and auto-prerendering

* Add typing for tty-aware-progress and use stdout

* Add fancier spinners

* Update spinner and add handling for logs while spinning

* Remove un-needed types package

* Remove progress and combine analyzing/prerendering messages

* Add failing test

* Add jest-worker to export

* Initial move

* Make emitting work

* Update paths

* Remove leftover files

* Add correct externals configuration

* Import correct path

* Update path to work with ts-server test

* Update lib directory

* Compile next-server/lib

* Fix conflict between defaultPathMap and default 404.html export，if have a page in pages/404.js

* Update packages/next/export/index.js
Co-Authored-By: NJJ Kasper <jj@jjsweb.site>

* add test export-override-404

* Throw error if next export detects an api route

* Moved the test to the export tests

* Added missing comma

* Move the API route comment

* Add comma after replace

* Show a warning instead

* Revert "Remove Old Prerender Implementation (#8218)"

This reverts commit 2ab300dd.

* Add contentHandler for page config

* Rename config from contentHandler to re-use
experimentalPrerender

* Remove un-needed changes

* Replace backslashes for manifest

* Update manifest output format

* Make prerender: true enable SPR behavior and update
to merge prerender-manifest for flying-shuttle

* Fix output path for / prerender file

* Add dynamic routes to test suite

* Add generating and previewing of skeletons
for prerendered dynamic routes

* remove inline prerender option

* update to not replace getInitialProps which allows
nested getInitialProps and add query when fetching prerender

* Apply suggestions from code review
Co-Authored-By: NJoe Haddad <timer150@gmail.com>

* Remove legacy prerender option

* Apply suggestions from review

* Apply more suggestions from review

* Apply suggestions from code review
Co-Authored-By: NJoe Haddad <timer150@gmail.com>

* Add handling of error when parsing json

* Update handling of moving exported pages

* Rename nextPreviewSkeleton to _nextPreviewSkeleton

* bump

* Experimental: Serverless Trace target
The Serverless Trace target produces Serverless-handler wrapped entrypoints, but does not bundle all of `node_modules`.

This behavior increases bundling performance to be more akin to `target: 'server'`.

This mode is expected to be used with smart platforms (like [ZEIT Now](https://zeit.co/now) that can trace a program to its minimum dependencies.

* Use more generic variables

* Add asset relocator for production mode of serverless trace

* Verify Firebase compatiblity

* Revert "Add asset relocator for production mode of serverless trace"

This reverts commit 8404f1dcf28b60edab41a56c94b38dcd3fddec20.

* Add serverless trace tests

* Add _isLikeServerless helper

* Make constants

* Fix export

* Update packages/next-server/server/config.ts
Co-Authored-By: NJJ Kasper <jj@jjsweb.site>

* Use a global helper for is like serverless

* Update import for isTargetLikeServerless

* Update packages/next/build/index.ts
Co-Authored-By: NJJ Kasper <jj@jjsweb.site>

* Module/nomodule implementation based on RFC 7563

* Remove comment

* Fixing issue with building amp pages

* Fixing test cases for serverless mode

* Adding safari 10 nomodule fix. Preloading modern js by default

* Fixing size-limit integration test

* Bug fix

* Adding testcase for modern build

* Trigger rebuild

* Setting default crossOrigin value

* Moving modern config option inside experimental flag

* Adding nomodule attribute to safari-fix script

* Changing safari10NomoduleFix default value to true

* Removing safari-fix flag

* Changing .es6 to .module

* Disable modern default

* Removing default crossOrigin value. Setting modern flag to false by default. Fixed test cases

* Remove confusing defaults and mark required instead

* Adjust blacklist

* Move behavior of page marking

* Fixing childCompiler errors not being captured

* Tweak names

* Revert

* whoops

* Fixing bug with page-loader.js

* Changing modern babel cache name

* Rename helper

* Iterate over both bundles

* Correctly clamp bundle sizes

* Revert test

* Add modern mode tests

* Fix test

* test

* test2

* Add default values for runtime config

* Add test to ensure default values are set
for runtime config

* Only add runtimeConfig if it's not empty

* Only add runtimeConfig if it's not empty

* Simplify default value adding

* Reduce bundle size

* remove comment changes

* Fix typos

* Add test for runtimeConfig in __NEXT_DATA__

* replace recursive-copy with own implementation

* update yarn.lock

* do not filter out not directories

* do not fail if folder already exists

* replace `\` by `/` when sending pathes to filter

* use fs-extra only in tests

* investigate and test recursive-copy npm module

* improve test by creating fixtures programmatically

* remove recursive-copy npm module test

* add recursive-copy to bench

* add bench:recursive-copy script

* fix Sema import in recursive-copy.ts

* small improvements

* Run prettier over packages/**/*.js

* Run prettier over packages/**/*.ts

* Run prettier over examples

* Remove tslint

* Run prettier over examples

* Run prettier over all markdown files

* Run prettier over json files

* Add canonicalBase config to allow setting
absolute path for canonical link

* Make sure canonicalBase is set for
export and serverless

* Move canonicalBase to amp.canonicalBase

* Update tests with canonicalBase config

* Update tests

* run lint-fix

* Fix canonicalBase config parsing

* Fix canonicalBase during export

* Update amphtml tests

* Add automatic exporting of pages with no getInitialProps

* Add support for exporting serverless to static
and serving the html files during next start

* Fix missing runtimeEnv when requiring page, re-add warning
when trying to export with serverless, and update tests

* Update flying-shuttle test

* revert un-used pagesManifest change

* remove query.amp RegExp test

* Fix windows backslashes not being replaced

* Re-enable serverless support for next start

* bump

* Fix getInitialProps check

* Fix incorrect error check

* Re-add check for reserved pages

* Fix static check

* Update to ignore /api pages and clean up some tests

* Re-add needed next.config for test and correct behavior

* Update RegExp for ignored pages for auto-static

* Add checking for custom getInitialProps in pages/_app

* Update isPageStatic logic to only use default export

* Re-add retrying to CircleCi

* Update query during dev to only have values
available during export for static pages

* Fix test

* Add warning when page without default export is
found and make sure to update pages-manifest
correctly in flying-shuttle mode

* Fix backslashes not being replaced

* Integrate auto-static with flying-shuttle
and make sure AMP is handled in flying-shuttle

* Add autoExport for opting in

* Add --no-subfolders argument to next export

* Use experimental.exportTrailingSlash instead of a cli flag

* Add experimental.exportTrailingSlash documentation

* Add tests for export with experimental.exportTrailingSlash

* Remove docs

* Remove comment

This PR adds the `_error`-page as an `404.html`-export, even when it is not explicitly defined in a custom `exportPathMap`.

It also fixes two false negative tests related to this. Previously the tests were matching the fallback 404-page from the test-server, rather than the `404.html`-page from next, which was actually not being generated. The test server is also not set up to serve `/404.html` as `/404` which the tests now reflect.

**Caveat**

In its current state, this PR removes `/404.html` from the `defaultPathMap` passed to the custom `exportPathMap`-functions, since it instead adds it after that function is run. While it is possible that someone is relying on this to exist, it is to my knowledge undocumented and also unlikely to be used for anything but merging it into the custom pathMap.

Since this would now merge `undefined` which would result in it being added later on anyway, I deemed it safe, but would be happy to undo that part of the PR if necessary as it was only cleanup.

**Examples**

As a way to demonstrate what this PR does, this is how examples changed:

* `basic-export` - Behaviour is unchanged, still has a `404.html`
* `with-static-export` - Now has a `404.html`

* Add WithAmp to enable AMP support for
pages instead of .amp.js

* Update handling for exporting AMP

* Fix ampPath in export for / path and
revert isAmp logic to handle right

* Update amphtml test suite

* Add handling for noDirtyAmp during
export and update amp-export test suite

* Update serverless and export-default-map
test suites

* Update require-page tests

* Added amp-toolbox-optimizer and added
optimizing AMP pages as dirty and clean

* Fix amp-optimizer breaking serverless build

* Exclude amp-toolbox-optimizer from serverless build

* Added check to make sure hybrid AMP pages
optimize cleanly during export

* Add check to make sure noDirtyAmp is applied

* Add err.sh link and pool validation results
to wait to show error until export is finished

* Fix wording in amp-export-validation err.sh

* Update validation error message
Co-Authored-By: Nijjk <22380829+ijjk@users.noreply.github.com>

* Update ways to fix text
Co-Authored-By: Nijjk <22380829+ijjk@users.noreply.github.com>

* Update why the error occurred wording
Co-Authored-By: Nijjk <22380829+ijjk@users.noreply.github.com>

* Update wording some more
Co-Authored-By: Nijjk <22380829+ijjk@users.noreply.github.com>

* Fix export not checking amp right for /index.amp

* Add test checking hybrid amp index behavior

* Add more export tests for AMP

* Make sure ampPath is correct when nested

* Remove console.log

* Add more export tests for AMP

* Remove console.log
Co-Authored-By: Nijjk <22380829+ijjk@users.noreply.github.com>

* remove extra line
Co-Authored-By: Nijjk <22380829+ijjk@users.noreply.github.com>

* Add support for amp to export

* Anchor canonical replace

* Disable profiling test for now

* Centralize amp utils to next-server/server/utils

* re-enable profiling test

Removes `rimraf` for a smaller custom lib

Benchmarks (in ms):
```
rimraf 1
518.536376
rimraf 2
416.112451
rimraf 3
451.905842
rimraf 4
525.117395
rimraf 5
434.230384
---- Average: 469.18ms
-----------
recursive delete 1
121.493979
recursive delete 2
130.335272
recursive delete 3
149.798316
recursive delete 4
182.184688
recursive delete 5
130.346207
--- Average: 142.83ms
```
`recursiveDelete` finishes in ~30% of the time it takes `rimraf` (3.3 times faster)