This adds a warning when `next export` and custom routes are defined  outside of a platform that supports them since they won't be applied anymore. 

* Expose dotenv loading under separate package

* Update pre-compiled

* Rename package to @next/env

* Update lint ignores

* Update package.json
Co-authored-by: NJoe Haddad <joe.haddad@zeit.co>

This PR fixes `info  - undefined (N/N)` log for `export` cli.

![image](https://user-images.githubusercontent.com/6259812/93186247-5d801500-f779-11ea-89ec-e20939d7b7c1.png)

Default parameter `label` for `createProgress()` was always ignored by `${Log.prefixes.info} undefined` 😅 .

This PR adds a second experimental post-processing step for the framework introduced by @prateekbh in #14746. The image post-processing step scans the rendered document for the first few images and uses a simple heuristic to determine if the images should be automatically preloaded.

Analysis of quite a few production Next apps has shown that a lot of sites are taking a substantial hit to their [LCP](https://web.dev/lcp/) score because an image that's part of the "hero" element on the page is not preloaded and is getting downloaded with lower priority than the JavaScript bundles. This post-processor should automatically fix that for a lot of sites, without causing any real performance effects in cases where it fails to identify the hero image.

This feature is behind an experimental flag, and will be subject to quite a bit of experimentation and tweaking before it's ready to be made a default setting.

By popular request, this pull request adds support for returning `fallback: 'blocking'` from `getStaticPaths`.

This new mode will cause unknown paths to be rendered on-demand ("SSR") without the static (placeholder) fallback.

This feature is **currently experimental and should not be used in production yet**. It's currently flagged behind `unstable_`:

```
fallback: 'unstable_blocking'
```

TODO:

- [x] Next.js tests
- [ ] Add Vercel support
- [ ] Vercel tests

---

Fixes #15637

Co-authored-by: NTim Neutkens <timneutkens@me.com>
Co-authored-by: NJJ Kasper <jj@jjsweb.site>

In terms of url rewriting, `trailingSlash` supports everything `exportTrailingSlash` does. We can just share all other code paths and deprecate `exportTrailingSlash`.

This PR shows a deprecation warning when `exportTrailingSlash` is used.

Also fixes https://github.com/vercel/next.js/issues/15774

We can update the tests now or later. (I kept them the same to prove it's non-breaking)

To do:
- [x] Do we want to keep this? => nope https://github.com/vercel/next.js/blob/841d4efc517a379d8947f491e3c6a1c4784bbccf/packages/next/next-server/lib/router/router.ts#L329
- [x] I kept `exportTrailingSlash` here. Do we want to rename that as well? => nope https://github.com/vercel/next.js/blob/2d9d649d492b77adeddea0242ab6b7fe73d18079/packages/next/build/index.ts#L959

Co-authored-by: Natcastle <atcastle@gmail.com>

Gets rid of the custom function for naming files by removing buildId from the file paths.

This moves the experimental `basePath` config out of the `experimental` section to prepare it for being stable

Fixes #13972

Went through and removed a bunch of internal options which are just pass-through values of buildManifest

Closes #13851

Disambiguate between pages/index.js and pages/index/index.js so that they resolve differently.
It all started with a bug in pagesmanifest that propagated throughout the codebase. After fixing pagesmanifest I was able to remove a few hacks here and there and more logic is shared now. especially the logic that resolves an entrypoint back into a route path. To sum up what happened:

- `getRouteFromEntrypoint` is the inverse operation of `getPageFile` that's under `pages/_document.tsx`
- `denormalizePagePath` is the inverse operation of `normalizePagePath`.

Everything is refactored in terms of these operations, that makes their behavior uniform and easier to update/patch in a central place. Before there were subtle differences between those that made `index/index.js` hard to handle.

Some potential follow up on this PR:
- [`hot-reloader`](https://github.com/vercel/next.js/pull/13699/files#diff-6161346d2c5f4b7abc87059d8768c44bR207) still has one place that does very similar behavior to `getRouteFromEntrypoint`. It can probably be rewritten in terms of `getRouteFromEntrypoint`.
- There are a few places where `denormalizePagePath(normalizePagePath(...))` is happening. This is a sign that `normalizePagePath` is doing some validation that is independent of its rewriting logic. That should probably be factored out in its own function. after that I should probably investigate whether `normalizePagePath` is even still needed at all.
- a lot of code is doing `.replace(/\\/g, '')`. If wanted, that could be replaced with `normalizePathSep`.
- It looks to me like some logic that's spread across the project can be centralized in 4 functions 
  - `getRouteFromEntrypoint` (part of this PR)
  - its inverse `getEntrypointFromRoute` (already exists in `_document.tsx` as `getPageFile`)
  - `getRouteFromPageFile` 
  - its inverse `getPageFileFromRoute` (already exists as `findPageFile ` in `server/lib/find-page-file.ts`)

  It could be beneficial to structure the code to keep these fuctionalities close together and name them similarly.
 - revise `index.amp` handling in pagesmanifest. I left it alone in this PR to keep it scoped, but it may be broken wrt nested index files as well. It might even make sense to reshape the pagesmanifest altogether to handle html/json/amp/... better

Was going through _document and noticed some variable shadowing going on. Added a rule for it to our eslint configuration and went through all warnings with @Timer.

This makes sure we don't duplicate the `/` route or any others while exporting with `exportTrailingSlash` enabled. We do still output `404.html` with `exportTrailingSlash` enabled in case anyone was relying on this file being present.

Fixes: https://github.com/zeit/next.js/issues/11008

Closes #9990 by collecting all paths with errors during `next export` and reporting them sorted in the error summary at the end.

It will produce an output similar to:

```
    Error: Export encountered errors on following paths:
        /nested/page
        /page
        /page-1
        /page-10
        /page-11
        /page-12
        /page-13
        /page-2
        /page-3
        /page-4
        /page-5
        /page-6
        /page-7
        /page-8
        /page-9
        at _default (/app/next.js/packages/next/dist/export/index.js:19:788)
        at process._tickCallback (internal/process/next_tick.js:68:7)
```

I tested the output with the `handle-export-errors` integration test suite, but I'm not sure how to gracefully test this added output.
I thought of collecting all page source files with [recursiveReaddirSync](https://github.com/zeit/next.js/blob/2ba352da39ee00b6595aecdc9ffb2f103e803a85/packages/next/next-server/server/lib/recursive-readdir-sync.ts) but it seems I can't import it in js test files:

```
SyntaxError: /app/next.js/packages/next/next-server/server/lib/recursive-readdir-sync.ts: Unexpected token, expected "," (11:5)

       9 |  */
      10 | export function recursiveReadDirSync(
    > 11 |   dir: string,
         |      ^
      12 |   arr: string[] = [],
      13 |   rootDir = dir
      14 | ): string[] {
```

The test itself could look like:
```js
  it('Reports failing paths', async () => {
    const { stderr } = await nextBuild(appDir, [], {
      stdout: true,
      stderr: true,
    })
    const pages = []
    // collect pages to be ['/page', '/page-1', ... etc.]
    pages.forEach(page => {
      expect(stderr).toContain(page)
    })
  })
```

This is purely adding types except for the line:

```js
params = getRouteMatcher(getRouteRegex(page))(path) || undefined
```

Also added a couple of `@ts-ignore` as to avoid having to make any more functional changes

Added a few extra types here and there while I was digging through the code

Fixes #13171

* Add basePath in link component and add/remove it consistently

* Update to not use regex for delBasePath

* Expose addBasePath as router method

* Revert "Expose addBasePath as router method"

This reverts commit 40fed596195c6affabf837e42d472452768e13a3.

* Expose basePath as router field

* Apply suggestion

* Expose basePath as router field

* remove un-used vars

* Update externals

* Apply lint fix

* Update size-limit test

* Update prefetch

* Remove req.env

* Rename NEXT_APP_ to NEXT_PUBLIC_

* Remove key exposing env config

* Update tests

* Update NEXT_APP_ test to NEXT_PUBLIC_

* Add initial support for new env config file

* Fix serverless processEnv call when no env is provided

* Add missing await for test method

* Update env config to .env.json and add dotenv loading

* ncc dotenv package

* Update type

* Update with new discussed behavior removing .env.json

* Update hot-reloader createEntrypoints

* Make sure .env is loaded before next.config.js

* Add tests for all separate .env files

* Remove comments

* Add override tests

* Add test for overriding env vars based on local environment

* Add support for .env.test

* Apply suggestions from code review
Co-Authored-By: NJoe Haddad <joe.haddad@zeit.co>

* Use chalk for env loaded message

* Remove constant as it’s not needed

* Update test

* Update errsh, taskr, and CNA template ignores

* Make sure to only consider undefined missing

* Remove old .env ignore

* Update to not populate process.env with loaded env

* Add experimental flag and add loading of global env values
Co-authored-by: NTim Neutkens <timneutkens@me.com>
Co-authored-by: NJoe Haddad <joe.haddad@zeit.co>

* Make sure to copy AMP SSG files during export

* bump

* bump

* Fix warning for API routes with next export

* Update warning copy

* Add custom amp optimizer and skip validation mode

* fix: type

* fix worker lint errors

* bump mkdirp to 1.0.3, which support promise

* fix mkdirp in spr-cache.ts

* bump fs-extra to 9.0.0

* remove mkdirp

* Check next.config.js settings

* Add tests

* test package.json case too

* Update to use existing util to de-dupe path check

* Update error message for requested/resolved mismatch

* Use correct dataRoute value for prerender manifest

* Fix pageUrl having double slash on Windows

* Changing variable name internally

* Add tests for custom AMP validator
Co-authored-by: NJJ Kasper <jj@jjsweb.site>

* Rename some spr related stuff to ssg

* Update packages/next/lib/constants.ts
Co-authored-by: NJoe Haddad <timer150@gmail.com>