Updates the way filenames are generated for browser compilation.
Notably:
- All entry bundles now have hashes in production, this includes pages (previously pages used a buildId in the path)
- The AmpFiles no longer depends on hardcoded bundle names, it uses the buildManifest instead (internals)
- All cases where we match the page name from the chunk/entrypoint name now use the same function `getRouteFromEntrypoint` (internals)
- In development we no longer include the "faked" `buildId` set to `development` for page files, instead we just use the `/_next/static/pages` path (was `/_next/static/development/pages`). This was changed as it caused unneeded complexity and makes generating the bundles easier (internals)
- Updated tons of tests to be more resilient to these changes by relying on the buildManifest instead of hardcoded paths (internals)

Follow up of these PRs:
https://github.com/vercel/next.js/pull/13759
https://github.com/vercel/next.js/pull/13870
https://github.com/vercel/next.js/pull/13937
https://github.com/vercel/next.js/pull/14130
https://github.com/vercel/next.js/pull/14176
https://github.com/vercel/next.js/pull/14268


Fixes #6303
Fixes #12087 
Fixes #1948
Fixes #4368
Fixes #4255
Fixes #2548

This moves the experimental `basePath` config out of the `experimental` section to prepare it for being stable

Co-authored-by: NJJ Kasper <jj@jjsweb.site>
Co-authored-by: Nkodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>

Extracted from https://github.com/vercel/next.js/pull/13333, the same exact code lives in that PR as well, but we can merge this separately if it makes reviewing https://github.com/vercel/next.js/pull/13333 easier

This PR does 3 things 
- deduplicate code from build and next-dev-server that loads custom routes from next.config.js  (`loadCustomRoutes`)
- in `loadCustomRoutes`, load these rewrites, headers and redirects configs concurrently instead of sequentially.
- in next-server, make `this.customRoutes` always defined, this allows us to remove the big `if` around its initialization code in `generateRoutes`, which in turn makes it possible to reuse this code for other routing than user defined routes, which is how https://github.com/vercel/next.js/pull/13333 adds its redirects.

Went through and removed a bunch of internal options which are just pass-through values of buildManifest

Closes #13851

Disambiguate between pages/index.js and pages/index/index.js so that they resolve differently.
It all started with a bug in pagesmanifest that propagated throughout the codebase. After fixing pagesmanifest I was able to remove a few hacks here and there and more logic is shared now. especially the logic that resolves an entrypoint back into a route path. To sum up what happened:

- `getRouteFromEntrypoint` is the inverse operation of `getPageFile` that's under `pages/_document.tsx`
- `denormalizePagePath` is the inverse operation of `normalizePagePath`.

Everything is refactored in terms of these operations, that makes their behavior uniform and easier to update/patch in a central place. Before there were subtle differences between those that made `index/index.js` hard to handle.

Some potential follow up on this PR:
- [`hot-reloader`](https://github.com/vercel/next.js/pull/13699/files#diff-6161346d2c5f4b7abc87059d8768c44bR207) still has one place that does very similar behavior to `getRouteFromEntrypoint`. It can probably be rewritten in terms of `getRouteFromEntrypoint`.
- There are a few places where `denormalizePagePath(normalizePagePath(...))` is happening. This is a sign that `normalizePagePath` is doing some validation that is independent of its rewriting logic. That should probably be factored out in its own function. after that I should probably investigate whether `normalizePagePath` is even still needed at all.
- a lot of code is doing `.replace(/\\/g, '')`. If wanted, that could be replaced with `normalizePathSep`.
- It looks to me like some logic that's spread across the project can be centralized in 4 functions 
  - `getRouteFromEntrypoint` (part of this PR)
  - its inverse `getEntrypointFromRoute` (already exists in `_document.tsx` as `getPageFile`)
  - `getRouteFromPageFile` 
  - its inverse `getPageFileFromRoute` (already exists as `findPageFile ` in `server/lib/find-page-file.ts`)

  It could be beneficial to structure the code to keep these fuctionalities close together and name them similarly.
 - revise `index.amp` handling in pagesmanifest. I left it alone in this PR to keep it scoped, but it may be broken wrt nested index files as well. It might even make sense to reshape the pagesmanifest altogether to handle html/json/amp/... better

Co-authored-by: NJoe Haddad <joe.haddad@zeit.co>

In serverless mode, it's best practice to propagate an unhandled error so that the function is disposed instead of recycled. This helps fix the "bad state" problem.

Was going through _document and noticed some variable shadowing going on. Added a rule for it to our eslint configuration and went through all warnings with @Timer.

Fixes https://github.com/vercel/next.js/issues/13524

To do:
- [x] fix dev mode
- [x] there's a ~route ordering or~ trailing slash issue with top level catch-all (current tests reflect that)
- [x] in this test `/get-static-paths/whatever` should fall back to `/[[...optionalName]].js` since `fallback` is `false` in its `getStaticPaths` method. ~Currently seems to 500~ must have been a glitch
- [x] add tests for `null`, `undefined` ~and `false`~ behavior as well (if decided these are valid)
- [x] ~add tests for string params as well~ this is not allowed for catch-all routes
- [x] test behavior when fallback is enabled and a top level catch-all exists

This addresses some errors for `/_next/data` requests where encoded `/` values in dynamic route param would cause invalid behavior, a headers already sent error would be shown when sending the fallback page in development, and when rendering the `_error` page for a data request the error response would still be treated as a data request. 

This also adds test cases for these errors to prevent regression

Since non-fallback pages don't rely on the URL for hydration we can allow them to be rewritten to but pages with fallback still can't be rewritten to because we won't be able to parse the correct `/_next/data` path to request the page's data from. I added a test for this behavior and ensured it works correctly on Now.

Example on with fallback false rewrite on Now:
https://tst-rewrite-cp9vge4bg.now.sh/about

Fixes https://github.com/zeit/next.js/issues/12935

Added a few extra types here and there while I was digging through the code

This continues https://github.com/zeit/next.js/pull/12501 and resolves the merge conflicts since we aren't able to update that branch and would like to land this change

Closes: https://github.com/zeit/next.js/pull/12501

Failing test case for https://github.com/zeit/next.js/issues/10488#issuecomment-584500081

This used to give a 500 in dev environment

* Add URL escaping for custom headers

* Update to handle more header values
Co-authored-by: NJJ Kasper <jj@jjsweb.site>

* Add support for params in header key/values

* Update todo

* Update with handling for named patterns

* Use compiled package
Co-authored-by: NJoe Haddad <joe.haddad@zeit.co>

* Add basePath in link component and add/remove it consistently

* Update to not use regex for delBasePath

* Expose addBasePath as router method

* Revert "Expose addBasePath as router method"

This reverts commit 40fed596195c6affabf837e42d472452768e13a3.

* Expose basePath as router field

* Apply suggestion

* Expose basePath as router field

* remove un-used vars

* Update externals

* Apply lint fix

* Update size-limit test

* Update prefetch

* Add error/warning when a page is rendered without a /

Throws an error for development and gives a warning in production

* Add tests for error when rendering without starting slash

* Update to always warn and add err.sh

* Update errors/render-no-starting-slash.md
Co-authored-by: NJJ Kasper <jj@jjsweb.site>
Co-authored-by: NTim Neutkens <tim@timneutkens.nl>

* Warn when user has pages/_error but no pages/404

* Update test

* Update custom-error-no-custom-404.md

* Apply suggestions from code review
Co-Authored-By: NTim Neutkens <tim@timneutkens.nl>
Co-authored-by: NTim Neutkens <tim@timneutkens.nl>

* Remove req.env

* Rename NEXT_APP_ to NEXT_PUBLIC_

* Remove key exposing env config

* Update tests

* Update NEXT_APP_ test to NEXT_PUBLIC_