There are a few more related to pretty printing.

This moves the custom sorting to `.sort_by`.

Right now the --crate-id and related flags are all process *after* the entire
crate is parsed. This is less than desirable when used with makefiles because it
means that just to learn the output name of the crate you have to parse the
entire crate (unnecessary).

This commit changes the behavior to lift the handling of these flags much sooner
in the compilation process. This allows us to not have to parse the entire crate
and only have to worry about parsing the crate attributes themselves. The
related methods have all been updated to take an array of attributes rather than
a crate.

Additionally, this ceases duplication of the "what output are we producing"
logic in order to correctly handle things in the case of --test.

Finally, this adds tests for all of this functionality to ensure that it does
not regress.

Fixes #9676.

Closes #11035

Understand 'pkgid' in stage0. As a bonus, the snapshot now contains now metadata
(now that those changes have landed), and the snapshot download is half as large
as it used to be!

This replaces the link meta attributes with a pkgid attribute and uses a hash
of this as the crate hash. This makes the crate hash computable by things
other than the Rust compiler. It also switches the hash function ot SHA1 since
that is much more likely to be available in shell, Python, etc than SipHash.

Fixes #10188, #8523.

This commit implements LTO for rust leveraging LLVM's passes. What this means
is:

* When compiling an rlib, in addition to insdering foo.o into the archive, also
  insert foo.bc (the LLVM bytecode) of the optimized module.

* When the compiler detects the -Z lto option, it will attempt to perform LTO on
  a staticlib or binary output. The compiler will emit an error if a dylib or
  rlib output is being generated.

* The actual act of performing LTO is as follows:

    1. Force all upstream libraries to have an rlib version available.
    2. Load the bytecode of each upstream library from the rlib.
    3. Link all this bytecode into the current LLVM module (just using llvm
       apis)
    4. Run an internalization pass which internalizes all symbols except those
       found reachable for the local crate of compilation.
    5. Run the LLVM LTO pass manager over this entire module

    6a. If assembling an archive, then add all upstream rlibs into the output
        archive. This ignores all of the object/bitcode/metadata files rust
        generated and placed inside the rlibs.
    6b. If linking a binary, create copies of all upstream rlibs, remove the
        rust-generated object-file, and then link everything as usual.

As I have explained in #10741, this process is excruciatingly slow, so this is
*not* turned on by default, and it is also why I have decided to hide it behind
a -Z flag for now. The good news is that the binary sizes are about as small as
they can be as a result of LTO, so it's definitely working.

Closes #10741
Closes #10740

In order to keep up to date with changes to the libraries that `llvm-config`
spits out, the dependencies to the LLVM are a dynamically generated rust file.
This file is now automatically updated whenever LLVM is updated to get kept
up-to-date.

At the same time, this cleans out some old cruft which isn't necessary in the
makefiles in terms of dependencies.

Closes #10745
Closes #10744

LLVM's JIT has been updated numerous times, and we haven't been tracking it at
all. The existing LLVM glue code no longer compiles, and the JIT isn't used for
anything currently.

This also rebases out the FixedStackSegment support which we have added to LLVM.
None of this is still in use by the compiler, and there's no need to keep this
functionality around inside of LLVM.

This is needed to unblock #10708 (where we're tripping an LLVM assertion).

This reverts commit c54427dd.

Leave the #[ignores] in that were added to rustpkg tests.

Conflicts:
	src/librustc/driver/driver.rs
	src/librustc/metadata/creader.rs

This function had type &[u8] -> ~str, i.e. it allocates a string
internally, even though the non-allocating version that take &[u8] ->
&str and ~[u8] -> ~str are all that is necessary in most circumstances.

This commit implements the support necessary for generating both intermediate
and result static rust libraries. This is an implementation of my thoughts in
https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html.

When compiling a library, we still retain the "lib" option, although now there
are "rlib", "staticlib", and "dylib" as options for crate_type (and these are
stackable). The idea of "lib" is to generate the "compiler default" instead of
having too choose (although all are interchangeable). For now I have left the
"complier default" to be a dynamic library for size reasons.

Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an
rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a
dynamic object. I chose this for size reasons, but also because you're probably
not going to be embedding the rustc compiler anywhere any time soon.

Other than the options outlined above, there are a few defaults/preferences that
are now opinionated in the compiler:

* If both a .dylib and .rlib are found for a rust library, the compiler will
  prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option
* If generating a "lib", the compiler will generate a dynamic library. This is
  overridable by explicitly saying what flavor you'd like (rlib, staticlib,
  dylib).
* If no options are passed to the command line, and no crate_type is found in
  the destination crate, then an executable is generated

With this change, you can successfully build a rust program with 0 dynamic
dependencies on rust libraries. There is still a dynamic dependency on
librustrt, but I plan on removing that in a subsequent commit.

This change includes no tests just yet. Our current testing
infrastructure/harnesses aren't very amenable to doing flavorful things with
linking, so I'm planning on adding a new mode of testing which I believe belongs
as a separate commit.

Closes #552

and librustpkg.

The reasons for doing this are:

* The model on which linked failure is based is inherently complex
* The implementation is also very complex, and there are few remaining who
  fully understand the implementation
* There are existing race conditions in the core context switching function of
  the scheduler, and possibly others.
* It's unclear whether this model of linked failure maps well to a 1:1 threading
  model

Linked failure is often a desired aspect of tasks, but we would like to take a
much more conservative approach in re-implementing linked failure if at all.

Closes #8674
Closes #8318
Closes #8863

These two attributes are no longer useful now that Rust has decided to leave
segmented stacks behind. It is assumed that the rust task's stack is always
large enough to make an FFI call (due to the stack being very large).

There's always the case of stack overflow, however, to consider. This does not
change the behavior of stack overflow in Rust. This is still normally triggered
by the __morestack function and aborts the whole process.

C stack overflow will continue to corrupt the stack, however (as it did before
this commit as well). The future improvement of a guard page at the end of every
rust stack is still unimplemented and is intended to be the mechanism through
which we attempt to detect C stack overflow.

Closes #8822
Closes #10155

New standards have arisen in recent months, mostly for the use of
rustpkg, but the main Rust codebase has not been altered to match these
new specifications. This changeset rectifies most of these issues.

- Renamed the crate source files `src/libX/X.rs` to `lib.rs`, for
  consistency with current styles; this affects extra, rustc, rustdoc,
  rustpkg, rustuv, std, syntax.

- Renamed `X/X.rs` to `X/mod.rs,` as is now recommended style, for
  `std::num` and `std::terminfo`.

- Shifted `src/libstd/str/ascii.rs` out of the otherwise unused `str`
  directory, to be consistent with its import path of `std::ascii`;
  libstd is flat at present so it's more appropriate thus.

While this removes some `#[path = "..."]` directives, it does not remove
all of them, and leaves certain other inconsistencies, such as `std::u8`
et al. which are actually stored in `src/libstd/num/` (one subdirectory
down). No quorum has been reached on this issue, so I felt it best to
leave them all alone at present. #9208 deals with the possibility of
making libstd more hierarchical (such as changing the crate to match the
current filesystem structure, which would make the module path
`std::num::u8`).

There is one thing remaining in which this repository is not
rustpkg-compliant: rustpkg would have `src/std/` et al. rather than
`src/libstd/` et al. I have not endeavoured to change that at this point
as it would guarantee prompt bitrot and confusion. A change of that
magnitude needs to be discussed first.

These methods are all excellent candidates for default methods, so there's no
need to require extra imports of various traits.

Big fish fried here:

    extra::json
    most of the compiler
    extra::io_util removed
    extra::fileinput removed

Fish left to fry

    extra::ebml

Who doesn't like a massive renaming?