* Make the roomserver output format more flexible

* Fix the sync server integration testdata

* Fix roomserver testdata

* Actually fix the sync server tests

* Fix typo

* Profile retrieval

* Saving avatar (without propagating it)

* Saving display name (without propagating it)

* Getters for display name and avatar URL

* Doc'd

* Introduced new Kafka topic and producer

* Updated config with new kafka topic

* Switched to samara producer and now sending messages

* Doc'd

* Put kafka update after the database insert

* Doc'd profileUpdate structure

* Removed unused parameter

* Moved user updates producer to clientapi/producers

* Add a component for sending event to remote matrix servers using federation

* Use named constants

* Comment on why we combine the deltas

* Comments

* Copyright headers!

* Fix the copy+Pasted comment

* Add tests for combineDeltas

* Review comments

* More review comments

* More comments

* Even more comments

* Name the mutexes

* Document what the mutexes are protecting

* Simplify the SQL, and more comments

* Set depth of events and whether they need to be federated.

Set the depth of each new event to one greater than the maximum depth
of it's direct ancestors.

Add a flag to each event passing through the roomserver that tells us
whether the event needs to be sent over federation.

We do this by passing the name of the server to send the event as.
We will need this capability if we want to support vhosting as it is
not possible to tell from the event alone which server to send it as.

(The reason for this is that sometimes a event needs to be sent on
behalf of a different remote matrix server)

* Update roomserver integration tests

* More comments

* Constants and comments

* Fix comments

* Document the int64 returned by LatestEventIDs

* Include the state before the event in roomserver output

* Fix roomserver integration test

* Remove VisibilityEventIDs from the JSON

* More comments

* Remove spurious TODO

* Move all the dendrite config in to a single place

* Add tests for config parsing

* replace syncserver config with common config

* Replace client API config with common config

* Replace federation API config with common config

* Replace media api config with common config

* Replace room server config with common config

* Remove unused readKey function

* Fix the integration tests

* Comment on hardcoding roomserver to HTTP

* Add a method for getting RoomServerURL

This moves the hardcoding of HTTPs into one place.

* Add a database for storing the server keys

* Tweak wording, and comment on the resolution of the timestamp

* Update gomatrixserverlib

* cmd/mediaapi-integration-tests: Add foundation for testing

* common/test: Add some server init and client request utilities

* common/test/client: Handle timed out requests for tests that passed

* cmd/syncserver-integration-tests: Port to new common/test infra

* common/test/client: Remove stray debug print

* cmd/mediaapi-integration-tests: Simplify slice initialisation

* cmd/mediaapi-integration-tests: Simplify getMediaURL argument

* cmd/mediaapi-integration-tests: Make startMediaAPI return listen address

* common/test/client: Fix uninitialised LastRequestErr

* common/test/server: Remove redundant argument

* common/test/server: Add StartProxy to create a reverse proxy

* cmd/mediaapi-integration-tests: Add proxies in front of servers

This is needed so that origins can be correctly configured and used for
remote media.

* travis: Enable media API integration tests

* travis: Build the client-api-proxy for media tests

* common/test/client: Don't panic on EOF in CanonicalJSONInput

* cmd/mediaapi-integration-tests: Add upload/download/thumbnail tests

* mediaapi/thumbnailer: Store thumbnail according to requested size

* cmd/mediaapi-integration-tests: Add totem.jpg test file

* cmd/client-api-proxy: Optionally listen for HTTPS

* common/test/client: Do not verify TLS certs for testing

We will commonly use self-signed certs.

* cmd/mediaapi-integration-tests: Make HTTPS requests

* cmd/mediaapi-integration-tests: Log size and method for thumbnails

* mediaapi/thumbnailer: Factor out isThumbnailExists

Appease gocyclo^w^w simplify

* mediaapi/thumbnailer: Check if request is larger than original

* travis: Install openssl and generate server.{crt,key}

* cmd/mediaapi-integration-tests: Add valid dynamic thumbnail test

* cmd/mediaapi-integration-tests: Document state of tests

* cmd/mediaapi-integration-tests: Test remote thumbnail before download

This ordering also exercises the cold cache immediate generation of a
size configured for pregeneration.

* travis: Explain openssl key+cert generation

* common/test/server: Clarify postgresContainerName

* Add API for querying events by ID.

* Fix tense

* Start implementing federation ingress

* More stuff

* Hook up federation event receiving

* Handle the case where we are missing state

* Fix docstring and comments

* Fix infinite loop when printing unknownRoomError

* vendor: Add bimg image processing library

bimg is MIT licensed. It depends on the C library libvips which is LGPL
v2.1+ licensed. libvips must be installed separately.

* mediaapi: Add YAML config file support

* mediaapi: Add thumbnail support

* mediaapi: Add missing thumbnail files

* travis: Add ppa and install libvips-dev

* travis: Another ppa and install libvips-dev attempt

* travis: Add sudo: required for sudo apt* usage

* mediaapi/thumbnailer: Make comparison code more readable

* mediaapi: Simplify logging of thumbnail properties

* mediaapi/thumbnailer: Rename metrics to fitness

Metrics is used in the context of monitoring with Prometheus so renaming
to avoid confusion.

* mediaapi/thumbnailer: Use math.Inf() for max aspect and size

* mediaapi/thumbnailer: Limit number of parallel generators

Fall back to selecting from already-/pre-generated thumbnails or serving
the original.

* mediaapi/thumbnailer: Split bimg code into separate file

* vendor: Add github.com/nfnt/resize pure go image scaler

* mediaapi/thumbnailer: Add nfnt/resize thumbnailer

* travis: Don't install libvips-dev via ppa

* mediaapi: Add notes to README about resizers

* mediaapi: Elaborate on scaling libs in README

This reverts commit ce82158a.

* Add API for querying events by ID.

* Fix tense

* Start implementing federation ingress

* More stuff

* Hook up federation event receiving

* Fix comments

* Comment on the order of the arrays

* Start implementing the join room API

* Hacks to get join room working

* Make the TLS fingerprint configurable

* Fix the client API proxy to handle '#' correctly

* Return a 200 OK response

* Write the join event along with current state to the room server

* Comment on the error handling

* Fix typos

* Fix tab

* Add TODO for moving authEventIDs to gomatrixserverlib

* Comment on why we ignore the key ID argument for local keys

* Avoid 'preceeded'

* Neaten the control flow

* Neaten the control flow even more

* Return pointers

* Rename err to lastErr for the loop

- Renamed `clientapi/auth/types` to `clientapi/auth/authtypes` for the same
  horrible namespace clashing reasons as `storage`.
- Factored out `makeAPI` to `common`.
- Added in `makeAuthAPI`.

Previously, all database stuff was under the helpfully named
package 'storage'. However, this convention is used throughout all
of dendrite, which will clash the moment we want to add auth to all
the CS API endpoints. To prevent the package name clash, add
sub-directories which represent what is being stored so the final
usage ends up being:

```
func doThing(db *storage.SyncServerDatabase, authDB *accounts.Database)
{
    // ...
}
```

* Start implementing the federation server keys API.

* Fix copyright

* Fix comments

* Comment on the key format

* Better explain what the ValidityPeriod is

* Return a 200 status code

- `s/Server/OutputRoomEvent/` in `consumers` to accurately reflect what is being consumed.
- `s/set/userIDSet/` in `notifier.go` for clarity.
- Removed lying comments.

* Add test for transition to leave

* Add test for join/leave in the same /sync response

This is only 'mostly' correct currently, because what should be no-op dupe
joins will actually trigger the entire room state to be re-sent.

Bizarrely, it's significantly easier to just do that than work out if we should,
and there are no client-visible effects to doing so, so we just do it for now.

Fixed exactly the same bug as before, but this time for incremental
sync. Yay tests!

- Test data for the sync server is now in its own file.
- Rejig the sync server tests to support multiple /sync requests and corresponding
  assertions.
- Fixed a minor bug which resulted in state events to appear twice in /sync
  responses when syncing without a `since` parameter.

* Split out notifying /sync requests and calculating sync responses

The logic for notifying /sync requests is about to get really
complicated as we optimise when to wake up requests, so split
out that code into a separate struct to isolate it and make
it easier to unit test.