Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
FinClip
ligase
提交
c78d9a59
ligase
项目概览
FinClip
/
ligase
通知
33
Star
8
Fork
1
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
ligase
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
提交
c78d9a59
编写于
10月 10, 2017
作者:
E
Erik Johnston
提交者:
GitHub
10月 10, 2017
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Clean up README a bit (#289)
上级
13107c6b
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
151 addition
and
140 deletion
+151
-140
DESIGN.md
DESIGN.md
+140
-0
README.md
README.md
+11
-140
未找到文件。
DESIGN.md
0 → 100644
浏览文件 @
c78d9a59
# Design
## Log Based Architecture
### Decomposition and Decoupling
A matrix homeserver can be built around append-only event logs built from the
messages, receipts, presence, typing notifications, device messages and other
events sent by users on the homeservers or by other homeservers.
The server would then decompose into two categories: writers that add new
entries to the logs and readers that read those entries.
The event logs then serve to decouple the two components, the writers and
readers need only agree on the format of the entries in the event log.
This format could be largely derived from the wire format of the events used
in the client and federation protocols:
C-S API +---------+ Event Log +---------+ C-S API
---------> | |+ (e.g. kafka) | |+ --------->
| Writers || =============> | Readers ||
---------> | || | || --------->
S-S API +---------+| +---------+| S-S API
+---------+ +---------+
However the way matrix handles state events in a room creates a few
complications for this model.
1) Writers require the room state at an event to check if it is allowed.
2) Readers require the room state at an event to determine the users and
servers that are allowed to see the event.
3) A client can query the current state of the room from a reader.
The writers and readers cannot extract the necessary information directly from
the event logs because it would take too long to extract the information as the
state is built up by collecting individual state events from the event history.
The writers and readers therefore need access to something that stores copies
of the event state in a form that can be efficiently queried. One possibility
would be for the readers and writers to maintain copies of the current state
in local databases. A second possibility would be to add a dedicated component
that maintained the state of the room and exposed an API that the readers and
writers could query to get the state. The second has the advantage that the
state is calculated and stored in a single location.
C-S API +---------+ Log +--------+ Log +---------+ C-S API
---------> | |+ ======> | | ======> | |+ --------->
| Writers || | Room | | Readers ||
---------> | || <------ | Server | ------> | || --------->
S-S API +---------+| Query | | Query +---------+| S-S API
+---------+ +--------+ +---------+
The room server can annotate the events it logs to the readers with room state
so that the readers can avoid querying the room server unnecessarily.
[
This architecture can be extended to cover most of the APIs.
](
WIRING.md
)
## How things are supposed to work.
### Local client sends an event in an existing room.
0) The client sends a PUT
`/_matrix/client/r0/rooms/{roomId}/send`
request
and an HTTP loadbalancer routes the request to a ClientAPI.
1) The ClientAPI:
* Authenticates the local user using the `access_token` sent in the HTTP
request.
* Checks if it has already processed or is processing a request with the
same `txnID`.
* Calculates which state events are needed to auth the request.
* Queries the necessary state events and the latest events in the room
from the RoomServer.
* Confirms that the room exists and checks whether the event is allowed by
the auth checks.
* Builds and signs the events.
* Writes the event to a "InputRoomEvent" kafka topic.
* Send a `200 OK` response to the client.
2) The RoomServer reads the event from "InputRoomEvent" kafka topic:
* Checks if it has already has a copy of the event.
* Checks if the event is allowed by the auth checks using the auth events
at the event.
* Calculates the room state at the event.
* Works out what the latest events in the room after processing this event
are.
* Calculate how the changes in the latest events affect the current state
of the room.
* TODO: Workout what events determine the visibility of this event to other
users
* Writes the event along with the changes in current state to an
"OutputRoomEvent" kafka topic. It writes all the events for a room to
the same kafka partition.
3a) The ClientSync reads the event from the "OutputRoomEvent" kafka topic:
* Updates its copy of the current state for the room.
* Works out which users need to be notified about the event.
* Wakes up any pending `/_matrix/client/r0/sync` requests for those users.
* Adds the event to the recent timeline events for the room.
3b) The FederationSender reads the event from the "OutputRoomEvent" kafka topic:
* Updates its copy of the current state for the room.
* Works out which remote servers need to be notified about the event.
* Sends a `/_matrix/federation/v1/send` request to those servers.
* Or if there is a request in progress then add the event to a queue to be
sent when the previous request finishes.
### Remote server sends an event in an existing room.
0) The remote server sends a
`PUT /_matrix/federation/v1/send`
request and an
HTTP loadbalancer routes the request to a FederationReceiver.
1) The FederationReceiver:
* Authenticates the remote server using the "X-Matrix" authorisation header.
* Checks if it has already processed or is processing a request with the
same `txnID`.
* Checks the signatures for the events.
Fetches the ed25519 keys for the event senders if necessary.
* Queries the RoomServer for a copy of the state of the room at each event.
* If the RoomServer doesn't know the state of the room at an event then
query the state of the room at the event from the remote server using
`GET /_matrix/federation/v1/state_ids` falling back to
`GET /_matrix/federation/v1/state` if necessary.
* Once the state at each event is known check whether the events are
allowed by the auth checks against the state at each event.
* For each event that is allowed write the event to the "InputRoomEvent"
kafka topic.
* Send a 200 OK response to the remote server listing which events were
successfully processed and which events failed
2) The RoomServer processes the event the same as it would a local event.
3a) The ClientSync processes the event the same as it would a local event.
README.md
浏览文件 @
c78d9a59
...
...
@@ -2,153 +2,24 @@
Dendrite will be a matrix homeserver written in go.
# Install
It's still very much a work in progress, but installation instructions can
be found in
[
INSTALL.md
](
INSTALL.md
)
Dendrite is still very much a work in progress, but those wishing to work on it
may be interested in the installation instructions in
[
INSTALL.md
](
INSTALL.md
)
.
An overview of the design can be found in
[
DESIGN.md
](
DESIGN.md
)
#
Design
#
Contributing
## Log Based Architecture
Everyone is welcome to help out and contribute! See
[
CONTRIBUTING.md
](
CONTRIBUTING.md
)
to get started!
### Decomposition and Decoupling
We aim to try and make it as easy as possible to jump in.
A matrix homeserver can be built around append-only event logs built from the
messages, receipts, presence, typing notifications, device messages and other
events sent by users on the homeservers or by other homeservers.
# Discussion
The server would then decompose into two categories: writers that add new
entries to the logs and readers that read those entries
.
For questions about Dendrite we have a dedicated room on Matrix
[
#dendrite:matrix.org
](
https://riot.im/develop/#/room/#dendrite:matrix.org
)
.
The event logs then serve to decouple the two components, the writers and
readers need only agree on the format of the entries in the event log.
This format could be largely derived from the wire format of the events used
in the client and federation protocols:
C-S API +---------+ Event Log +---------+ C-S API
---------> | |+ (e.g. kafka) | |+ --------->
| Writers || =============> | Readers ||
---------> | || | || --------->
S-S API +---------+| +---------+| S-S API
+---------+ +---------+
However the way matrix handles state events in a room creates a few
complications for this model.
1) Writers require the room state at an event to check if it is allowed.
2) Readers require the room state at an event to determine the users and
servers that are allowed to see the event.
3) A client can query the current state of the room from a reader.
The writers and readers cannot extract the necessary information directly from
the event logs because it would take too long to extract the information as the
state is built up by collecting individual state events from the event history.
The writers and readers therefore need access to something that stores copies
of the event state in a form that can be efficiently queried. One possibility
would be for the readers and writers to maintain copies of the current state
in local databases. A second possibility would be to add a dedicated component
that maintained the state of the room and exposed an API that the readers and
writers could query to get the state. The second has the advantage that the
state is calculated and stored in a single location.
C-S API +---------+ Log +--------+ Log +---------+ C-S API
---------> | |+ ======> | | ======> | |+ --------->
| Writers || | Room | | Readers ||
---------> | || <------ | Server | ------> | || --------->
S-S API +---------+| Query | | Query +---------+| S-S API
+---------+ +--------+ +---------+
The room server can annotate the events it logs to the readers with room state
so that the readers can avoid querying the room server unnecessarily.
[
This architecture can be extended to cover most of the APIs.
](
WIRING.md
)
## How things are supposed to work.
### Local client sends an event in an existing room.
0) The client sends a PUT
`/_matrix/client/r0/rooms/{roomId}/send`
request
and an HTTP loadbalancer routes the request to a ClientAPI.
1) The ClientAPI:
* Authenticates the local user using the `access_token` sent in the HTTP
request.
* Checks if it has already processed or is processing a request with the
same `txnID`.
* Calculates which state events are needed to auth the request.
* Queries the necessary state events and the latest events in the room
from the RoomServer.
* Confirms that the room exists and checks whether the event is allowed by
the auth checks.
* Builds and signs the events.
* Writes the event to a "InputRoomEvent" kafka topic.
* Send a `200 OK` response to the client.
2) The RoomServer reads the event from "InputRoomEvent" kafka topic:
* Checks if it has already has a copy of the event.
* Checks if the event is allowed by the auth checks using the auth events
at the event.
* Calculates the room state at the event.
* Works out what the latest events in the room after processing this event
are.
* Calculate how the changes in the latest events affect the current state
of the room.
* TODO: Workout what events determine the visibility of this event to other
users
* Writes the event along with the changes in current state to an
"OutputRoomEvent" kafka topic. It writes all the events for a room to
the same kafka partition.
3a) The ClientSync reads the event from the "OutputRoomEvent" kafka topic:
* Updates its copy of the current state for the room.
* Works out which users need to be notified about the event.
* Wakes up any pending `/_matrix/client/r0/sync` requests for those users.
* Adds the event to the recent timeline events for the room.
3b) The FederationSender reads the event from the "OutputRoomEvent" kafka topic:
* Updates its copy of the current state for the room.
* Works out which remote servers need to be notified about the event.
* Sends a `/_matrix/federation/v1/send` request to those servers.
* Or if there is a request in progress then add the event to a queue to be
sent when the previous request finishes.
### Remote server sends an event in an existing room.
0) The remote server sends a
`PUT /_matrix/federation/v1/send`
request and an
HTTP loadbalancer routes the request to a FederationReceiver.
1) The FederationReceiver:
* Authenticates the remote server using the "X-Matrix" authorisation header.
* Checks if it has already processed or is processing a request with the
same `txnID`.
* Checks the signatures for the events.
Fetches the ed25519 keys for the event senders if necessary.
* Queries the RoomServer for a copy of the state of the room at each event.
* If the RoomServer doesn't know the state of the room at an event then
query the state of the room at the event from the remote server using
`GET /_matrix/federation/v1/state_ids` falling back to
`GET /_matrix/federation/v1/state` if necessary.
* Once the state at each event is known check whether the events are
allowed by the auth checks against the state at each event.
* For each event that is allowed write the event to the "InputRoomEvent"
kafka topic.
* Send a 200 OK response to the remote server listing which events were
successfully processed and which events failed
2) The RoomServer processes the event the same as it would a local event.
3a) The ClientSync processes the event the same as it would a local event.
# TODO
# Progress
There's plenty still to do to make Dendrite usable! We're tracking progress in
a
[
spreadsheet
](
https://docs.google.com/spreadsheets/d/1tkMNpIpPjvuDJWjPFbw_xzNzOHBA-Hp50Rkpcr43xTw
)
.
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录