Network

Socket Server

class paddle::SocketServer

class for holding all parameters processing for current port

Note

each parameter server inherits from one socket server, each server contains serveral woker threads which are to parallelize the processing of computation, but share some common datas stored in child class of socketserver.

Inherits from paddle::Thread

Subclassed by paddle::ProtoServer

Public Types

typedef std::function<void(const std::vector<iovec>& outputIovs)> ResponseCallback

Public Functions

SocketServer(const std::string &addr, int port, int rdmaCpu)

class constructor for SocketServer

Note

start one socket server which hosts parameter server process. rdmaCpu is passed to rdma deamon for better performance, and start tcp socket instead of rdma socket if rdmaCpu is equal to -1. Each trainer process starts one connection to one socket server, and use ports_num to build more connections to harness fat communication channel if necessary. each connection is controlled by single thread with blocking read and write.

Parameters

• addr -

  sock bind address

• port -

  sock bind port

• rdmaCpu -

  rdma sock bind cpu core

~SocketServer()

virtual void run()

start a socket server

Note

framework for starting socket server

Protected Functions

virtual void handleRequest(std::unique_ptr<MsgReader> msgReader, ResponseCallback callback) = 0

std::unique_ptr<SocketChannel> createChannel(int sock, const std::string &peerName)

std::unique_ptr<SocketChannel> createChannel(struct sxi_sock *sock, const std::string &peerName)

Protected Attributes

ChannelType tcpRdma_

int rdmaCpu_

std::string rdmaUri_

sxi_socket *rdmaSocket_

int port_

std::string addr_

int socket_

int maxPendingConnections_

bool stopping_

Private Functions

void rdmaServer()

start one rdma server which hosts parameter server

Note

do rdma bind and listen, which calling self-defined socket like rdma library. it will spawn one thread for each connection

void tcpServer()

start one tcp server which hosts parameter server

Note

do tcp socket bind and listen. it will spawn one thread for each connection

void detach()

Friends

friend paddle::SocketServer::SocketWorker

Socket Worker

class paddle::SocketWorker

class for holding one connection from one trainer

Note

all parameter processing will run in the context of this worker

Inherits from paddle::Thread

Public Functions

SocketWorker(std::unique_ptr<SocketChannel> &&channel, SocketServer *server)

virtual ~SocketWorker()

virtual void run()

worker thread main context

Note

each connection from client(trainer) is controlled by single worker thread, which is for handling all parameter server requests

Protected Attributes

std::unique_ptr<SocketChannel> channel_

SocketServer *server_

ChannelType tcpRdma_

Socket Client

class paddle::SocketClient

management for client connection which are from trainers

Note

it contains one channel descriptor which used to write and read data

Subclassed by paddle::ProtoClient

Public Functions

paddle::SocketClient::SocketClient(const std::string & serverAddr, int serverPort, enum ChannelType channelType)

class constructor

Note

responsible for building one connection to specified pserver port

Parameters

• serverAddr -

  pserver ip address

• serverPort -

  pserver port

• ChannelType -

  F_TCP or F_RDMA

SocketChannel *getChannel()

Protected Attributes

std::unique_ptr<SocketChannel> channel_

struct sxi_socket *socketDaemon_

ChannelType tcpRdma_

Private Functions

void RdmaClient(const std::string &serverAddr, int serverPort)

start one RDMA connection to rdma server

Note

each object contains one channel which accept byte stream for rdma, low level sock also provide byte stream api.

Parameters

• serverAddr -

  rdma server ip

• serverPort -

  rdma server port

void TcpClient(const std::string &serverAddr, int serverPort)

start one tcp connection to tcp server

Note

each object contains one channel which accept byte stream

Parameters

• serverAddr -

  tcp server ip

• serverPort -

  tcp server port

Socket Channel

class paddle::SocketChannel

APIs for reading and writing byte stream data or naive iov data from the APIs both RDMA and TCP exhibits byte stream style

Public Functions

SocketChannel(int socket, const std::string &peerName)

SocketChannel(struct sxi_sock *socket, const std::string &peerName)

~SocketChannel()

const std::string &getPeerName() const

size_t read(void *buf, size_t size)

read size bytes.

Note

keep reading until getting size bytes or sock is closed is closed

size_t write(const void *buf, size_t size)

write size bytes.

Note

keep writing until writing size bytes or sock is closed

size_t writev(const std::vector<struct iovec> &iov)

write a set of buffers.

rdma::readv and rdma::writev can take advantage of RDMA blocking offload transfering

Note

keep writing until all buffers are written or sock is closed

size_t readv(std::vector<struct iovec> *iov)

read a set of buffers.

Note

keep reading until all buffers are full or sock is closed.

void writeMessage(const std::vector<struct iovec> &iov)

write a set of buffers.

Note

keep writing until all buffers are passed or sock is closed

std::unique_ptr<MsgReader> readMessage()

return null to indicate socket is closed

Protected Attributes

int tcpSocket_

struct sxi_sock *rdmaSocket_

const std::string peerName_

ChannelType tcpRdma_

struct MessageHeader

Public Members

int64_t totalLength

int64_t numIovs

include the header

int64_t iovLengths[0]

Message Reader

class paddle::MsgReader

reading a set of blocks of data from SocketChannel.

Public Functions

MsgReader(SocketChannel *channel, size_t numIovs)

~MsgReader()

size_t getNumBlocks() const

number of remaining parts

size_t getNextBlockLength() const

lenght of next block

size_t getTotalLength() const

get the total length of all the remaining blocks

size_t getBlockLength(size_t i) const

Get the length for block currentBlockIndex + i.

void readBlocks(const std::vector<void *> &bufs)

read blocks data and store it to buf

void readNextBlock(void *buf)

Protected Attributes

SocketChannel *channel_

std::vector<size_t> blockLengths_

size_t currentBlockIndex_