CS 60 Computer Networks

Lecture 11

Simple Reliable Transport (SRT)

As part of cs60 we will build our very own communications stack called DartNet. The ideas is that protocols remain abstract but neat mechanisms but without hands on experience to reinforce the knowledge few skills are burnt in. We are challenging you to think about the design of a transport and network protocol which functionally are similar to TCP and IP protocols that drive the Internet. We cannot run our code on routers in the Internet so DartNet’s transport - called the Simple Reliable Transport (SRT) and network - called Simple Network Protocol (SNP) layers will run on an overlay that we will also construct. You have already came across overlay networks - P2P networks are overlay networks.

In this lecture we will discuss the designed design of SRT and will leave SNP for a future lecture. As we progress we will move down the DartNet stack.

Much of the information presented in this lecture is important for completing Lab4 - Implementation of the Simple Reliable Transport.

In this lab we will write a client and a server.


We plan to learn the following from these lectures:

OK, let’s get going.


DartNet Communications Stack

The DartNet communications stack shown below in Figure 1 allows clients and servers to create sockets, open connections between hosts for reliable delivery of data. DartNet is a layered architecture with each layer at n-1 offering layer n a service and API. For example, SRT offers applications the reliable delivery of byte streams. SNP offers the transport layer connectivity between end-systems similar to IP. The overlay layer sets up a network of end hosts that run the DartNet stack. We show for correctness that TCP/IP runs under the overlay. Put you can think of SNP+overlay as being the network layer and the lower layers (in this case TCP/IP) as part of the lower layers of the architecture akin to the data link and physical layers.


Figure 1: DartNet Communications Stack

SRT provides the reliable deliver of in order byte stream just like TCP. However, we would be here all year coding TCP (which would be fun granted) so SRT makes a number of assumptions to simplify the design and implementation of the protocol.

The following mechanisms are implemented as part of SRT:

In Lab4 we do not consider DATA segments. We only consider control (or signaling messaging) used by connection management.

APIs: SRT, SNP and Overlay Control

There are a number of APIs between the different layers of DartNet. In n Fig.2 we show the full stack. The SRT API offers the ClientApp and ServerApp a number of function calls - collectively called APIs. Over the course of the DartNet thematic labs you will implement all of these API functions. We discuss these APIs in more detail later in these notes.

For Lab4 we remove SNP to simplify the assignment and focus on the implementation of a restricted set of functions associated with the SRT API for the client and server respectively. We only consider function calls associated with set up and tearing down connections between the client and server. Note, SRT is capable of supporting multiple simultaneous connections like TCP. We will discuss the data structures and Finite State Machines (FSMs) need to support this later in this text. Also note that the SRT sits directly on the “thin” overlay layer when SNP is removed. The overlay in this case simply implements a TCP connection between the client and server SRT transport protocols. The overlay just contains a direct TCP connection between the client and the server. The overlay_start() and overlay_stop() functions create the drect TCP connection between the client and the server. overlay_start() should return the TCP socket descriptor. overlay_stop() closes the TCP connection. The code needed to do this is essentially the C code we developed for the socket programming assignment. Therefore, you already have 90% of the code for the overlay. For lab4, the API functions need to implement are shown in Fig.3. These are a subset of the DartNet API function calls.


Figure 2: The DartNet API Function Calls (Complete Set)

You will need to implement snp_sendseg() and snp_recvseg() function calls to send and receive segments using the overlay’s TCP connection. In the case of TCP data is sent as byte stream. Note, in order to send data segments over the overlay TCP connection, we need to add delimiters in the byte stream. In SRT uses the special characters “!&” to indicate the start of a segment and the special characters “!#” to indicate the end of a segment. Two notes on these delimiters. First, these characters should not occur in the data sent between the client and the service. This is a restriction (and the protocol would operate incorrectly if “!&” or “!#” were to occur in the byte stream from the client. So you should make sure your data does not include these pairs of special control characters. Second, delimiters are necessary because the receiving side of a STR packet (i.e., STR header + STR segment) needs to know when it successfully receives an SRT packet. This is one way to do it. Probably the simplest approach.

Because the overlay is built from TCP rather than say UDP we do not have any loss. To allow the overlay to look as if it runs across lossy links and that packets can be lost in the Internet we implement loss() function invoked in snp_recvseg() when an SRT packet is received to discard the segment with probability PROBABILITY_LOSS to simulate the packet loss to force SRT to recover the missing segments which can be SRT DATA packets, SYN, SYNACK, FIN, and FINACK. The SRT protocol must be able to recover correctly from loss of any of these packets.


Figure 3: The Lab4 API Function Calls to Implement

The flow of the functions are as follows. The app_client.c and app_server.c, the application layer code first start the overlay by creating a direct TCP link between the client and the server in overlay_start(). In Lab4 students implement overlay_start() in the app_client and app_server. The server initializes the SRT server by calling srt_svr_init() and the client initializes the SRT client by calling srt_client_init(). Then the server calls srt_svr_sock() to create a server side socket and srt_svr_accept() to accept a connection request from the client (i.e., control message SYN at the SRT layer). The client creates a socket and connects to the server socket by calling srt_client_sock() and srt_client_connect(), respectively. In Lab4 the client and server establish two connections on different ports to prove that the SRT can cope with multiple connections: the server creates another socket and waits for incoming connection, and the client creates another socket and connects to the new server socket. By doing this, we show that SRT can have multiple connections at the same time. In Lab4, the client disconnects from the server after a wait period by calling srt_client_disconnect() for each connection. Finally, the server closes the socket by calling srt_svr_close() and the client closes the socket by calling srt_client_close(). The app_client and app_server stop the overlay before terminating their processes by calling overlay_end() at the client and server. In Lab4 students implement not only overlay_stop() but all the SRT functions used in srt˙client.c and srt_server.c.

Data Structures

SRT defines a number of important data structures that implement the control and packetization of the data into segments.

An SRT packet comprises an SRT header (see srt_hdr_t below) and SRT segment (see seg_t - data bytes stream goes here). The SRT packet is encapsulated into a SNP packet which we will discuss when we get to the SNP lecture. Notice that the SRT header includes important information associate with the packet include for instance unsigned short int type;.

The type field in the packet header defines the type of packet:

#define SYN 0
#define SYNACK 1
#define FIN 2
#define FINACK 3
#define DATA 4
#define DATAACK 5

SRT protocol sends and receives data in segments. The segment header format is shown in Fig. 4 and the C language equivalent is shown below:


Figure 4: Segment Header Format

It is defined in seg.h:

typedef struct srt_hdr {
  unsigned int src_port;        //source port number
  unsigned int dest_port;       //destination port number
  unsigned int seq_num;         //sequence number
  unsigned short int length;    //segment data length
  unsigned short int  type;     //segment type
  unsigned short int  rcv_win;  //currently not used
  unsigned short int checksum;  //currently not used
} srt_hdr_t;

typedef struct segment {
  srt_hdr_t header;
  char data[MAX_SEG_LEN];
} seg_t;

The SRT maintains all the state associated with a connection using a Transport Control Block (TCB). For each connection the client and server side initialize and maintain a TCB. When a connection is closed the TCB is returned to the pool of unused TCB. The figure below shows a server side TCB maintained for each connection. A TCB table is initialized at the client and server on startup when srt_svr_init() and srt_client_init() are called by the server_App and client_App, respectively. Implementation tip: always use malloc()/free() for data structures and tables of data structures when coding the SRT.

typedef struct svr_tcb {
  unsigned int svr_nodeID;        //node ID of server, similar as IP address
  unsigned int svr_portNum;       //port number of server
  unsigned int client_nodeID;     //node ID of client, similar as IP address
  unsigned int client_portNum;    //port number of client
  unsigned int state;             //state of server
  unsigned int expect_seqNum;     //sequence number of the data seg server is expecting
  char* recvBuf;                  //a pointer pointing to a receive Buffer
  unsigned int  usedBufLen;       //size of received data in receiver buffer
  pthread_mutex_t* bufMutex;      //a pointer pointing to a mutex for receive buffer access
} svr_tcb_t;

For each connection, the SRT client maintains a client and serever TCB.

typedef struct client_tcb {
  unsigned int svr_nodeID;        //node ID of server, similar as IP address
  unsigned int svr_portNum;       //port number of server
  unsigned int client_nodeID;     //node ID of client, similar as IP address
  unsigned int client_portNum;    //port number of client
  unsigned int  state;            //state of client
  unsigned int next_seqNum;       //next sequence number
  pthread_mutex_t* bufMutex;      //a pointer pointing to a mutex for send buffer access
  segBuf_t* sendBufHead;          //used by send buffer
  segBuf_t* sendBufunSent;        //used by send buffer
  segBuf_t* sendBufTail;          //used by send buffer
  unsigned int unAck_segNum;      //number of unAcked segments
} client_tcb_t;

Connection Management

SRT has a similar setup and teardown of connections as TCP. However SRT simply uses a two-way handshake.

Connection Setup

A connection is established when the client sends a SYN and the server responds with a SYNACK as shown in shown in Fig. 5.


Figure 5: SRT Connection Setup

But life is not that simple: corruption and loss of control messages get in the way

In class today we discussed the following problems that occur with set up and tear down:

What happens in the SYN is lost or corrupt; client knows it sent it, server has no knowledge. So how is this operation issue solved? Furthermore, what happens if the server received the SYN, send the SYNACK but the SYNACK is lost. The server knows it sent an SYNACK and does not know that the client did not get it.

Solutions to these anomalies is as follows. In the case of the lost SYN the str_client needs to set a SYN_TIMEOUT and if it does not receive a SYN ACK in that period it sends another SYN and set the timer again. It repeats this until it either gets a SYNACK or the SYN_MAX_RETRY is exceeded. For example, the client sends a SYN and moves into a CONNECT state. In that state it is likely to receive SYNs. What should it do? Send a SYNACK and let the client move to CONNECT to. We discuss the correct state transitions later on client and server FSMs.

Connection Teardown

A connection is taken (tear) down when the client sends a FIN and the server responds accordingly with a FINACK as shown in 6


Figure 6: SRT Connection Teardown

But we can lose packets here. What happens in a FIN is lost? The client needs to send another FIN after FIN_TIMEOUT up to FIN_MAX_RETRY. What happens if the FINACK is lost? The client side has to wait for retransmissions of FIN_MAX_RETRY in an attempt to get a FINACK. If it doesn’t get the FINACK it transitioned to CLOSE state after a wait period and assumes the connection had time to close in an orderly manner.

Finite State Machines (FSMs)

The client and server side implement FSMs. FSMs are made up of a limited number of states. The transitions to a state is governed by the state machines and the EVENT/ACTION. Basically, in a state e.g., on the client FSM, e.g., assume it is in state = SYNSENT and the EVENT SYN_TIMEOUT occurs then the appropriate ACTION is send SYN. Check the client FSM to see if this is indeed the case.

Client-side FSM

The states maintained in the client TCB in the “state” element of the structure is as follows:

//states of client

#define CLOSED 1
#define SYNSENT 2
#define CONNECTED 3
#define FINWAIT 4

When implementing your protocol you will need to consult with the state machines shown here to determine that state transitions for the client and the server due to the exchange of segments during connection establishment and tear down. Here the SRT server state machine is shown in Fig. 8. SRT client state machine is shown in Fig. 7.


Figure 7: SRT Client FSM

Server-side FSM

The states maintained in the server TCB in the “state” element of the structure is as follows:

//states of server

#define CLOSED 1
#define LISTENING 2
#define CONNECTED 3
#define CLOSEWAIT 4


Figure 8: SRT Server FSM

Implementation Tip: Fired up, ready to go!

With the major data structures defined for the messages (SYN, SYNACK, FIN, FINACK) on the “wire” (i.e., to be sent to the overlay) and the internal TCB used to maintain the “state” information associated with connections (note, that the index into the TCB table is used as the sockfd for the connection on the client and server side); and the message interactions between the client and server defined for setup and teardown; and the detailed description of the client and server FSMs; then, you have most of the information to get rockin and implement the SRT. There is missing information that you can think about and design accordingly, but, this important information is sufficient to implement SRT.

Important Application, SRT and SNP Files

There are a number of files that show examples of the app_client.c and app_server.c that show the SRT Socket API. In addition, we provide a number of header files that define constants used by SRT that you should include in your code.

Clients application - examples of client using SRT Socket API.


Client side data structures and prototypes needed to implement the client side SRT protocol


Server application - examples of server side SRT Socket API.


Server side data structures and prototypes needed to implement the server side SRT protocol


SNP APIs - header file. SRT header and segement data structure, SNP prototypes.