What is the transport layer in Network Security?

The transport layer is the second layer in the TCP/IP model. It is directly beneath the application layer and provides services to it. For the applications, it seems as if the two hosts are connected directly, however, they may actually be separated by a plethora of routers and other types of links.

Transport layer protocols have nothing to do with the actual network system, but they are implemented in the end systems and provide process-to-process communication. There may be many processes running on one host system, but the transport layer ensures communication between the right processes.

Transport layer protocols

Two major protocols of the transport layer are:

• TCP
• UDP

TCP

TCP is a connection-oriented protocol that ensures data reliability and no packet losses. It uses a three-way handshake to first establish the connection between hosts before transmitting any data. It is used in critical applications where packet loss cannot be tolerated.

UDP

UDP is a connectionless protocol that does not ensure that all the data sent by the sender will be received. It is faster than TCP and is used for video and audio transmission because some data loss can be tolerable.

Fundamental functions of the transport layer

1) Connection between hosts

It is responsible for end-to-end connection and data transfer between two different end systems. It relies on the lower layers to achieve this.

2) Multiplexing and Demultiplexing

To allow process-to-process communication, the transport layer uses socketsCombination of IP address and port number. One port number is assigned to a running process in a host system. These port numbers are used to distinguish the different processes that may be running at the same host.

When multiple applications need to send data from one host, this data is collected and encapsulated into transport layer segments with information related to the host and process that they are being sent to. This is called multiplexing – the information in the transport layer headers is used by the receiver for demultiplexing.

Demultiplexing refers to delivering the received data to the correct process in a host.

3) Error detection

During transmission, it is possible that the data may get corrupted. Data integrity is checked by the transport layer by using different mechanisms such as a checksum.

Moreover, in TCP, acknowledgments and negative acknowledgments are sent by the receiver back to the sender to indicate whether or not the correct(or any) data was received.

4) Reliable data transfer

TCP provides additional reliability services on top of the ones provided by the network layer. It ensures that there are no errors in the data, no losses, no duplicates, and that all the data chunks are received and put together in order.

5) Congestion and flow control

When there is a lot of traffic in the network, packets start to get dropped by the routers because their limit is exceeded. This results in retransmission of the lost data which increases the congestion even more. TCP implements congestion control using an algorithm that has the following parts:

• Slow Start
• Congestion Avoidance
• Fast Recovery

Another problem solved by TCP is that of flow control. A receiver may be overwhelmed if a lot of data is received at the same time and its capacity is reached. TCP ensures that the sender does not transmit more data than the receiver’s capacity.

Security threats

• Fingerprinting: A TCP segment may be sent to a system and the response analyzed to find out the host’s OS, network configuration, hardware devices, and more information. This can be the first step of a bigger attack.
• Downgrade Attack: The host is forced to employ an older, weaker method of data transmission that is actually meant for backward compatibility.
• TCP/UDP Flooding Attack: A kind of denial of service attack. The attacker overwhelms the victim by sending a huge amount of pings.
• TCP Sequence Prediction Attack: TCP uses sequence numbers to ensure that the packets are received in order. An attacker may observe the data being transmitted to a system, predict the next sequence number that would be sent to the victim, spoof the IP of the sender, and send malicious data to the receiver with the predicted sequence number so that the victim trusts this source.