TCP/IP Network for Programmers: Part One
[This Article was originally published for the first time at https://molo17.com/2019/05/tcp-ip-network-for-programmers-part-one/ ]
The first thing I want to let you know is that for years I have been dealing with game development, rather than business software. However, over the years, the multiplayer aspect has become more and more preponderant and so I have developed a strong competence in the field of networks. I hope you will excuse me if in some of the examples, I will mention game consoles or if I define the user as a player. In the end, even consoles are computers and everything I am going to write about can be easily be applied to different areas.
Origins of the Internet
The Internet as we know it today began in 1969 as a four-node network. Formerly known as ARPANET, it was developed by the USARPA agency (the United States Advanced Research Projects Agency) with the stated goal of providing geographically distant scientists with access to very powerful computers, which are also geographically dispersed throughout the world.
This is what we can read in any school book, what we never realize is that the real technological breakthrough was moving from a connection logic between networks based on “circuit switching” to “packet switching”. All systems up until then, in fact, used long-distance connections through circuit changes, which effectively prevented multiple communications from passing through the same circuit at the same time. So a line between New York and Los Angeles passing through Chicago and Denver could be blocked by any communication even between Chicago and Denver.
With the advent of “packet switching”, this was no longer a problem, each node of the network could decide which next node to pass the packet with the data and always keep the communication active and allowing more communications to take place simultaneously.
ARPANET describes very well the first implementations of the network connection and management protocols in its BBN 1822 report (https://tools.ietf.org/html/rfc878), now known only as 1822 report. All the indications and procedures described in it were the basis for the collection of protocols known as TCP / IP Suite.
TCP / IP Layer Cake
The TCP / IP suite has always been considered both beautiful and scary. Beautiful because the layers of which it is composed offer a great variety of interchangeable protocols and guarantee proper management and reliability. Frightening because often the basic implementations are violated to ensure higher performance or more complex motivations.
As programmers, our task is to understand both the advantages and the disadvantages of the TCP / IP suite and use it practically and efficiently. Usually, we will always only touch the last layer of the TCP / IP stack, but we will also need to know the lower levels to be able to understand all the problems.
There are, of course, also a whole series of other models that explain the iteration of internet communications. RFC 1122, for example, an old essential requirement of web hosts uses only four layers: the link layer, the IP layer, the transport layer and the application layer. The alternative model to TCP/IP known as OSI (Open Systems Interconnection) uses seven layers instead: the physical layer, the data link layer, the network layer, the transport layer, the session layer, the presentation layer and the application layer. In this series of articles, we will focus on the TCP/IP Layer Cake.
Each layer in the diagram above supports the overlying layers and has a series of tasks:
- Accepting data blocks to be transmitted from a higher level
- Packaging data with a header and sometimes a footer
- Sending data to a lower layer for other transmissions
- Receiving data transmitted from a lower layer
- Unpackaging data transmitted by removing the header
- Transmitting data to a higher layer for data processing
The way each layer deals with these tasks is not written in stone. In reality, for each layer, different protocols can be used, and many others are always invented. In more object-oriented terms, it can be useful to imagine the different layers as interfaces and each protocol as a collection of protocols that implements the interfaces.
The Physical Layer
At the lowest level of the diagram lies the most rudimentary support: The physical layer. The task of this layer is to provide for the physical connection between the different network connection elements: cables, telephone lines, radio systems or even new methods.
The Link Layer
The Link Layer is the point at which we move from the physical part to the computer science part. In the Link Layer, every single transmission unit is known as Frame. The layer has specific tasks such as sending frames between different hosts, ensuring their size and physically identifying the different hosts.
It should be noted that the delivery of each Frame is never guaranteed but only very likely. The link layer does not provide any security system capable to guarantee the arrival of the data and to keep their contents.
Each different physical medium implements a different link layer protocol. For example, we have Ethernet 10BASET for network cables or EoC (Ethernet over Copper) or the different variants of 802.3 for radio waves.
It is crucial to keep in mind that any connection between two distant hosts is never based on a single Link Layer since each connection can pass invisibly between one type of protocol to another in an ideal way.
Ethernet is not a single protocol. However, it’s a group of protocols originated from the Ethernet Blue Book Standard published in 1980 by Dec, Intel and Xerox (https://tinyurl.com/y7mjvyad). Currently, the protocol standard is defined under the IEEE 802.3 standard. There are several variants, each developed to support a different physical layer.
To assign an identity to each host, Ethernet introduces the idea of a MAC address. A MAC address is a unique 48-bit number assigned to every piece of hardware that can connect to the Ethernet network. Usually, this hardware is referred to as a network interface controller or NIC. To keep the MAC address unique, the NIC is focused on the hardware at the time of production. In theory, all this should guarantee with certainty the uniqueness of the MAC address, but in recent times this certainty has become unreliable. Many network interfaces, in fact, now allow you to change the MAC address through the software. Thus 64bit addresses have been introduced through a known EUI64 standard. Each 64bit Mac address can be converted to a EUI64 by merely adding 2 FFFE bytes after the first 24 bits. The first 24 bits are decided by the IEEE and inserted by the hardware manufacturer and are defined as OUI (Organizational Unique Identifier)
This scheme represents the structure of an Ethernet Frame data packet.
The Preamble and the start frame delimiter (SFD) consist of hex bytes 0x55 0x55 0x55 0x55 0x55 0x55 0x55 0xD5, they are the same for each packet. This binary pattern helps the hardware to synchronize and prepare the sending and receiving of every single Ethernet Frame. Both the preamble and the SFD are then removed directly from the hardware, and the remaining bytes within the Frame are passed to the Ethernet.
After the SFD there are 6 bytes that contain the destination MAC address and then the 6 bytes with the source address. There is a unique exclusive MAC address that corresponds to FF: FF: FF: FF: FF: FF, it is known as broadcast address and allows sending the same Frame to all hosts in the same LAN.
The Length / Type field can contain either the length or the type of data contained in the Frame. If it contains the type, an EtherType value is encoded (a detailed list is available here: https://en.wikipedia.org/wiki/EtherType). When the Ethernet receives a frame with a coded type, it can define different methods to interpret or manage the payload.
To assist with the interpretation of the data and their transmission, the Ethernet standard defines a maximum length of transmission of the payload attested at 1500 bytes, known as maximum transmission unit or MTU.
The Ethernet standard also defines that the minimum value of EtherType is 0x0600, which is 1536. If the Length / Type field is less than 1500, the value represents the length of the payload if instead it is higher than 1536 represents a type of data encoded in an EtherType.
Although not standard, many Ethernet NICs support frames with MTU higher than 1500 bytes. These frames also called “Jumbo Frames” can have a large payload up to 9000 bytes. To guarantee the compatibility of these frames, a particular Ethertype is used both in the header and in the Length / Type, and subsequently, the receiving NIC is entrusted with the possibility of converting it into smaller frames.
The Frame Check Sequence (FCS) contains a cyclic redundancy check (CRC32) generated by the union of the Length / Type with the payload. In this way, when the data is read, it can be checked quickly for any data loss or corruption, even if Ethernet does not guarantee the arrival of the data and their reliability.
For now, we end here, in the next article, we will dedicate ourselves to the Network Layer and the Transport Layer.