How computers communicate over the internet: An introduction to TCP/IP

Sending Data Over the Internet #

When you send an email, watch a video on YouTube, or browse the web, you are sending or recieving data from your computer to another computer. The internet is a network of computers that are connected to each other.

Data in computer system at the smallest level is represented in binary form, which is a series of 0s and 1s. These binary digits need to be sent from once device to another.

Think of what is involved in sending this data.

• The target computer needs to know when the data has started and when it has ended
• The data needs to be sent in a way that the target computer can understand
• Deal with the fact that the data may be lost or corrupted during transmission
• If the target computer is halfway across the world, the data needs to be routed through multiple computers to reach its destination

And so on…

Coding all of this into a program is a lot of work. Even if you could do it, you would have ensure that every computer that is using your solution, is running the your code. This is where the TCP/IP protocol comes in.

What is TCP/IP? #

TCP/IP is the ‘common code’ used on the internet and every device that connects to the internet, uses this code. TCP/IP is a set of rules that govern how data is sent and received over the internet. It is made up of two main protocols (standards): Transmission Control Protocol (TCP) and Internet Protocol (IP).

What this means for the developers, is that they do not have to worry about the low-level details of sending data over the internet. They can use libraries that implement the TCP/IP protocols, leaving the developers to focus on the business logic of their applications.

But the TCP/IP protocol is not the only protocol or standard used on the internet. It does not know how to physically send data over the wire, it does not know how a web server should respond to a request, and so on. There are other protocols that are used in conjunction with TCP/IP to make the internet work.

Layering the Communication Stack #

TCP/IP is part of a software/hardware stack. The TCP/IP designers, developers may not have a full understanding of the lower-level details of the hardware that the data is being sent over. The network card in your computer, the cables that connect the computers, the routers that route the data, and so on.

So the communication stack is divided into layers. Each layer is responsible for a specific task, and each layer builds on top of the previous layer. Experts in each layer can focus on their specific area of expertise, without having to worry about the details of the other layers.

The lower layers are responsible for the physical transmission of data, over the wire or airwaves. The middle layers are responsible for routing the data to the correct destination, while the higher layers are responsible for the application-level tasks such as sending an email or browsing the web.

Historically, the stack is divided into 7 layers, with each layer building on top of the previous layer. This is known as the OSI model.

The TCP/IP stack is a simplified version of this model, with only 4 layers. It is more commonly used in practice as majority of the internet is built on top of the TCP/IP protocol.

The OSI Model #

The 7 layers of the OSI model are:

The TCP/IP Model #

The TCP/IP model is composed of 4 layers:

The Connection Parameters #

Before data can be sent from one host (e.g. computer, phone) to another host, a connection needs to be established between the two hosts.

This connection is made up of two parts: the IP address and the port number.

The IP address is a unique identifier for each device on the network, and the port number is a unique identifier for each application running on the target device. Together, the IP address and port number form a unique address for the target computer.

When you send data to a computer, you specify the IP address and port number of the target computer. The data is then sent to the target computer, which listens for incoming data on the specified port number.

As an analogy, you can think of the IP address as the street address of a house. When posting a letter to a house you would put the street address on the envelope. The port number is like the name of the person who lives in the house. When the letter arrives at the house, it is delivered to the person whose name is on the envelope.

The port numbers are in a range from 0 to 65535. The port numbers are divided into three ranges:

Ports 0 to 1023 are reserved for well-known services, such as HTTP(web), FTP(file transfer), and SMTP(email), and applications should not use these ports unless they are providing the corresponding service.

A web server, for example, listens for incoming data on port 80. When you type a URL into your web browser, the browser sends a request to target port 80.

Note that even though you don’t see the port number in the URL. i.e. when you type http://www.example.com in your browser, port 80 is assumed, the url will be treated as http://www.example.com:80.

Custom applications should use port 1024 to 49151. Common Java based servers like Tomcat default to port 8080, while Javascript Node.js would use port 3000.

The dynamic ports are used by the operating system when a client application connects to a server application.

Domain Name System (DNS) #

You may not know the IP address of the target computer, even though it is required to send data over the internet. You will be more familiar with the hostname or domain name of the target computer, such as www.example.com. This is the human-readable name of the target server, and is used in the URL of a website.

However, since the TCP/IP protocol requires an IP address, the domain name needs to be converted to an IP address. A solution is to use a lookup service. This is where the Domain Name System (DNS) comes in.

The DNS is a distributed database that maps hostnames to IP addresses. Distruibuted means that the database is not stored in a single location, but is spread across multiple servers around the world. This ensures that the DNS service is reliable and can handle a large number of requests.

When you type a URL into your web browser, the browser sends a request to a DNS server to resolve the hostname to an IP address. The DNS server then returns the IP address to the browser, which can then connect to the target computer.

Think of the DNS as a phone book for the internet. When you want to call someone, you look up their name in the phone book to get their phone number. The DNS performs a similar function, but instead of phone numbers, it returns IP addresses.

TCP/IP Packets #

The TCP/IP protocol breaks up the data into packets, as sending large amounts of data in a single packet can be inefficient and prone to errors. The packets are then sent over the network to the destination computer.

The packets can also take different routes to reach the destination and can arrive out of order. If a packet is lost or corrupted during transmission, the recipient can request that the sender resend the packet.

Going back to the analogy of sending a letter to a house, the data that is sent over the internet is broken up into smaller pieces called packets. If the letter is too large to fit in a single envelope, it is broken up into multiple envelopes, each containing a part of the letter.

The envelopes are then sent separately and can arrive at the destination in any order. The recipient can then reassemble the envelopes in the correct order to read the letter. Data sent over the internet works in a similar way.

The packets are composed of two main parts: the TCP packet and the IP packet.

The IP packet contains the TCP packet and is targeted to the IP address of the destination computer. The TCP packet contains the data that is being sent, and is targeted to the port number of the application running on the destination computer, such as a web server.

Data Encapsulation #

Each layer in the TCP stack is responsible for a particalar task in the data transfer process. To perform this task it needs to add some information to the data. This information is added to the data in the form of a header. This process is called data encapsulation.

So data from an application has a TCP layer header added to it. The IP layer then adds its header to the TCP packet, and so on.

When the data reaches the destination, the headers are removed in the reverse order. The data is then passed to the application layer of the destination computer.

TCP Packet Structure #

The TCP packet header contains the following fields to communicate between applications across the network.

• Source Port: The port number of the sender.
• Destination Port: The port number of the recipient.
• Sequence Number: The sequence number of the first byte in the packet.
• Acknowledgment Number: The sequence number of the next byte the sender expects to receive.
• Data Offset: The length of the TCP header in 32-bit words.
• Reserved: Reserved for future use.
• Flags: Used to regulate communication.
  SYN: Synchronize sequence numbers to initiate a connection..
  ACK: Acknowledge receipt of data..
  FIN: Finish the connection..
  RST: Reset the connection..
  
• Window Size: The size of the receive window, which specifies how much data the sender can send before receiving an acknowledgment.
• Checksum: A checksum that is used to detect errors in the packet.
• Urgent Pointer: Indicates the end of urgent data in the packet.
• Options: Optional fields that can be used to provide additional information.
• Data: The application data that is being sent.

Notable fields are the destination port, and the data. The destination port is used to route the packet to the correct application on the destination computer. The data is the actual data that is being sent.

IP Packet Structure #

The IP packet wraps the TCP packet, and is used to route the packet to the destination computer. Thus it will contain the IP address of the destination computer. So it is targeted to the IP address of the destination computer.

• Version: The version of the IP protocol being used.
• Header Length: The length of the IP header in 32-bit words.
• Type of Service: Specifies the quality of service for the packet.
• Total Length: The total length of the IP packet.
• Identification: A unique identifier for the packet.
• Flags: Control bits that indicate fragmentation information.
• Fragment Offset: The offset of the fragment in the original packet.
• Time to Live: The maximum number of hops the packet can take before being discarded.
• Protocol: The protocol used in the data portion of the packet (e.g., TCP, UDP).
• Header Checksum: A checksum that is used to detect errors in the header.
• Source IP Address: The IP address of the sender.
• Destination IP Address: The IP address of the recipient.
• Options: Optional fields that can be used to provide additional information.
• Data: The TCP packet that is being sent.

TCP/IP Data Transfer Flow #

A data transfer flow between two computers can be broken down into the sequence diagram below.

Note that data is not immediately sent from the client to the server. The client first needs to obtain the IP address of the server, negotiate a connection with the server, send the data, and then finally close the connection. The various fields in the TCP and IP headers are used to manage this process.

  
sequenceDiagram
    participant Client
    participant DNS Server
    participant Server
    
    Note over Client, Server: Obtain IP address of server
    Client->>DNS Server: DNS Query (What's the IP of www.example.com?)
    DNS Server-->>Client: DNS Response (IP = 192.168.1.10)
    
    Note over Client, Server: Establish connection (3-way handshake)
    Client->>Server: SYN (Request to initiate connection)
    Server-->>Client: SYN-ACK (Acknowledge and request to initiate)
    Client->>Server: ACK (Connection established)

    Note over Client, Server: Send Data
    Client->>Server: Send Data (e.g., HTTP GET request)
    Server-->>Client: Send Data (e.g., HTML page response)

    Note over Client, Server: Close connection
    Client->>Server: FIN (Request to close connection)
    Server-->>Client: ACK (Acknowledge close request)
    Server->>Client: FIN (Request to close connection)
    Client-->>Server: ACK (Acknowledge close request)

Conclusion #

By understanding the TCP/IP protocol, developers can build reliable applications that communicate over the internet. In addition, issues and errors that arise during data transfer can be more easily diagnosed and resolved.