Protocols
Part of Networking
The rules and standards that enable different machines to communicate.
Why This Matters
A protocol is a set of rules that two communicating parties agree to follow. Without shared protocols, two computers — even two computers directly connected by a cable — cannot communicate meaningfully. They might transmit bits, but neither would know what to do with the received bits, how to indicate the end of a message, or how to respond to errors.
Protocols are what make interoperability possible. A computer built by one manufacturer, running software written by a different company, can communicate with any other computer that implements the same protocol. The protocol is the contract — each party knows exactly what to send, when to send it, and what to do when things go wrong, because the protocol specifies all of this.
Understanding protocols gives you the ability to troubleshoot network problems at a deep level, implement network applications, and evaluate whether a proposed protocol design will actually work in practice.
The Protocol Stack
Networking uses a layered architecture where multiple protocols work together, each handling one aspect of the communication. The TCP/IP protocol stack is the dominant model for modern networking.
At the bottom, the link layer protocol (Ethernet, Wi-Fi) handles delivery of frames between directly connected devices on the same network segment. The link layer does not route packets across multiple network segments.
Above that, the Internet Protocol (IP) handles routing — delivering packets from source to destination across multiple network segments, even if the source and destination are on different continents. IP is a best-effort service: it delivers packets when possible, but makes no guarantees about reliability, ordering, or timing.
Above IP, the Transport layer provides end-to-end communication services. TCP (Transmission Control Protocol) provides a reliable, ordered byte stream — it retransmits lost packets, reorders out-of-sequence packets, and provides flow control to prevent the sender from overwhelming the receiver. UDP (User Datagram Protocol) provides simple unreliable datagram delivery, with minimal overhead but no reliability guarantees.
At the top, Application layer protocols define how specific applications communicate: HTTP for web content, SMTP for email, DNS for name resolution, SSH for secure remote access. Each application protocol runs on top of TCP or UDP.
TCP: Transmission Control Protocol
TCP is the workhorse of the internet. Most application protocols (HTTP, SMTP, SSH, FTP) use TCP because it provides the reliable byte stream that applications need without requiring each application to implement its own reliability mechanisms.
TCP uses a three-way handshake to establish a connection:
- SYN: Client sends a segment with the SYN (synchronize) flag set and an initial sequence number.
- SYN-ACK: Server responds with SYN and ACK flags set, acknowledging the client’s sequence number and providing its own.
- ACK: Client acknowledges the server’s sequence number.
After this handshake, both sides have established the initial sequence numbers and the connection is ready for data transfer. Every byte of data is numbered with a sequence number; every received byte is acknowledged. The sender retransmits any segment that is not acknowledged within a timeout period.
TCP’s flow control uses a receive window: the receiver advertises how many bytes of buffer space it has available, and the sender limits the amount of unacknowledged data to this window size. This prevents a fast sender from overwhelming a slow receiver’s buffers.
TCP’s congestion control limits the sender’s transmission rate based on observed network conditions. The key algorithm is Additive Increase Multiplicative Decrease (AIMD): the sender slowly increases its transmission rate until it detects congestion (packet loss or ECN signal), then quickly halves its rate and begins increasing again. This algorithm, followed by all TCP senders simultaneously, causes the network to converge on a fair allocation of available capacity.
UDP: User Datagram Protocol
UDP adds only minimal service on top of IP: port numbers (to identify which application receives the data) and a checksum. There is no connection establishment, no acknowledgment, no retransmission, and no ordering.
UDP is appropriate when the application handles reliability itself, or when the application cannot tolerate the delay of TCP retransmission. Real-time voice and video use UDP — a retransmitted voice packet would arrive too late to be useful, so it is better to play silence or use error concealment than to wait for retransmission. DNS uses UDP for queries because the simple request-response pattern does not benefit from TCP’s connection overhead, and the DNS client handles retransmission by simply resending the query if no response arrives.
UDP’s simplicity also makes it the foundation for custom protocols that need control over reliability and ordering that TCP does not provide exactly. QUIC (the protocol underlying HTTP/3) is built on UDP and implements its own, more flexible, reliability and multiplexing.
DNS: Domain Name System
DNS translates human-readable names (www.example.com) into IP addresses (93.184.216.34). Without DNS, every application would need to know the IP addresses of all services it communicates with, and every address change would require updating every application.
DNS is a distributed database organized as a hierarchy. The root zone (.) is the top level, served by the root name servers. Below that, top-level domains (com, org, net, country codes) are served by TLD name servers. Below that, individual domain names are served by authoritative name servers configured by the domain owner.
A DNS resolver (running on your machine or your local network) resolves names by querying this hierarchy. If the answer is not in the resolver’s cache, it queries the root, gets a referral to the TLD server, queries the TLD server, gets a referral to the authoritative server, and queries the authoritative server to get the final answer.
DNS caching is essential for performance. Each DNS record has a TTL (Time to Live) that specifies how long the record can be cached. A resolver that has recently answered a query for an address serves subsequent queries from cache without querying the authoritative server. High TTL values reduce DNS traffic and latency; low TTL values allow faster propagation of changes.
HTTP: Hypertext Transfer Protocol
HTTP is the protocol of the World Wide Web — every web request uses HTTP. It is a request-response protocol: the client sends a request with a method (GET, POST, PUT, DELETE), a URL, and headers; the server responds with a status code, headers, and optionally a body.
HTTP is stateless: each request-response exchange is independent. A server does not inherently remember previous requests from the same client. This statelessness simplifies server implementation but requires applications that need state (user login sessions, shopping carts) to maintain state explicitly through cookies, session tokens, or other mechanisms.
HTTP/1.1 (the most widely implemented version) uses a text-based format — request and response headers are human-readable ASCII. This makes HTTP easy to inspect and debug with simple tools. A minimal HTTP server can be implemented in a few hundred lines of code in any language, making HTTP the practical choice for building networked applications in a reconstruction context.
Implementing Simple Protocols
When standard protocols are overkill or unavailable, simple custom protocols can be designed for specific applications. Good protocol design follows these principles:
Define the message format precisely: specify every field, its size in bytes, its valid values, and what happens if an invalid value is received. Ambiguous protocol definitions lead to interoperability failures.
Include a length or end-of-message indicator: the receiver must know when a message is complete. Common approaches are fixed-length messages (simple but inflexible), length-prefixed messages (the first N bytes contain the total message length), and delimiter-terminated messages (messages end at a specific byte sequence, used by HTTP for headers).
Include version negotiation: protocols evolve. If two implementations support different protocol versions, they must agree on a common version. Include a version field in the initial handshake so both sides can negotiate down to the highest common version.
Define error handling: specify what each party does when it receives an unexpected message, a malformed message, or no message. Protocols that define only the happy path fail unpredictably when real errors occur.