Messaging

Part of Networking

Network-based messaging systems for communication between people and processes.

Why This Matters

Messaging is the most fundamental human use of networks. Before file sharing, before distributed computing, before the World Wide Web, people used networks to send messages to each other. Electronic messaging transforms communication by making it asynchronous — the sender and receiver need not be present simultaneously — and by enabling communication across distances that would otherwise require physical travel or slow physical mail.

In a reconstruction context, electronic messaging provides organizational infrastructure that dramatically improves coordination. A medical clinic can communicate with specialists remotely. An agricultural cooperative can coordinate planting and distribution. An engineering team spread across multiple locations can share designs and make decisions without gathering in one place. The value of messaging scales with the number of people connected — a network connecting 10 people enables 90 unique communication pairs; a network connecting 100 people enables 9,900.

Understanding messaging systems means understanding both the basic protocols that enable message delivery and the design principles that make messaging systems reliable and useful in practice.

Email: Store-and-Forward Messaging

Email is the foundational asynchronous messaging system. It uses a store-and-forward model: messages are sent to a server, which stores them until the recipient retrieves them. This means the recipient need not be online when the message is sent — the server holds the message indefinitely until retrieved.

The email system uses two distinct protocols. SMTP (Simple Mail Transfer Protocol) handles sending and routing: when you send a message, your email client connects to your mail server via SMTP and hands off the message. Your server then uses SMTP to connect to the recipient’s server and deliver the message. The recipient’s server stores the message in the recipient’s mailbox.

Retrieval uses either POP3 (Post Office Protocol 3) or IMAP (Internet Message Access Protocol). POP3 downloads messages to the client and typically deletes them from the server — simple but inflexible. IMAP leaves messages on the server and synchronizes between server and multiple clients, so the same messages appear on phone, desktop, and web interface. For any multi-device environment, IMAP is preferable.

SMTP is a text-based protocol designed for interoperability. Commands like HELO, MAIL FROM, RCPT TO, and DATA are human-readable strings. The response to each command is a three-digit status code. This transparency made SMTP easy to implement and debug, though it also made it easy for spammers to abuse — which is why modern email systems add authentication layers (DKIM, SPF, DMARC) on top of the basic protocol.

Instant Messaging Protocols

Unlike email, instant messaging is synchronous or near-synchronous — messages are delivered within seconds rather than at the recipient’s next login. This immediacy makes it useful for interactive conversations but requires both parties to be online simultaneously (or for the system to buffer messages for delivery when the recipient connects).

XMPP (Extensible Messaging and Presence Protocol, formerly Jabber) is the open standard for instant messaging. It uses XML-based messages over persistent TCP connections. Each user has a JID (Jabber ID) in the format user@domain, similar to email. Messages are routed through the user’s server, which routes to the recipient’s server, which delivers to the recipient’s client.

XMPP servers can federate — servers operated by different organizations can exchange messages, just as email servers do. This federation makes XMPP interoperable: a user on one XMPP server can message any user on any other federated server. This design is in contrast to proprietary messaging systems (WhatsApp, Signal, etc.) that are siloed.

For a small network without internet connectivity, running a local XMPP server (such as ejabberd or Prosody) provides immediate, reliable messaging for all users on the local network. Configuration is straightforward and requires minimal resources — a basic XMPP server for a few dozen users runs comfortably on very modest hardware.

Message Queuing for Process Communication

Beyond human-to-human messaging, networks need mechanisms for process-to-process message exchange — one program sending data to another program asynchronously. This is message queuing.

A message queue is a buffer maintained by a server. A producer puts a message on the queue; a consumer takes messages off the queue and processes them. The producer and consumer are decoupled: the producer does not need to wait for the consumer to be ready, and the consumer processes messages at its own pace. If the consumer is temporarily unavailable, messages accumulate in the queue and are processed when the consumer resumes.

Simple message queue systems can be built on top of a database: one process inserts rows into a database table; another process selects and deletes rows in order. This approach works for low-throughput applications and requires no additional infrastructure. For higher throughput or more complex routing requirements, dedicated message queue software (RabbitMQ, Redis queues) provides better performance and features.

Message queues are fundamental to distributed systems design. They decouple producers from consumers, smooth out load spikes, enable retry logic for failed operations, and allow systems to scale producers and consumers independently. Understanding when to use a message queue versus direct function calls or HTTP requests is a key architectural skill.

Designing Reliable Messaging

Reliable messaging requires more than just transmitting a message and hoping it arrives. Three properties define reliable messaging: guaranteed delivery, ordering, and exactly-once delivery.

Guaranteed delivery means a message is delivered even if the recipient is temporarily unavailable. Email achieves this through server storage — the message is held until the recipient’s server accepts it. For process queues, a persistent queue (backed by disk rather than memory) ensures messages survive server restarts.

Ordering means messages are delivered in the order they were sent. TCP provides ordering within a single connection. For messages across multiple connections or after reconnection, ordering requires sequence numbers and the receiver to buffer and reorder received messages.

Exactly-once delivery is the hardest to achieve. A message might be lost (no delivery), delivered once (correct), or delivered multiple times (if the network retransmits a message that was actually received but whose acknowledgment was lost). Exactly-once requires both deduplication at the receiver (to discard duplicate messages) and acknowledgment tracking at the sender (to know which messages need retransmission). Most real systems achieve at-least-once delivery and make message handlers idempotent (safe to execute multiple times with the same result) rather than the more complex exactly-once mechanism.

Building a Simple Messaging System

For a small network without access to existing messaging software, a minimal store-and-forward messaging system can be built with minimal complexity.

Each user has a mailbox: a directory on the server where their messages are stored as individual files. To send a message, the sender writes a file to the recipient’s mailbox directory. To check messages, the recipient lists their mailbox directory and reads the files. This is the basic maildir format used by real email servers.

Adding network access requires a simple server that accepts connections, authenticates users (matching username and password against a stored credential), accepts message upload (write to recipient mailbox), and serves message download (list and read from sender mailbox). A few dozen lines of code in any language with socket and file I/O support is sufficient for a functional system.

This minimal system lacks many conveniences (search, threading, attachments, forwarding) but provides the essential function: reliable asynchronous message delivery between users on the network. It can be built, understood, and operated by anyone with basic programming skills — which is exactly what is needed in a reconstruction context where commercial software may be unavailable.