FABRKNT
P2P Networking Internals — From devp2p to Custom Gossip
P2P Networking
Lesson 2 of 4·CONTENT17 min45 XP

Treat this page as a workbench, not a blog post. The goal is to extract a reusable mental model from the source and carry it into the rest of the Fabrknt stack.

Course
P2P Networking Internals — From devp2p to Custom Gossip
Lesson role
CONTENT
Sequence
2 / 4

Lesson 2 — Reading reth's network crate

Question

You want to add a custom sub-protocol to your chain — a settlement finality hint, an MEV bundle gossip channel, whatever. Where in the reth tree does that code live, and which existing pieces does it plug into? Reth's network layer lives under crates/net/ and is spread across six sub-crates totalling roughly 30k lines of Rust.

Principle (minimum model)

  • Six sub-crates. net/discv5 / net/eth-wire / net/network / net/network-api / net/peers / net/dns.
  • Read in five passes. network-api (the API surface) → network (orchestration) → eth-wire (protocol messages) → peers (peer state machine) → discv5 (DHT internals).
  • NetworkManager is the central orchestrator. Wires together Swarm + NetworkHandle + Discovery + NetworkState; three input streams (peer messages + new peers + commands) feed into one dispatcher.
  • Swarm state machine. NewConnection → Handshake → Negotiation → Active → Disconnected. Hard cap of 25–50 peers; eviction is score-based.
  • eth-wire defines message structs with RLP-derive. #[derive(RlpDecodable, RlpEncodable)] auto-generates the wire format from the struct definition.
  • Peer state machine has four parts. Capability set + Score (response time / error rate) + Stats (bytes / message count) + connection state.
  • Custom sub-protocols are the extension point for chain-specific gossip. Implement NAME + VERSION + MESSAGES_COUNT + on_message; they run alongside eth/68 on the same RLPx connection.
  • Peer scoring is a strategic knob. Not just for ejecting bad peers — also for prioritising trusted infra (MEV / privacy / performance chains all use this as a strategy lever).

Worked example + steps

Reading reth's network crate

You want to add a custom sub-protocol for your chain — say, payment-finality hints or MEV bundle gossip. Where in the reth tree does that code go, and what existing pieces does it plug into? Reth's network layer lives at crates/net/ — ~30k lines of Rust split across six sub-crates handling discovery, transport, gossip, and peer management. This lesson is the orientation: what's where, what each crate does, where the extension points are.

1. The network crate map

CrateRole
net/discv5discv5 implementation (Kademlia DHT)
net/eth-wireWire encoding/decoding for eth/68 messages
net/networkTop-level orchestration
net/network-apiPublic API for app code
net/peersPeer management, scoring, eviction
net/dnsDNS-based peer discovery (alternate to discv5)

Reading order if you want to understand the whole thing:

  1. network-api (API surface)
  2. network (main orchestration)
  3. eth-wire (the protocol messages)
  4. peers (the peer state machine)
  5. discv5 (the discovery DHT)

2. The NetworkManager — central orchestrator

Every peer message, every discovery hit, every "broadcast this tx" command from the rest of the node flows through one struct. That struct is NetworkManager, in crates/net/network/src/manager.rs:

pub struct NetworkManager<C> {
    swarm: Swarm<C>,                    // Peer connections
    handle: NetworkHandle,              // Public API handle
    from_handle_rx: UnboundedReceiver<NetworkHandleMessage>,
    discovery: Discovery,                // discv5 / DNS
    state: NetworkState<C>,             // Internal state
    // ...
}

The run loop is small:

  1. Poll swarm for peer messages
  2. Poll discovery for newly discovered peers
  3. Poll from_handle_rx for commands (e.g., "broadcast this tx")
  4. Dispatch each event

Three input streams, one dispatcher. That's the heart of reth's networking.

🔍 Find in repo. Open crates/net/network/src/manager.rs and find the main poll_next or run method. What's the polling order? Why might that matter?

3. The Swarm — peer connection pool

Swarm is the pool of active peer connections under NetworkManager. Each connection runs through a small state machine:

NewConnection → Handshake → Negotiation → Active → Disconnected

For each peer:

  • NewConnection: TCP connect or accept
  • Handshake: RLPx authentication
  • Negotiation: agree on supported sub-protocols (eth/68 etc.)
  • Active: exchange messages
  • Disconnected: graceful close or error

The Swarm enforces peer limits (typically 25-50 active) and eviction policy (drop low-scoring peers when new ones want in).

4. eth-wire — the protocol messages

Wire-format code lives in one crate. Each eth/68 message is a Rust struct with RLP-derive macros doing the encoding for you:

#[derive(Debug, RlpDecodable, RlpEncodable)]
pub struct NewBlock {
    pub block: Block,
    pub total_difficulty: U256,
}

#[derive(Debug, RlpDecodable, RlpEncodable)]
pub struct NewPooledTransactionHashes {
    pub types: Vec<u8>,
    pub sizes: Vec<u32>,
    pub hashes: Vec<TxHash>,
}

The derive macros generate the wire format. Every message is RLP — the same encoding used for transactions and blocks.

For custom sub-protocols, you define your own message structs and register them with the network. (We do that in lesson 3.)

5. The peer state machine

crates/net/peers/src/peer.rs tracks per-peer state:

  • Capability set: what sub-protocols does this peer support?
  • Score: based on response time, error rate, banhammer events
  • Stats: bytes sent/received, messages by type, timing
  • Connection state: handshake done, sub-protocol negotiated, etc.

Peer scoring matters: peers that misbehave get evicted. The default scoring penalizes:

  • Slow responses
  • Invalid messages (bad RLP, wrong hashes)
  • Misbehavior (sending old txs repeatedly, claiming to have data they don't have)

6. Adding custom sub-protocols

This is the extension point most Reth-based chains use. Need chain-specific gossip — merchant attestations, payment finality hints, sequencer coordination? You ship a sub-protocol:

// In your chain's crate
pub struct TempoSubProtocol {
    // Your state
}

impl SubProtocol for TempoSubProtocol {
    const NAME: &'static [u8] = b"tempo";
    const VERSION: u8 = 1;
    const MESSAGES_COUNT: u8 = 5;

    fn on_message(&mut self, peer: PeerId, msg: Bytes) -> eyre::Result<()> {
        let parsed: TempoMessage = decode(&msg)?;
        match parsed {
            TempoMessage::MerchantAttestation(att) => self.handle_attestation(peer, att),
            TempoMessage::PaymentFinalityHint(hint) => self.handle_hint(peer, hint),
            // ...
        }
    }
}

Register this with the network manager and your custom protocol runs alongside eth/68 on the same RLPx connections. No new TCP ports, no separate discovery — it rides on the existing peering. Tempo likely uses this pattern for payment-specific gossip.

7. The peer scoring opportunity

Default peer scoring is generic — it punishes bad actors. But scoring is also a steering wheel for specialized chains:

  • MEV-relevant chains: score peers based on tx propagation speed
  • Privacy-focused chains: score peers based on metadata leakage
  • Performance-focused chains: score peers based on bandwidth + latency

For Tempo: a payment-priority chain might score peers by whether they're known merchant infra vs. generic peers. This is a chain-specific networking decision.

8. Practice

  1. Browse crates/net/network/src/manager.rs — find the main poll loop
  2. Open crates/net/eth-wire — find the message enum
  3. Identify: how would you add a custom message type for payment finality hints?
  4. Estimate: how many peers does a typical reth node maintain? How does this scale to 1000+ chains?

Final check: in one sentence, where in reth would you add a custom sub-protocol for chain-specific gossip? If your answer doesn't reference NetworkManager or sub-protocol registration, re-read §6.

Pass criteria

  • Name the six sub-crates under crates/net/ and what each owns.
  • Recite the five-pass reading order (network-api → network → eth-wire → peers → discv5).
  • Sketch NetworkManager: three input streams, one dispatcher.
  • Walk the Swarm state machine from NewConnection through Disconnected.
  • Describe the four pieces inside the peer struct.
  • Explain where a custom sub-protocol plugs in (NAME / VERSION / MESSAGES_COUNT / on_message).
  • Give two reasons peer scoring is strategic, not just defensive.

Summary (3 lines)

  • Reth's network crate = six sub-crates (~30k lines), centred on NetworkManager (three inputs, one dispatcher), best read API-surface-first.
  • eth-wire uses RLP-derive macros to keep message structs and wire format in lock-step; the peer state machine has four parts and is the substrate for scoring.
  • Custom sub-protocols are first-class: same RLPx connection, separate NAME/VERSION namespace. Next lesson builds one for MEV-style messaging.