Lesson 6 — Shared ownership: Arc, Mutex, RwLock

Question

Rust ownership = "one owner". When you need to share across threads, you need Arc (reference counting) + Mutex / RwLock (exclusion). Three primitives that appear constantly in concurrent code.

Principle (minimum model)

• Rc<T>. Reference-counted, single-thread only. Rc::clone bumps the counter; the last drop frees. !Send / !Sync.
• Arc<T>. Atomic reference-counted, cross-thread. The Send + Sync version of Rc. Constant fixture in Tokio code.
• Mutex<T>. Mutual exclusion. .lock() returns a MutexGuard; drop releases. Beware deadlocks. Sync version (std::sync::Mutex) vs async (tokio::sync::Mutex).
• RwLock<T>. Read-write split. .read() allows many; .write() is exclusive. Wins when reads dominate. Sync vs async variants.
• Arc<Mutex<T>>. The most common pattern. Share T across threads with exclusive write. let shared = Arc::new(Mutex::new(value)); then shared.clone() to a thread.
• Atomic primitives. AtomicUsize / AtomicBool etc. Lock-free read-modify-write; Ordering controls memory ordering. Lightest weight.
• Cell<T> / RefCell<T>. Interior mutability. Mutate behind &self. Cell only for Copy types; RefCell does runtime borrow-check. Single-thread only.
• tokio::sync::*. Async versions of Mutex / RwLock / oneshot / mpsc — .await to wait. Use in async tasks. Holding std::sync::Mutex across .await is a classic deadlock.

Worked example + steps

Shared ownership: Arc, Mutex, RwLock

Rust's "one owner" rule is sharp — and helpful 90% of the time. The other 10%, you need multiple parts of the program to hold the same value, possibly across threads, possibly with mutation. That's what this lesson is about.

Reth and ExEx code is densely sprinkled with Arc and Mutex. Without this lesson those wrappers feel like noise. With it, they're the load-bearing pieces of how Reth shares state across async tasks.

The problem — why ownership isn't enough

Imagine an indexer that:

• Receives every new block from the chain (one task)
• Lets the RPC server query its current state (many tasks)
• Periodically snapshots state to disk (one task)

All three need access to the same in-memory state. Plain Rust ownership says "one owner" — but here we need shared access.

Two questions to answer:

1. Who owns the value? Multiple places.
2. Who can mutate it? It depends.

Arc — multiple owners, immutable shared

Arc<T> ("Atomically Reference-Counted") lets multiple owners share a value, with the value freed when the last owner drops it.

use std::sync::Arc;

let data = Arc::new(String::from("hello"));
let clone1 = Arc::clone(&data);   // refcount: 1 → 2
let clone2 = Arc::clone(&data);   // refcount: 2 → 3

std::thread::spawn(move || {
    println!("{}", clone1);       // can move into another thread
});
println!("{}", clone2);
// clone2 dropped → refcount: 3 → 2
// clone1 dropped (in spawned thread) → refcount: 2 → 1
// data dropped → refcount: 1 → 0 → memory freed

Two important properties:

Arc::clone is cheap

Arc::clone(&x) does not deep-copy the inner T. It only:

1. Atomically increments a counter
2. Returns a new pointer to the same heap allocation

Cost: one atomic add. The inner T isn't touched. This is why you'll see Arc::clone(&shared_state) everywhere in Reth — it's effectively free.

Arc gives you read-only sharing

Through an Arc<T>, you can only call &self methods on T. You cannot mutate. That's by design — multiple threads holding the same Arc<T> can't simultaneously read and write without synchronization, and Rust enforces this at the type level.

If you need mutation, wrap T in a Mutex or RwLock first.

Arc vs Rc

Rc<T> is the same shape but single-threaded. It uses a regular (non-atomic) counter, which is faster but unsound across threads. Reth and ExEx use Arc exclusively — they're multi-threaded by design.

Mutex — exclusive read/write access

Mutex<T> wraps a value with a lock. To read or write, you must lock() first:

use std::sync::{Arc, Mutex};

let counter = Arc::new(Mutex::new(0u64));

let c = Arc::clone(&counter);
std::thread::spawn(move || {
    let mut guard = c.lock().unwrap();   // acquire lock
    *guard += 1;                          // mutate
    // guard dropped here → lock released
});

lock() returns a MutexGuard<T> — a smart pointer that:

• Implements Deref so you can use it like &T (reading) or &mut T (writing)
• Releases the lock when dropped (RAII pattern — no manual unlock)
• Blocks the calling thread if another thread already holds the lock

What .unwrap() panics on — poisoning

You'll see .lock().unwrap() everywhere. Why unwrap?

Because lock() returns Result<MutexGuard, PoisonError>. The error case is mutex poisoning — if a thread holds the lock and panics, the mutex is marked "poisoned" so subsequent lockers know the data might be inconsistent.

Most Reth code chooses to propagate (panic on) poisoning rather than handle it specially. .unwrap() is the right call when you can't reason about partial-update inconsistency.

If you ever see Mutex poisoning in your own logs, the underlying bug is the original panic, not the lock — find what panicked and fix that.

The pattern: Arc<Mutex<T>>

You'll see this combination constantly:

let shared_state: Arc<Mutex<MyState>> = Arc::new(Mutex::new(MyState::default()));

for _ in 0..10 {
    let s = Arc::clone(&shared_state);
    std::thread::spawn(move || {
        let mut guard = s.lock().unwrap();
        guard.do_work();
    });
}

Arc provides the multi-owner pointer; Mutex provides the mutation safety. Together: thread-safe shared mutable state.

RwLock — many readers OR one writer

Mutex is exclusive — only one thread at a time, even for reads. That's wasteful when reads vastly outnumber writes.

RwLock<T> allows:

• Many concurrent readers (.read())
• Or one exclusive writer (.write())
• Never both at the same time

use std::sync::{Arc, RwLock};

let cache = Arc::new(RwLock::new(HashMap::<String, u64>::new()));

// Many threads can read simultaneously
let c = Arc::clone(&cache);
std::thread::spawn(move || {
    let guard = c.read().unwrap();      // shared read access
    if let Some(v) = guard.get("key") {
        println!("{}", v);
    }
});

// One thread writes (blocks all readers and other writers)
let c2 = Arc::clone(&cache);
std::thread::spawn(move || {
    let mut guard = c2.write().unwrap();
    guard.insert("key".to_string(), 42);
});

When to choose:

RwLock is slightly heavier than Mutex for the writer (more bookkeeping), so don't reach for it unless reads dominate.

The async story — tokio::sync::Mutex

Standard std::sync::Mutex is synchronous: .lock() blocks the OS thread. In an async context (Tokio), blocking the thread is bad — it starves other tasks on the same worker.

For async code, use tokio::sync::Mutex:

use tokio::sync::Mutex;
let m = Arc::new(Mutex::new(0u64));

let m_clone = Arc::clone(&m);
tokio::spawn(async move {
    let mut guard = m_clone.lock().await;   // .await, not .unwrap()
    *guard += 1;
});

The differences:

• .lock() returns a Future — you .await it
• The task yields to the runtime while waiting (doesn't block the worker)
• No poisoning concept (panicking inside an async task is handled by Tokio)

Reth uses both: std::sync::Mutex for fast critical sections that won't span an .await, tokio::sync::Mutex for guarded state shared across awaits.

The rule of thumb: if your critical section contains an .await, use tokio::sync::Mutex. Otherwise std::sync::Mutex is fine and slightly faster.

A real Reth pattern — sharing a database handle

A simplified version of how Reth shares its database across tasks:

struct Node {
    db: Arc<Database>,                     // shared, immutable handle
    blockchain_tree: Arc<RwLock<Tree>>,    // shared, mostly read
    metrics: Arc<Mutex<MetricsCollector>>, // shared, write-heavy
}

impl Node {
    fn spawn_indexer(&self) {
        let db = Arc::clone(&self.db);
        let metrics = Arc::clone(&self.metrics);
        tokio::spawn(async move {
            // db: read-only access through Arc, no lock needed
            // metrics: lock to update counters
            metrics.lock().unwrap().increment("blocks_indexed");
        });
    }
}

Three different sharing patterns in one struct. Each choice is deliberate — the lock granularity matches the access pattern.

When you read Reth source and see Arc<RwLock<Foo>> next to Arc<Mutex<Bar>>, the author has thought about which one is right. Now you can read those decisions.

Reading list

1. Rust Book chapter 16 (Fearless Concurrency) — read the section on Arc and Mutex.
2. tokio::sync docs — skim the page for Mutex, RwLock, broadcast, oneshot. Each maps to a real pattern in Reth's task code.
3. Search Reth source for Arc<RwLock and Arc<Mutex. Pick a couple. The choice between them is always intentional — try to explain why.

What you should walk away with

• Arc<T> is multi-owner, immutable, cheap to clone (one atomic add).
• Mutex<T> is exclusive (one reader-or-writer at a time), and .lock() blocks the thread.
• RwLock<T> is many-reader OR one-writer — use when reads dominate.
• tokio::sync::Mutex is the async-aware version — use when the critical section spans .await.
• Arc<Mutex<T>> is the canonical "shared mutable state" pattern — you'll see it everywhere in Reth/ExEx.

When Intermediate lesson 6 (ExEx) shows you a struct with three Arc<...> fields and you wonder "why all the wrappers," you'll know each one is the load-bearing piece of how that component is shared across the runtime's tasks.

Summary (3 lines)

• Arc<T> for cross-thread sharing; Mutex / RwLock for exclusion. Arc<Mutex<T>> is the canonical pattern. Atomics are lock-free for simple cases.
• Interior mutability: Cell / RefCell (single-thread) / Mutex / RwLock (multi-thread). Mutate behind &self.
• Use tokio::sync::* in async code. Holding std::sync::Mutex across .await = deadlock. Next lesson: unsafe Rust.