Informed Diffusion: making home builders publish fast

Дата публикации: 13-10-2026 06:36:48

This post continues EIP-8411: what segmented payload diffusion is made of (2026), which built segmented payload diffusion up in three tiers, and its cross-check, EIP-8411 payload segmentation under the Shadow simulator (2026). Here we add a fourth tier and give the design as a whole. The card reference describes each mechanism on its own; the companion note Informed Diffusion, measured has the measurements; and the draft specification, Informed Diffusion: the protocol in pseudocode, gives every rule exactly, with a toy that runs it.
Author: @cskiraly
TL;DR
Today’s broadcast leaves a home builder behind. Mainnet is in time largely because payloads are small and datacenters carry them. Assuming a network where 20% of the nodes have a 1 Gbps uplink, while the rest have only 50 Mbps (the mixed-bandwidth network), a datacenter builder delivers a 1 MiB payload to all but the last percent of nodes (p99) in 1.8 s. A home builder, in the same network, delivers only in 4.2 s. If instead we ask what the protocol could deliver without 1 Gbps nodes (the all-home network), with every node on 50 Mbps up and 100 down, it would take 6.3 s.
Tier 4 of Informed Diffusion gets that 1 MiB payload delivered in about 0.6 s even in the most challenging all-home network. It also reduces bandwidth utilization, delivering only 1.7 copies of the payload on average instead of the 4.4 there today.
The design is a stack of small mechanisms, which we call cards, organized in four tiers. Tier 1 cuts the payload into chunks that a node forwards before it holds the whole, so the other nodes pass the payload on while the builder is still sending it; it carries most of the latency gain. Tier 2 trades pushes for announcements to save bytes. Tier 3 codes the payload so that no chunk is the last one. Tier 4, the main contribution of this post, lets every node act on what it knows about its neighbours, and keeps the source’s uplink for what only the source can send.
Tier 4 adds only one thing to the wire: a HAVE bitmap per payload per peer, with the ask by index that comes with it. The partial-messages extension uses such bitmaps too; here the chunks stay ordinary gossipsub messages, and the bitmap, carried in the existing control messages, only prunes and aims the pushes. Everything else in tier 4 is a local decision a client can adopt one at a time.
In our simulations, tier 4 of Informed Diffusion stayed inside the 3 s budget in every stress scenario we ran, including up to half the nodes withholding or silent, lost data and requests, every uplink cut to 10 Mbps, and payloads from 128 KiB to 2 MiB.
1. The problem
A payload has to reach the network within a budget: as in the earlier posts, we take the time from the builder’s reveal to the payload’s due time, which the payload timeliness committee checks, as about 3 s. Today’s mainnet meets it largely because payloads are small, and because datacenter builders and high-bandwidth nodes carry them. A builder at home, such as a solo staker building its own payload, has a home connection’s uplink, and every copy of its payload starts there. Today’s whole message spends that uplink badly, and at 1 MiB a home builder’s payload arrives seconds after a datacenter builder’s. If only a builder in a datacenter can publish in time, building moves to datacenters.
We want a protocol that works even when every node, the builder included, is on a home connection, which we put at 50 Mbps up and 100 down, what EIP-7870 recommends for a local block builder: the all-home network. We ask for this not for its own sake: a protocol that works in these conditions gives Ethereum, the protocol rather than today’s deployment in good conditions, real resilience and decentralization. We also study the mixed-bandwidth network, closer to today: the same network with a fifth of the nodes in datacenters at 1 Gbps. We run it with a datacenter builder, at 1 Gbps, and, between the two, with a home builder: a home builder in a network that is otherwise today’s.
The previous post introduced three tiers of the protocol, and closed with what remained open: control traffic had grown with the number of segment ids. Partial messages (variant B) address that with bitmaps, but in the previous post variant B was, in all three scenarios, about a fifth slower than variant A, which sends the chunks as ordinary messages on one topic. Nothing kept segments on independent paths after their first hop, and one topic’s mesh limited what a fast node could contribute; per-segment topics (variant C) were one way around both, and with a datacenter builder they were faster than tier 3. And queue management, priorities and timers had not been looked at. Tier 4 answers most of what it left open on one topic and one mesh, by asking what each node could do better with information it can have cheaply.
In this post we use the term chunk for the unit on the wire, as EIP-8411 does. The earlier posts said segment, which can get confusing once compression, segmentation and coding are combined in various ways to derive the pieces.
2. Four tiers
Each tier of Informed Diffusion answers a different question.
Tier 1, pipelining: why is the whole message slow? Because every hop stores the payload before it forwards, and because the source starts eight copies at once on one uplink, which from a home link delays every one of them. Cut the payload into chunks that a node can verify alone, against a commitment in the bid, and a hop forwards a chunk as soon as it checks. Have the source queue a first copy of every chunk before a second copy of any, so that on a home link the last chunk does not wait for eight copies of the rest. Latency falls to a fraction. This is the tier EIP-8411 advocates for: at revision 41ba25cc it specifies the chunks, 64 of them with their root in the bid, in its own encoding, and leaves batch publishing to the publisher. This tier carries most of the gain, and it is where the bid and the message format change. Tier 3 asks one more thing of the bid: a commitment that also states the count and the length, and covers the parity chunks; tier 4 asks nothing more of it.
Tier 2, push and pull: why does tier 1 cost so many bytes? Because every chunk is pushed to every mesh peer, as the whole message was, and most of those copies are duplicates. We know that notify-and-pull, gossipsub’s IHAVE and IWANT, avoids most duplicates, at a latency price. In other words, a push can be seen as a pull with the round trip removed and a duplicate risk added. Thus, a better approach is to push where a duplicate is unlikely and announce where it is likely: push a chunk to a few peers early in the chunk’s life, announce it to the rest, and let them ask. Bytes fall to a third, and the tail shortens too.
Tier 3, the code: why is the tail of the delay distribution still long? Because a node waits for its last chunk, and the last chunk comes down the slowest path. Code the payload so that any K of 2K chunks rebuild it, and no chunk is the last one: whichever K arrive first are enough. The tail shortens (although bandwidth utilization increases by about 25–45%).
Tier 4, informed scheduling: what can a node do with what it can cheaply know? Three things:
With a compact picture of what each neighbour holds, it can cancel pushes that became duplicates, send the rarest chunk first, stop early, and aim its pushes at the peers who need them.
The source, the one node that holds everything at the start, can spread its first copies wide when its uplink is fast; on a home link it matters more that it keeps its backlog where its own rules still reach it, and answers less of its own gossip.
A forwarding node with uplink to spare can push more, because a duplicate is cheap there.
None of this needs new topics, new meshes or a change to gossipsub’s mesh construction, degree or scoring.
3. The cards
Figure 1. The cards added one at a time, in the all-home network (blue), and on the mixed-bandwidth network with a home builder (green) and with a datacenter builder (red). Horizontal: receiver p99; vertical: bytes received per node, data and control, expressed as copies of the payload compressed whole; bars: one standard deviation over the simulated networks. Lower left is better. The key names every card; the large points are the tiers. Card 17 changes nothing in the all-home network, where no node is fast, so the blue line ends at 16.
Figure 1 is the design in one picture: each point adds one card to the ones before it. The first six steps are the previous post’s ladder. The accompanying card reference describes every card in detail, with its rule, the problem it answers, what it needs and costs, when it matters and where the idea comes from. In this post we summarize each card’s logic in a line or two.
Tiers 1 to 3, from the previous post
Segmentation (1). The builder cuts the payload into chunks, commits to them in its bid, and publishes each as an ordinary gossip message; a node forwards a chunk as soon as it verifies it against the commitment.
Batch publishing (2). The source queues a first copy of every chunk before a second copy of any, so the chunks start on different paths. It matters most where the builder’s uplink is the bottleneck, as at home.
Announce instead of push (3). A node pushes a chunk to a few mesh peers and announces it to the rest, who ask if they still need it.
The phase (4). The push budget for a chunk shrinks with every mesh peer already known to hold it: push early in a chunk’s life, announce late.
Disciplined pulls (5). Ask one announcer at a time, move on after a timeout, and stop asking a peer for this payload once it breaks its promise.
The erasure code (6). Compress, then code into K data chunks and K parity chunks so that any K rebuild the payload; stop asking once it decodes.
Tier 4: knowing who holds what
HAVE bitmaps (7). One bitmap per payload per peer, carried in gossipsub’s existing IDONTWANT and IHAVE messages, replaces one message id per chunk: a peer’s whole holding in a few bytes, and a current picture of every neighbour for the cards below. FullDAS (2024) proposed it for gossipsub, and the partial-messages extension, the previous post’s variant B, is built on it.
Cancel at dispatch (8). Drop a queued chunk, a push or an answer, at the moment it would be sent, if the peer has meanwhile said it holds it.
Rarest at pop (9). Choose which queued chunk goes next as late as possible, at dispatch: when several chunks wait for a peer, dispatch first the one the fewest neighbours hold.
Decode claim (10). A node that can decode says at once that it holds every chunk, so pushes still queued toward it are cancelled; once it has checked the group, it pushes the chunks it rebuilt to the mesh peers still short. The measured stack has the claim alone.
A push budget of two (11). With the code a receiver needs any K of 2K chunks, so at 1 MiB the push budget can be reduced to 2 instead of 3.
Projection (12). Skip a push to a peer projected to reach K chunks without it, on the assumption that another pusher is as fast.
Together these cards take away a third of the bytes tier 3 carried and more than half of its control traffic. They do not make a clean, uniform network faster, since the bytes they save were not on anyone’s critical path, but where uplinks are uneven the slow nodes stop carrying copies nobody needed. Part of this resembles what variant B (partial messages) had over A: using bitmaps to share per-link state of who holds what, but used here without paying the price of an exchange before every pull. Tier 4 has a fast path with pushes, which wait for no exchange, and uses the bitmaps to prune, order and aim them.
Tier 4: the source
Striped publishing (13). A source whose uplink is well above its peers’ spreads the first copies over all its connections, each peer getting a different stripe of the chunks, so the payload starts from seventy points instead of eight. This is the reach that per-segment topics (variant C) gave the source, but achieved on a single topic and without the standing meshes. A source at its peers’ speed, such as a home builder, does not gain from the stripe.
The static window (14). Each node limits the chunks it has in flight to a fixed window sized from its own uplink, and keeps the rest in its own queues, where cancel, rarest at pop and projection still apply; the receiver’s bitmap, sent back to the sender, is the acknowledgement. Without it, a wide first hop shares one uplink among seventy transport streams and the chunks that matter wait in the transport, out of the protocol’s reach. On home links it binds mainly at the builder, and matters the more the thinner the links.
Paced gossip (15). Gossipsub’s heartbeat announces new ids to a sample of peers all at once. When this announcer is the source, whose uplink is busy with its first copies, this can add many copies in parallel: in whole-message diffusion a home builder answers its own gossip with dozens of copies of the payload. Pacing releases the announcements one peer per short tick, so fewer peers ask the source before a push reaches them.
A builder with a fast uplink, such as one in a datacenter, answers many requests for its own payload. It may instead push its first copies and announce to nobody, leaving the requests to the other nodes: its upload falls to about a quarter at the same latency. We treat that as a mode the source can choose, not a property of the protocol.
Tier 4: aiming the pushes
Furthest behind first (16). A node pushes to the mesh peers that hold the fewest chunks of the payload, counting the pushes already queued to them: the peers for whom a push is least likely to be a duplicate, and who would otherwise have to ask. Because the rule trusts what peers say about themselves, a peer that under-reports would draw every contested push, so the rule draws at random among the lower half of the ranking and demotes a peer that takes pushes but reports nothing.
Push degree by uplink (17). Forwarding nodes with a fast uplink start from a push budget of six instead of two: a duplicate is cheap on an abundant uplink, so push more there instead of answering requests a round trip later. The cost on the downlink still has to be paid, but bandwidth asymmetry makes it worth it.
4. What it comes to
all-home network
home builder, mixed bandwidth
datacenter builder, mixed bandwidth
p99
copies
p99
copies
p99
copies
whole message (tier 0)
stock gossipsub
6.3 s
4.4
4.2 s
3.4
1.8 s
3.4
whole message (tier 0+)
paced gossip, three copies in flight
2.7 s
3.6
2.3 s
3.2
1.8 s
3.4
tier 1
chunks, batch publishing
1.26 s
6.5
1.03 s
6.2
0.87 s
5.7
tier 2
announce, the phase, disciplined pulls
0.87 s
2.0
0.83 s
2.0
0.73 s
2.0
tier 3
the erasure code
0.71 s
2.5
0.66 s
2.6
0.55 s
2.8
tier 4
state, the source, aimed pushes
0.63 s
1.7
0.58 s
2.1
0.41 s
2.1
Table 1. Warm receiver p99 and bytes received per node by the network’s completion, in copies of the payload compressed whole, 500 nodes, 1 MiB: the real Prysm and go-libp2p-pubsub code over a simulated network on a virtual clock. A node counts as done when it holds the payload, or enough chunks to rebuild it; decoding, the checks that follow and any wait for the block are outside the times. A home builder is at 50 Mbps, a datacenter builder at 1 Gbps. The tier rows are Figure 1’s large points; the second row is the whole message with paced gossip and an acknowledged cap on copies in flight. The tier-4 rows ran with every source striping; for a home builder, not striping ties with it at this size. The companion note has all the details.
Read across, the table is the home builder’s story. With today’s whole message, a home builder on the mixed-bandwidth network is more than two seconds behind a datacenter builder in the same network. At every tier it is less than a fifth of a second behind, and each tier makes both faster. At tier 4 a home builder’s p99 is about a third of a datacenter builder’s with the whole message, with or without datacenter nodes around it.
Tier 4 is faster than every earlier tier in all three scenarios and lighter than tier 3; in the all-home network it is the lightest of all. Against variant C with the code, the fastest mapping at a datacenter builder last time, it is faster and lighter too: the regime the previous post gave to C belongs to the single topic now.
The second row of the table (tier 0+) is a protocol improvement we could make even without a hard fork. Two of tier 4’s rules carry over to plain gossipsub: paced gossip, and the window counting whole messages, acknowledged by the receiver’s “don’t want”, gossipsub’s IDONTWANT, returned to the sender, a behaviour its peers must share. They take the all-home network’s whole-message tail to about twice tier 1’s time without a new message type. Tier 0+ cannot compete with what segmentation enables: the start still shares the uplink between whole copies, and every hop still stores before it forwards.
What changes on the wire. Beyond tiers 1 to 3, only the HAVE bitmap and the ask by index that comes with it are new on the wire, and both ends must support them. The bitmap also serves as the decode claim and as the acknowledgement the window counts. Everything else in tier 4 is a local decision of the node that makes it: cancel, rarest at pop, projection, the push budget, striping, the window’s size, pacing and whom to push to. A client can adopt them one at a time.
Under stress, tier 4 holds inside the 3 s budget in every case we ran: loss of data or requests, withholding nodes up to half the network, nodes that also forward or acknowledge nothing, a source that omits as many chunks as the code can spare, every uplink thinned to 10 Mbps, uplinks spread widely around the same mean, and payloads from 128 KiB to 2 MiB.
Some ideas did not make it to the tier 4 stack: an adaptive window, estimating the link’s capacity from the traffic, a lower-rate code, and keeping chunk paths apart; the companion note explains each in detail, including measurements, and says what would reopen them.
5. Where the rest is
This post cannot contain all the details; it comes with several accompanying documents and code:
The card reference is the place to read to learn more about any specific card: its rule, what goes wrong without it, what it needs, its parameters, its costs and risks, when it matters, and pointers to literature where the idea comes from. It also studies what would happen without it: a table of tier 4 with one card removed at a time.
The companion note, Informed Diffusion, measured, provides all the measurement results, card by card. It also details protocol choices such as when striping and the window pay. It runs tier 4 through various stresses, and provides the negative results with their conditions.
Informed Diffusion: the protocol in pseudocode contains an early version of an executable specification of the cards and Informed Diffusion as a whole. Moreover, the repository also contains a toy runner to execute that specification, allowing comparison against the Go implementation.
6. Closing
Informed Diffusion brings large gains for a datacenter builder too, but the story is about the home builder, whose uplink is much less of a bottleneck than today, and who can compete on more equal terms.
Informed Diffusion, as measured here, was built as an extension of gossipsub, in go-libp2p, and integrated in Prysm. However, the protocol is not bound to these: the executable specs run its rules without go-libp2p or Prysm. It can be used over various substrates: we measured it over libp2p over QUIC, and the earlier tiers also ran over libp2p over TCP under Shadow. It can also be built natively on top of QUIC (or datagrams in general), which enables various interesting cross-layer optimization possibilities. Our earlier papers, writeups and talks already discussed some of these, such as datagrams for the unreliable transport of erasure-coded chunks (Data Availability Sampling from the Networking Perspective, 2023), a lightweight UDP request for fast sample retrieval (FullDAS, 2024), rate-controlled chunk transport over UDP in peer-to-peer streaming (A performance comparison of hose rate controller approaches for P2P-TV applications, 2014), or network measurements taken for free from a DHT’s own UDP packets (Crawling the Ethereum discv5 network - fast, 2024); they are still interesting, just not the focus of this post.
Informed Diffusion is inspired by the FullDAS design, but not just that. It also takes inspiration from a very long history of P2P networking research (see the card reference for references card by card), and applies it to Ethereum’s payload broadcast.
Informed Diffusion is not just for Ethereum payloads. It could serve the many large objects we foresee introducing in Ethereum in the near or more distant future: block access lists (BALs, EIP-7928), for which EIP-8411 already mentions a possible bal_chunks topic, consensus blocks with post-quantum (PQ) signatures, payloads carrying SNARK proofs of their transactions’ signatures, block execution proofs, or even individual large transactions. These are all different in their requirements, but each can most probably find the mix of cards and parameters that serves it.
In FullDAS, diffusion is sampled, while in this post it is not. Yet the broadcast is just a special case of the sampled diffusion we designed in our FullDAS posts: the case where every node’s interest is the whole payload. The more generic case, sampled diffusion, is the harder problem. A follow-up post will revisit sampled diffusion, and show how a unified protocol can handle both.

Схожие новости

#	Наименование новости	Тональность	Информативность	Дата публикации
1	EIP-8411: what segmented payload diffusion is made of	0	8.61	17-09-2026
2	PQ anonymity for Stealth Address Protocol	0	10.76	10-10-2026
3	Rapidblocks, take two: a pipelined consensus for independent blocks	0	5.78	13-10-2026
4	Public-mempool gas sponsorship needs escrow, a bond, or trust	0	6.46	14-09-2026
5	Scaling Ethereum with recursive STARKs and the Trustless Log Index	0	6.14	15-09-2026
6	Rapidblocks with split proofs: execution proofs that outlive their state	0	5.9	14-10-2026
7	All Core Devs - Consensus (ACDC) #183, July 23 2026	0	15.18	13-07-2026
8	EIP-8334: Bundled Attestation Propagation	0	13.2	14-07-2026
9	Cohort Order Book: an O(1) fully on-chain limit order book via generational fungible liquidity	0	12.86	15-07-2026
10	Proposal for a minimal compute-anchored purchasing power signal	0	7.88	11-10-2026

Классификация: . Схожих патентов: 0. Схожих новостей: 10. Тональность: 0. Информативность: 9.3. Источник: ethresear.ch.