Upgrading Finality - Edition 1

Ben Edgington¹

---

Ethereum is strongly placed to become the global settlement layer: it is the most credibly neutral, trust-minimised, reliable, permissionlessly available platform in the world. But true transaction settlement – finality – remains relatively slow. That slowness degrades user experience, forces platforms and applications built on Ethereum to use complex workarounds and risk mitigations, and imposes an economic friction that reduces the efficiency of capital.

This update is the first in a series on how we plan to improve Ethereum's time to finality, cementing its place as the settlement layer for the world.

Feel free to skip the background and jump to the The Plan if you already know your finality from your flibbertigibbet.

Background

Informally, finality is the property that what goes onchain stays onchain. Pure proof of work blockchains, as Ethereum used to be, do not have this guarantee - they are always vulnerable to a powerful attacker rebuilding the chain and reverting any number of past transactions, reordering them or erasing them from history entirely. This is how the classic "double spend" attack is set up, for example.

By contrast, the validators on a proof of stake chain are able to agree a point in the chain before which they will never revert blocks. The entire chain up to that point is then immutable, and you can completely rely on its history (absent any social consensus to intervene and rewrite history, which has never happened on Ethereum).

Ethereum under proof of stake comes with two specific guarantees around finality.

The first is that, if less than one-third of stakers are hostile (in the sense that they want to attack the chain's integrity), then finality is absolute. There will only ever be a single, irrefutable finalised chain history.

The second is that, even if attackers control more than one-third of the stake, and they succeed in finalising two conflicting histories, they will suffer enormous slashing penalties: at least one-third of all staked Ether, which, at the time of writing, is almost 13 million ETH.

Finality is a powerful property to have. It's what makes Ethereum capable of securing many billions of dollars worth of assets.

The Hazards of Unfinality

For a deeper dive into the importance of the available chain, see this article by Luca Zanolini.

On Ethereum, we prize availability. The chain will keep processing transactions and adding blocks even while finality follows far behind or stalls completely. This is a design choice, and resembles the situation under proof of work where the chain progressed even though it never truly finalised.

Availability is great for uptime and user experience, but it leaves the most recent part of the chain – between the last finalised block and the current block – vulnerable to being re-organised (reorged) either accidentally, or deliberately. Today, when things are running normally, somewhere between 63 and 95 of the most recent blocks are unfinalised. And when there is an issue with the network, the unfinalised tail of the chain can become much longer. This unfinalised tail is theoretically vulnerable to attack, though forcing a long reorg is extremely hard to pull off.

Accidental short reorgs are not usually a big problem, and can happen when there is a little network disruption or a late block. Usually, the transactions that were reorged out of the chain go back into the mempool and get automatically re-added in another block, and there's no significant impact (though this can play havoc with private mempools).

Though very hard to achieve, a deliberate reorg can be more troublesome. A powerful attacker might try to do things like steal MEV transactions from another proposer, temporarily censor transactions (perhaps oracle updates), manipulate the protocol's RANDAO giving the attacker an oversized influence, and even, in the most extreme case, perform a double-spend attack.

In short: being vulnerable to reorgs is bad, and the longer that the tip of the chain is vulnerable, the worse things can be.

Because finality currently takes so long, applications on Ethereum often use heuristics and workarounds to minimise the impact of any reorg while not waiting for full finality.

For example:

• Centralised exchanges usually wait for a certain number of block confirmations before recognising a deposit. This is a compromise between risking a deposit being reorged and waiting all the way until full finality.
• Similarly, OP stack-based layer 2 (L2) chains often wait for only a few slots before they recognise deposit transactions rather than waiting for full finality. So, an Ethereum L1 reorg of several slots could lead to a reorg on the L2.
• Arbitrum L2s wait for "unrealised justification" to happen on L1, which is a step towards finality but lacks its full guarantees. Nevertheless, it can be reached 8.5 minutes quicker than full finality.
• Circle's latest Cross-chain Transfer Protocol (CCTP) version has two modes. The standard path waits for Ethereum finality, taking around 15 minutes to make a transfer. The fast path waits for only two block confirmations, about 20s, but limits the total amount of USDC in flight to reduce risk in the event of a reorg, as well as charging a fee.

All of these could be simpler and more secure if finality were faster. And, of course, end-users of these platforms will benefit too! I expect that the most visible benefits of fast finality will be in fast and cheap cross-chain bridging and interop, making our many-chain reality much more seamless than it is today.

Most of the time, the new fast confirmation rule will help enormously. When the network is running smoothly, and any potential attacker is not too powerful, it will very quickly (in about 13 seconds) be able to confirm that a block will not be reorged. For most purposes, most of the time, this is a huge upgrade in user experience. However, fast confirmation comes without the full guarantees of finality: it is not tolerant of network delays, and is not backed by economic security. When there is network trouble – just the moment when you need those guarantees – out of caution it will fall back to finality, putting safety over 13 minutes away again.

Time to Finality

For all these reasons – to reduce the amount of the chain that is vulnerable to reorg and the consequent risks for applications – we need to reduce Ethereum's time to finality (TTF).

To reach finality, the chain has to process votes representing at least two-thirds of the stake, which is roughly two-thirds of the validators - this is a well-known result in distributed systems design and not unique to Ethereum. That means collecting well over 600,000 votes, aggregating them down to a minimal size, and inserting them into blocks.

In fact, this has to be done twice. The design of Casper FFG (our current "friendly finality gadget") requires two rounds of voting: the first to "justify" a checkpoint on the chain, the second to finalise it. Casper FFG pipelines these votes so that we can declare a finalised checkpoint every round, but it still takes two full rounds to reach finality.

In Casper FFG, we call a voting round an epoch: an epoch is 32 slots long, and a slot is 12 seconds long. Therefore, the current minimum TTF is 2 × 32 × 12 = 768 seconds (actually 12s less since a checkpoint is the first slot of an epoch, but close enough). And the average (expected) TTF for a transaction is another half an epoch, giving 960 seconds.

It's fair to say that the current design is a conservative choice, and it's been in place since the beacon chain's genesis in December 2020. The question is, how much better can we do?

The Plan

Fast finality is a complex mix of inter-related challenges: networking, validator set management, validator consolidation, attestation aggregation, interactions with slot length, consensus algorithm design, the relationship between the available chain and the finalised chain, to name a few.

Past attempts at getting to fast finality (one-slot finality, three-slot finality) have foundered on being essentially all-or-nothing: they required that we solve everything before we can solve anything. And these are hard problems; solving them all at once would be a huge challenge, possibly taking years.

Our new approach breaks down the challenge into a series of smaller upgrades, each of which makes finality faster. First, we will decouple finality from from the rest of the consensus process. This is the big unlock. That, then, gives us the freedom to incrementally improve time to finality over subsequent upgrades, by working individually on each of the things I mentioned two paragraphs ago.

An important benefit of proceeding in stages is that it gives us the opportunity to measure and optimise at each step. This de-risks delivery and helps us to deliver a more efficient protocol while making fewer assumptions.

Decoupling Finality

For a deeper dive into decoupling finality, see this article by Francesco D'Amato.

One of the things that currently hampers our quest for faster finality is that, today, finality votes are tightly coupled with fork-choice votes.

To unpack that, in Ethereum, we have things happening on two different time scales. The faster time scale is the slot-by-slot block building. In every slot there is a designated block proposer. That proposer must decide which block to build its block of transactions on top of. Usually this will be easy and obvious. But if there has been some network delay, or if an attacker has published multiple blocks at the previous slot, for example, we need a way to select the best "head block" from among the candidates. This is the role of the fork choice, which in Ethereum is called LMD GHOST. Fork choice is heavily influenced by votes received from validators within the last few seconds, which are from a smallish subset of the whole validator set.

Alongside the slot-by-slot fork choice, we have the much slower finality process described above, called Casper FFG, that aims to gather votes from the whole validator set. Together, these two have become known as Gasper.

The point is that our current design couples the votes for these separate processes into a single structure called an attestation:

class AttestationData(Container):
    slot: Slot
    index: CommitteeIndex
    # LMD GHOST vote
    beacon_block_root: Root
    # FFG vote
    source: Checkpoint
    target: Checkpoint

You can see that the attestation contains both the short timescale LMD GHOST vote and the long timescale Casper FFG votes. This tightly couples the slot-by-slot progress with progress towards finality, limiting our scope for achieving faster finality without making significant changes to the structure of a slot. Slot structures are already quite complex, and getting more so with ePBS, even while there is a drive to make them shorter. In fact, decoupling finality could be a huge help in delivering shorter slot times: currently we use attestations from about 30,000 validators per slot to drive the fork choice; the tight coupling basically forces us to do this. But it would actually be fine to use many fewer, say a committee of 256 or 512 validators whose votes would be much faster to gather.

Therefore, the first thing we plan to do is to decouple finality from the slot-by-slot fork choice. In essence, validators will make separate votes for finality and fork choice that will traverse the network independently and be processed independently.

As discussed in Francesco's article, there will still be a finality gadget, similar to Casper FFG, but potentially using a slightly different consensus algorithm. However, unlike FFG, this finality gadget will work on its own timescale, using all the available bandwidth, not constrained by the slot-time and current epoch lengths.

I expect that decoupling alone will give us a nice boost in time to finality. Looking at today's network we can see that attestation propagation is quite bursty and only takes up a fraction of the available time in the slot. This is driven by the need for as many of the LMD GHOST head votes as possible to reach the next block proposer in time for it to run its fork choice. When finality votes are decoupled from head votes they will be able to make use of more of this currently idle bandwidth, and will be able to collect them more swiftly.

There are likely some other easy optimisations we can make to finality vote aggregation and distribution that will give us a low single-digit finality speed up even at this first decoupling step.

Speeding Finality

But a low single-digit speedup is not our ultimate goal. Once finality has been decoupled, we then have scope to incrementally improve time to finality with minimal impact on the rest of the beacon chain.

We plan to tackle this on several fronts. We can deliver these improvements together or separately; the roadmap is very flexible. The following gives a flavour of current work in progress, and other approaches might emerge in due course. Please note that this is not exactly a roadmap - at this point it's a collection of things that we might or might not do depending on further investigation.

The swiftest and simplest path to fast finality is to drastically reduce the number of validators on the network. The good news is that there is a way to do this with no impact on validator diversity. For historical design reasons, big staking providers run many 32 ETH validators. Since the beacon chain's Electra upgrade it has been possible to consolidate a batch of up to 64 validators into a single validator, hugely reducing the overheads of finality voting. If large operators were to optimally consolidate all their stake, then we would already be most of the way to finality in seconds. Unfortunately, consolidation progress is slow. If this does not improve, we might want to look at ways to incentivise consolidation or penalise non-consolidation.

A lighter-weight alternative to consolidating validator stake might be to batch attestations at origin. Like consolidation, this recognises that many validators managed by the same large entity might sit behind a single beacon node. Rather than broadcasting each validator's vote individually, this proposal aggregates them up before they hit the network, with a potentially large saving on bandwidth. This is work in progress (removing slashing conditions is a priority to ease adoption). A similar but alternative approach might be NetClusters.

Another way to reduce the overhead from having so many validators is to do validator set rotation with something like the Orbit approach. No validators are excluded, and the number of validators is not capped, but only a subset of the validators is participating in finality at any given time.

The peer-to-peer networking layer is a rich vein of potential improvements. Broadcasting and aggregating the separate finality votes is one target for optimisation; reducing the bandwidth of other activities (like block transmission) to allow more space for finality votes is another. Many such improvements are underway as we work towards ethp2p - I hear rumours of huge developments in the pipeline.

Part of the networking aspect is aggregating all of the individual attestations into much smaller units. The beacon chain's present approach is not as efficient as it might be because we have no way to combine two aggregated attestations that both contain attestations from the same validator - we can only combine completely disjoint aggregates. Potentially we can remove this inefficiency by bringing forward SNARK aggregation from the Lean Ethereum work.

Outside of networking and validator management, a very interesting innovation on the roadmap is single round finality. As discussed above, classic finality protocols require two full rounds of voting, which gives us the maximum possible safety margin of 33% hostile stake. If, however, we are prepared to lower our fault tolerance (perhaps to 20% or 17%), we can achieve finality after only one round of voting, almost halving TTF. Given the amount at stake today, this safety margin reduction could be quite acceptable.

Finally, over time, we can generally expect global infrastructure to continue to improve: bandwidth trends upward; latencies trend downward. In due course we should be able to revisit our Hardware and Bandwidth Recommendations for stakers, without excluding any existing participants.

Each of these things alone might give us a low single-digit speedup in finality. Deploy, say, four improvements that each give us a 3x speedup, then we've practically reached the 100x overall speedup that we're looking for. This incremental approach resembles Ethereum's successful incremental delivery of higher gas limits and higher blob limits on the execution and data layers.

Skeletons in the Closet

There is one more reason why it would be good to overhaul Ethereum's finality gadget. The truth is that LMD GHOST and Casper FFG have never played perfectly nicely with each other. Ethereum was the first blockchain to attempt anything like the ebb-and-flow model of combining a dynamically available chain providing liveness with a trailing finality gadget providing safety. By and large it has been very successful, but, like all first-attempts, it's rough around the edges. The interaction between LMD GHOST and Casper FFG has been a rich source of issues, requiring various fixes and hacks over time, and remains more complex and less secure than we would desire.

The source of many issues is the ability of an attacker to store up attestations and release them at strategic times of their own choosing. Moving away from LMD GHOST towards a "memory-less" fork-choice protocol such as Goldfish fixes this. The switch to a decoupled protocol is an ideal opportunity to change things up.

Conclusion

Delivering faster finality on Ethereum has been an aspiration pretty much since The Merge. My goal in this article has been to outline an actual, practical roadmap towards making aspiration reality: first decouple finality from slot-by-slot fork choice, and then incrementally improve time to finality as a broad range of further improvements get deployed onchain. In future editions I will dive more deeply into concrete progress, the technical details, and actual numbers - consider this an orientation on our direction of travel.

My personal goal is to achieve fast-enough finality without compromising the diversity of the validator set. That is, without excluding home and solo stakers and stakers in remote locations. It remains to be seen whether we can get there: at some point we might need to weigh the merits as a community of even faster finality at the cost of reducing staking diversity. But I first want to see how well we can do without compromise, since validator diversity is the most important differentiator between Ethereum and all the other Alt-L1 chains.

Wen fast finality? As a stretch goal, the Strawmap places decoupled consensus as a headliner candidate for the I* fork, likely sometime in 2027. There remains a lot of theoretical and practical work to do. We don't yet have a fully-fledged design for decoupled consensus, although progress is strong. There are some knotty problems remaining like how to handle the inactivity leak (aka accountable liveness), how to rework our slot-by-slot consensus, and how to be post-quantum compatible among many other things. There's a lot of work involved in formally proving the new protocol, and, of course, getting it implemented in clients and fully tested on devnets and testnets. But some of the smartest minds in Ethereum are focused on getting us there.

Many thanks to Barnabé, Mikhail, Roberto and Vitalik for review and helpful feedback.

Footnotes

1. This article represents my somewhat informal personal view as cat-herder for fast finality rather than any "official" plan or roadmap. I am grateful to all the incredible Ethereum researchers and devs who so freely share their work. Everything is, of course, subject to Ethereum's governance process. ↩