Last Irreversible Block: Huge Hive Upgrade

@blocktrades and friends just keep pumping out sweet code.

Color me impressed. I forgot this was even happening until I read @taskmaster4450's post on it yesterday.

Link to OP, May 8, 2022

In a normally well-connected Hive p2p network, this should result in most blocks becoming irreversible on a node within a second or less after they are produced. The exact time required depends on the message latency between nodes and number of network hops between the node and the block producers.

WEIRD!

So it sounds like this solution is completely off-chain (but also on-chain because the code is being run directly on the Hive nodes). Basically Hive nodes will talk to each other without putting that information into the blocks and come to a kind of gentlemen's agreement about whether blocks are valid before they become technically irreversible. Then (correct me if I'm wrong) the node API will act as though the block is irreversible even though it technically might not be. This is largely speculation on my part, as I haven't even read @blocktrades post on it yet.

So lets quote some of @blocktrades technical jargon:

Block producer 1 (the block producer scheduled to produce the next block) generates and broadcasts a block to the p2p network.

Other nodes receive this block and temporarily apply it as the next block in their local copy of the blockchain to test if the block is valid. If the block is valid, the node’s local state will be updated with the transactions contained in the block. If the block is invalid, the node will roll back the changes made by the block to their local state. So far, this is how the DPOS protocol currently works.

New OBI step: If the node is one of the scheduled block producers, the node signs and broadcasts a new type of p2p message call a block_validity_vote if it considered the block valid and made it the new head block for it’s local copy of the blockchain. This message, signed with the block producer’s signature, contains the block producer’s name and the block id of the newly applied block.

New OBI step: Each node will keep a temporary buffer of the valid block_validity_votes it receives (and also propagate these votes to their peers using the normal p2p rules for message propagation). If a nodes receives the required ¾ majority of distinct block producer votes for a block, that block can be marked as irreversible and written to its block_log.

It's funny how important this rollback procedure is even though it hardly ever gets used.

Without being able to roll back bad blocks, the blockchain becomes worthless. So even though it hardly ever happens, the process is extremely important. The main question we have to ask ourselves is: does this new one-block irreversibility open the Hive network up to new attack vectors?

The answer is: yes?

So imagine a tiny cluster of top 20 witnesses decided to get together and run code that claimed a block was irreversible, but then were like "just kidding" a few blocks later and voted to remove the block?

Again, correct me if I'm wrong, but it sounds like this is possible given my understanding of virtual operations on Hive. That's because all the information used to make a Hive block actually irreversible isn't contained in the blocks themselves, but rather determined virtually by each node independently based on network consensus rules.

Thus this code to make blocks irreversible in under a second can easily become a problem, because some Hive nodes will have data signaling that a block is unchangeable, and some will not have that data. It will only become officially irreversible using the old rules of the system. Of course I'm totally talking out my ass and it might not work like that at all.

Each node will keep a temporary buffer of the valid block_validity_votes it receives (and also propagate these votes to their peers using the normal p2p rules for message propagation). If a nodes receives the required ¾ majority of distinct block producer votes for a block, that block can be marked as irreversible and written to its block_log.

So, if the network has these signatures saying the block is valid... I guess it would be possible for the Hive network to actually not allow those nodes to backtrack and say "just kidding" a few seconds later. The devil is in the details. I trust that @blocktrades will implement the solution in the manner that has the least attack vectors (or hopefully none at all).

Why are irreversible blocks important?

The most important thing about all irreversible blocks is eliminating the double spend attack. On Bitcoin, there is no such thing as an irreversible block, but it becomes so difficult to reorganize the chain after 3 block confirmations that basically all centralized exchanges only wait for 3 (~30 minutes) before unlocking the money to their users. Meanwhile, on Hive, an irreversible block is basically set in stone in less than a minute, which is pretty cool on a technical level compared to other solutions.

This upgrade could cut that minute down to a fraction of a second, which would be quite the spectacle on a technical level. Imagine sending Hive to an exchange and just... getting the money instantly and not having to wait. That's a user experience (UX) that literally nobody gets in crypto, and it completely eliminates the worst part about crypto: Transfer Anxiety (which is a post I should have written months ago but just haven't yet). I guess I'll have to make that my next post.

So what's the real threat?

It's basically impossible to do a double-spend attack in under a minute, especially on a centralized exchange. It's not really possible to send Hive to an exchange, trade the Hive for something else like Bitcoin, and then cash out and rollback the chain before the centralized exchange stops you from doing that. There's a reason why centralized exchanges don't just process withdrawals instantly even when it is technically possible.

The real threat of rolling back irreversible blocks (which by definition is probably impossible even after this upgrade goes though) is that it might mess up the second layer of Hive. If a dapp gets information from a Hive node saying the block is irreversible and then proceeds to act on that information and the block gets rolled back... that could be devastating to that particular second layer depending on the circumstances.

Even if it could happen, would it?

Highly highly unlikely that even if bugs exist in this code that they would be exploited. The exploit would be obvious and the witnesses that perpetrated it? Well, they'd be kicked to the curb within 24 hours. There's almost no possible way such a hack would be worth losing the reputation of being a top 20 witness on Hive. That's the beauty of these systems. The incentives align so even if people at the top can cheat, they don't because it's simply not financially viable.

At the inception of the idea, the design behind One-block Irreversibility included storing the approval vote messages into the next block as a means of proving irreversibility of the prior block. But this adds unnecessary bloat to the size of blockchain, because the existing mechanism for proving a block is irreversible already works well for all but the most recent blocks.

Ah so they were actually thinking about putting the info into the blocks. This virtual solution is much cleaner. Nice work.

until they have received a sufficient number of follow-on blocks that build off the block. At that point, the block_validity_votes for that block can be discarded.

Discarding worthless data is the best in the world of inefficient blockchain solutions.

New votes by a block producer override its old votes at a given block number

Nodes employing the OBI protocol only track the most recent vote cast by each block producer. So if a block producer switches to a different fork, all the votes it cast for blocks that will be discarded during the fork switch will be “overwritten” by the votes it casts for the new blocks at those block positions. In other words, at any given time, every node will only consider one vote by a specific block producer at a specific block position.

_{references to "votes" are block_validity_votes
Ah this was the information I was looking for.
So it looks like block producers can change their mind whenever they want, right up until blocks become actually irreversible. It is still somewhat unclear to me in that it sounds like irreversible blocks could be reversed, but I guess I just have to assume that's not the case.

basically how DPOS 1.0 treated subsequent blocks as votes for the previous blocks.

This statement is key to understanding how the system works right now before the upgrade. The reason it takes 1 minute for a block to become irreversible is that a super-majority of witnesses need to put their block onto the chain without rolling back the block in question. By not rolling back the block in question, it is implied in a virtual manner that they must accept the block is legitimate (because they didn't roll it back).
This upgrade jumps the gun and says, "Hey, why are we waiting a minute to figure this out? Let's just do it right now behind the scenes." Once 16 witnesses sign the block as a valid block (in under 1 second), it will become permanent. If a group of witnesses then tries to backpedal and say "just kidding", then all the other witness can deny them that right using their own signed signature against them. Pretty cool.
And concerning attack vectors
Anyone who reads @blocktrades post on the matter can see that he clearly knows a bit more than I do about the situation, coming up with examples of fractured internet and witnesses being voted in dynamically and messing stuff up. So obviously I'm a bit out of my element trying to do the game-theory myself. But it's fun to try.
Centralized exchanges may end up using this code by default.
It is possible that exchanges like Binance and Bittrex will upgrade their node to the next hardfork, and this irreversible code will immediately start being used after wallets come back online. That would mean we could send hive to exchanges and the money would be available instantly.
On the backend, the exchange is running their own Podunk Hive node, and they are probably using code that finds irreversible blocks and doesn't credit user accounts until it's irreversible. Assuming this is the case, after the next hardfork... Hive will essentially become irreversible instantly... and thus we will get our money instantly. Again, no other crypto in the world can do this. Even just as a novelty, it's a bad ass upgrade, but it's no much more than a novelty upgrade (even though it might seem unnecessary).
Put quite simply, we live in a culture that values instant gratification, often times to a fault. If Hive is the only network providing that instant gratification, we are going to get a lot of attention.

Speed matters.
But not only that... if dapps on Hive can suddenly go from a minute of lag between actions to a matter of seconds, that's very powerful. Lowering lag by a factor of ten or more is obviously a gigantic upgrade for a lot of dapps built on Hive. That's a massive improvement in the user experience, and may even open up entirely new models that were previously impossible to implement.
As someone who wants to build dapps on Hive myself, I can tell you from personal experience that waiting 1 minute for the last irreversible block is a pain in the ass (especially while testing code). Closing that gap between the head-block and the last-irreversible-block is an exciting prospect.
Optimization opportunity

Another interesting aspect of the OBI protocol is that it allows for much better monitoring of the status of the Hive P2P network when it is experiencing connectivity problems than was previously possible. Every node in the network (block producer or regular hived node) is tracking the current head block of every block producer it is connected to, effectively knowing how many block producers are connected to its network and which forks they are on.

A more stable baseline.
Anything Hive can do to lessen the damage of poor connectivity during Internet outages or whatever else is a blessing. We need upgrades like this if we want a robust network that people can depend on. This upgrade is going to allow nodes to optimize their connectivity.
Rather than waiting for a node to throw out a surprising blockchain reorganization at the last second when they go to put their block on the chain, that communication will be available long before then. This will make it much easier to figure out consensus before reorganizations happen rather than dynamically trying to figure it out in the middle of a shitstorm. It will also make it easier to figure out which nodes are bugging out the most and perhaps should be temporarily ignored if they are throwing too many errors.
Conclusion
Even though I flirted with the concept that this upgrade might have attack vectors, upon further inspection it appears to be mostly secure. Of course if we check the comments on the original post it is filled with doubts and comments from developers with far more experience than me, so check them out if you're so inclined. Still, I imagine it will be a very nice upgrade.
Even though it seems like such a small thing, the benefits likely greatly outweigh any risks. Again, we live in a culture of instant gratification. People want what they want, and they want it now, the second they decide they want it. No waiting. Hive being able to offer head-block irreversibility is the ultimate step in that direction and places us as leaders of the pack.
Posted Using LeoFinance ^Beta}