Ethereum creator Vitalik Buterin recently teased a proof of concept scalability solution on Twitter: “sharding” the blockchain’s ledger into smaller blocks that are spread across the network’s nodes.
Sharding the network has been talked about for a while as a way to scale out the blockchain and was suggested in the Plasma whitepaper authored by Buterin and Joseph Poon, co-creator of the Lightning Network, back in August last year. Plasma is a separate scaling solution also being worked - that is a framework for smart contracts on which Dapps can run on their own blockchains instead running on the public (consensus) chain, analogous to Bitcoin’s Lightning.
Ethereum has been trying to make the conversion from a proof-of-work (PoW) consensus to a proof-of-stake (PoS) consensus with the Casper protocol, which is a hybrid PoW/PoS. As far back as 2015, developers tried to incentivize miners to switch to PoS by ramping up the difficulty rate to mine through PoW, which lead to far higher costs for miners and negated their rewards.
Casper is in the second stage of testing and in the meantime Buterin is focused on sharding to introduce quadratic scaling. Buterin said: “The primary goal is massive scalability improvement. Each one of the shards (12 in that simulation, likely 100 live) will have as high capacity (and likely more) than the current existing Ethereum chain.
"The limit is basically that every node will have to verify the block headers of all the shards, and a node’s capacity to do this bounded above by their computational capabilities. Hence, quadratic sharding.”
What is sharding and how does it apply to Ethereum?
Sharding is a scaling out process (as opposed to scaling up) of dividing data across multiple servers instead of one, as the main server becomes a bottleneck of traffic. Tables of data are split horizontally into partitioned cells to save load and search effort.
Sharding is required not just for blockchain projects but any kind of online or mobile app as their databases can’t keep up with the load, so scalability is critical to maintain performance and if done right it can significantly improve performance. Scaling out is critical in the age of big data - that is tables with upwards of 50 billion rows - and as sharding has shared-nothing architecture, its scalability is huge.
In the case of Ethereum, in lay terms, the full ledger which is currently on all servers will be split across many servers each with different fragments of the full ledger. Transactions will therefore not have to be broadcast or updated on every server across the network but only on those servers with the relevant part of the ledger. Thus, instead of having every single node go through the entire transaction history of the Ethereum blockchain, sharding allows certain nodes to be assigned certain shards, streamlining the process of nodes verifying transactions.
On reddit Buterin explained “This is a proof of concept of (part of) a fork choice rule-based mechanism for how sharding can be bolted on top of the current ethereum main chain, with a specialized random beacon and shard block times of <10 seconds.”
“Fork choice rules” are what prevent double spending.
The point of all this sharding is to create “parallelization”. As Trustnodes explains in an example, “instead of all 10,000 nodes processing, validating, storing, all one million Eth transactions, 1,000 nodes process 100,000 transactions, and then they’re all tied together”.
“Once it is built and goes live, then each of the 100 shards will be able to process around 1.4 million transactions a day, giving the network a capacity of around 140 million transactions a day, or around 1,000 transactions a second.”
What is the Plasma scaling solution?
Another Ethereum scaling project underway is Plasma - a second layer of smart contracts over the main blockchain that doesn’t require every contract transaction to be broadcast or validated by the entire network. Conceivably, Plasma would replace server farms with a peer-to-peer network where everybody collectively runs these applications in a scalable, decentralized manner.
"The main benefit here is that basically the amount of data that clients need to process goes down by a lot," Buterin explained at an ethereum community conference in Paris in March.
The end vision is a system where nodes don’t have to download the entire Plasma history but users can instead generate “plasma coins” by sending a deposit to the contract. The technology appears to be most applicable to securing ethereum or ERC20 tokens in case of a hack on a crypto exchange.
Talking to Laura Shin on her Unchained podcast, Buterin explained how Plasma could insure your ERC20 token in case of an exchange hack.
“Basically it allows you to have what the bitcoin people promised in 2013, private chains anchored into a public chain. The anchor is: if you have one plasma coin in the Plasma chain and if the Plasma chain breaks [hacked] then you can use that to recover one coin on the public chain, so coins on the Plasma chain are equal to coins on the public chain without transactions taking up space on the public chain.”
“That means that you have a system that can be used for issuing tokens at enterprise scale, whether it’s governments or private individuals, while paying very low public chain transaction fees. And at the same time you get a lot of the public chain benefits.”
Will ETH still be ASIC chip resistant?
Ethereum developers are also contending with a growing threat to their current PoW algorithm (called Ethash) from the impending release of Bitmain’s ASIC chips. The Antpool E3 chip is designed specifically to process ethereum transactions and claim the protocol’s rewards.
Developers have been long aware of and preparing defenses for this development but Buterin has acknowledged that inevitably the chip technology will prevail over developers’ best intentions to keep their code optimized for general-purpose computers. Developers have suggested many ways of blocking Bitmain’s ASICs, including a hard fork which Buterin and other core developers have been avidly trying to avoid.
They argue that ASIC chips would have minimal effect on the Casper PoS consensus once the blockchain moves off Ethash as chips are hardware intensive, which is suited for mining, but as PoS intends to bypass mining it is more memory intensive - which is something that an ASIC chip, or GPU for that matter, would not have any effect on as they are encumbered by storage requirements.
Although there is no release date for Casper it is still an estimated five months away, and Bitmain announced the sale of its ASIC chips at the start of April so time is getting tight for Ethereum’s developers to change the game — and sharding may expedite that process.