Ethereum Sharding: All You Need To Know on Ethereum’s Scaling Solution

At Ethereum’s peak usage, growing transaction fees and gas costs make the sending of tokens extremely expensive. Obviously, a scaling fix is needed just like with any other blockchain.

Ethereum had its best year in 2017. The usage of the smart contracts featuring platform was huge with a record-high number of ICOs raising billions of dollars and the price of the Ether token (ETH) soaring 9800% in value. However, Ethereum is still in its early days and ICOs are still the main use case for the technology. Because there are so many ICOs that are running on top of Ethereum now, the network is processing 750,000 transactions per day. In comparison, Bitcoin is processing only 230,000 transactions per day. This usage is slowing down the network congesting it and making it expensive to use at peak times, which brings up the question of scaling.

There are many ways to scale a blockchain. Some of the scaling technologies for Ethereum are Sharding (on-chain), Raiden, Plasma, and GoNetwork (off-chain). Here I will focus on Sharding as part of the on-chain scaling development of Ethereum.

Sharding

A shard is used in database architecture and it “is a horizontal partition of data in a database or search engine“. A “shard” is called the individual partition. The purpose of having shards is to spread load because each shard can be held on separate hardware. Sharding is used for horizontal scalability, increasing capacity by connecting multiple entities to “work as a single logical unit.”

This technology greatly improves the performance of databases because the distribution of data and calculations is over a large number of machines. Data remains present in all shards but certain data can only appear in a single shard.

However, some of the issues with sharding include increased reliance on interconnectivity between servers and high latency when querying. Also, sharding is a complex thing to do and it is often inflexible.

Full node problem in Ethereum

Full nodes store all states of the blockchain and process all transactions. A state is the set of information that describes a system at any given moment in time. In Ethereum, the state is the current address balances, smart contracts, output codes, storage, and nonces. With each new transaction, this state changes into an entirely new state.

In the case of Ethereum, each full node has to do all calculations and execute every smart contract that was ever created and update the state on disk. This creates a troublesome situation where in order to run an Ethereum full node requires a lot of resource power (disk space, bandwidth, disk I/O). Syncing a full node with using an HDD is impossible because the disk speed is too slow to catch up with the current blockchain state. To run a full node, you must use an SSD which is not cheap.

Sharding in Ethereum

The idea of sharding is that some of the nodes can do some calculation. The calculations are split up into several parts. Therefore, the state of the blockchain is split into several domains, called shards. Each shard contains its own independent state and transaction history. This system is like a network of mini-blockchains that have their own network of nodes processing transactions only for the specific shard. The goal is to relax some of the load on the system allowing for higher throughput of processed transactions across all shards.

Those separate blockchains (shards) all inherit the same security characteristics and consensus of the main blockchain. There is no separate independent consensus for each shard.

The basic premises of sharding is that separate mini-blockchains can scale better because they exist for a specific purpose. This way the workload is not concentrated on one chain only.

Validator Manager Contract

Inside the mainchain there will be a Validator Manager Contract (VMC) that will run on a Proof of Stake consensus which will be responsible for the sharding block creation. It will manage the rights of block creators in each shard (also called collations) and keep track of validators. The VMC will act as a light client for the communication among shards and the mainchain and among shards themselves.

At a later stage the mainchain will transition completely to a full Proof of Stake. Meanwhile, each shard shall function under Proof of Stake via the VMC. The big picture is to have a main shard which will focus on security, while the new shards will focus on deployment of new features.

Criticism of Sharding

One of the main concerns with Sharding is the fear that it would create further centralization of the network of nodes in Ethereum and will not solve the syncing problem.

The current situation in Ethereum is that normal people are only able to run light nodes (validating only the block headers) and not full nodes (validating new blocks and verifying transactions) because of the already mentioned issues with syncing. Validating headers is not block validation. Only data centers will be able to run full nodes that contain the full record of the blockchain (have a full copy of the original genesis block). The light nodes will have to trust those data centers.

With Sharding in Ethereum, the outlook of the node network will change. There will be no full nodes that do everything. Instead, in each shard-chain, there will be fully validating nodes (shard nodes) that will validate new blocks and verify transactions within a specific shard and only validate the headers of other shards. And there will be the light nodes that will only validate the headers of different shards and the headers within a specific shard.

More specifically, the different type of nodes in the new Ethereum will look like this.

  • Collator Nodes make collation (block), and then present it to the Executor nodes to “execute”.
  • Executor Nodes validate transactions and compute the contracts inside each collation.
  • Light Nodes check transactions and have no power over the validity of transactions.

Most people will still be able to run light nodes in the new Ethereum. The problem with running full nodes will still persist. Full nodes in the new Ethereum will be called Executor nodes which still will be hard to maintain by regular people. The centralization aspect is not tackled with sharding. If people are fine with relying on centralized validators they can maybe do better by choosing other centralized blockchains such as NEO or EOS. They do not try to claim they are decentralized like Ethereum does.

Final remarks

Sharding is a cutting-edge technology that would solve some of the scaling problems in Ethereum but it is still nascent.

Implementing Sharding will take several years. The chance to see some kind of alpha in 2018 are very slim. There might be some more written specs and research in 2018 but an actual implementation that other developers can use will probably be seen in 2019.  The implementation could possibly take place in some distant future fork of Ethereum after the Serenity fork.

Some informative links to consider in gathering more info about sharding in Ethereum.

Ethereum sharding FAQ: https://github.com/ethereum/wiki/wiki/Sharding-FAQs

The first Sharding implementation for Ethereum: https://medium.com/prysmatic-labs

A very detailed read on centralization due to Sharding: https://hackernoon.com/sharding-centralizes-ethereum-by-selling-you-scaling-in-disguised-as-scaling-out-266c136fc55d