Kubernetes releases etcd v3.4 with better backend storage, improved raft voting process, new raft non-voting member and more

Source: .packtpub.com

Cloud computing stocks Box (NYSE: BOX) and Dropbox (NASDAQ: DBX) compete in the cloud content management market. Both companies provide software as a service (SaaS), known otherwise as enterprise file synchronization and sharing. In simple terms, this means uploading files to the cloud, storing them there, and sharing them with others.

4 min read

Last Friday, a team at Kubernetes announced the release of etcd 3.4 version. etcd 3.4 focuses on stability, performance and ease of operation. It includes features like pre-vote and non-voting member and improvements to storage backend and client balancer.

Key features and improvements in etcd v3.4

Better backend storage

etcd v3.4 includes a number of performance improvements for large scale Kubernetes workloads. In particular, etcd experienced performance issues with a large number of concurrent read transactions even when there is no write (e.g. “read-only range request … took too long to execute”). Previously, the storage backend commit operation on pending writes, blocks incoming read transactions, even when there was no pending write. Now, the commit does not block reads which improve long-running read transaction performance.

The team has further made backend read transactions fully concurrent. Previously, ongoing long-running read transactions block writes and upcoming reads. With this change, write throughput is increased by 70% and P99 write latency is reduced by 90% in the presence of long-running reads. They also ran Kubernetes 5000-node scalability test on GCE with this change and observed similar improvements.

Improved raft voting process

etcd server implements Raft consensus algorithm for data replication. Raft is a leader-based protocol. Data is replicated from leader to follower; a follower forwards proposals to a leader, and the leader decides what to commit or not. Leader persists and replicates an entry, once it has been agreed by the quorum of cluster. The cluster members elect a single leader, and all other members become followers. The elected leader periodically sends heartbeats to its followers to maintain its leadership, and expects responses from each follower to keep track of its progress.

In its simplest form, a Raft leader steps down to a follower when it receives a message with higher terms without any further cluster-wide health checks. This behavior can affect the overall cluster availability.

For instance, a flaky (or rejoining) member drops in and out, and starts campaign. This member ends up with higher terms, ignores all incoming messages with lower terms, and sends out messages with higher terms. When the leader receives this message of a higher term, it reverts back to follower.

This becomes more disruptive when there’s a network partition. Whenever the partitioned node regains its connectivity, it can possibly trigger the leader re-election. To address this issue, etcd Raft introduces a new node state pre-candidate with the pre-vote feature. The pre-candidate first asks other servers whether it’s up-to-date enough to get votes. Only if it can get votes from the majority, it increments its term and starts an election. This extra phase improves the robustness of leader election in general. And helps the leader remain stable as long as it maintains its connectivity with the quorum of its peers.

Introducing a new raft non-voting member, “Learner”

The challenge with membership reconfiguration is that it often leads to quorum size changes, which are prone to cluster unavailabilities. Even if it does not alter the quorum, clusters with membership change are more likely to experience other underlying problems.

In order to address failure modes, etcd introduced a new node state “Learner”, which joins the cluster as a non-voting member until it catches up to leader’s logs. This means the learner still receives all updates from leader, while it does not count towards the quorum, which is used by the leader to evaluate peer activeness. The learner only serves as a standby node until promoted. This relaxed requirements for quorum provides the better availability during membership reconfiguration and operational safety.

Improvements to client balancer failover logic

etcd is designed to tolerate various system and network faults. By design, even if one node goes down, the cluster “appears” to be working normally, by providing one logical cluster view of multiple servers. But, this does not guarantee the liveness of the client. Thus, etcd client has implemented a different set of intricate protocols to guarantee its correctness and high availability under faulty conditions.

Historically, etcd client balancer heavily relied on old gRPC interface: every gRPC dependency upgrade broke client behavior. A majority of development and debugging efforts were devoted to fixing those client behavior changes. As a result, its implementation has become overly complicated with bad assumptions on server connectivity. The primary goal in this release was to simplify balancer failover logic in etcd v3.4 client; instead of maintaining a list of unhealthy endpoints, whenever client gets disconnected from the current endpoint.

The cloud-based file sharing market is crowded and competitors in the business range from a bevy of small private companies to a smattering of mega cap giants like Amazon’s Amazon Drive, Microsoft’s OneDrive, and Alphabet, Inc.’s Google Drive (I’m sure brand consultants labored tirelessly to conceive those inimitable names).

Image Source: Getty Images

Investors looking to own a cloud-based file sharing stock should focus on Box and Dropbox. Both companies focus solely on the cloud-based workplace automation and collaboration market. They are “pure play” providers. Here’s how they compare.

Box and Dropbox are similar in some respects

Box and Dropbox allow consumers and large enterprises to securely sync and share documents, photos, and videos across multiple devices. Their corporate customers include an impressive roster of Fortune 500 companies. Ultimately, these large-scale businesses are their primary targets. They compete for their customers by offering similar product lines.

Both Box and Dropbox have open architectures that aren’t dependent on a specific platform or application. This allows them both to integrate easily with other cloud services their customers use. They both make it possible for their customers to accelerate business processes and empower workforce collaboration. And they each provide a high level of data security. But this is largely where the similarities end.

Dropbox has a Better Enterprise Offering Than Does Box

Dropbox offers one of the fastest sync algorithms on the market. Dropbox invented the method that allows you to sync files to the cloud, and their algorithms are used by nearly all cloud service providers. Box doesn’t use the same algorithms, so syncing files to the cloud is slower.

Dropbox also makes it easier for teams to work together. Users can collaborate on a sharing page that makes company information easier to access. Box doesn’t do this. What’s more, Dropbox lets teams coordinate tasks and timelines and communicate with each other in real time. Dropbox can be integrated with other cloud providers’ apps to allow teams to upload and download content, share it, and collaborate and comment on it regardless of the platform it originated from.

Teams using Dropbox can open Microsoft and Google files (Docs, Sheets, or Slides), and can work across applications that couldn’t otherwise talk to each other, and can access files wherever they are saved: in the cloud, on a hard drive, or on some other device.

Finally, while Box has been in business a couple of years longer than Dropbox, Dropbox has done a much better job branding itself with both consumers and businesses. 

Dropbox is Executing Their Business Model Better Than Box

Given the two companies’ similarities, one would expect that their operating numbers would likewise be similar. But a comparison of some basic fundamental metrics shows that Dropbox is much healthier than Box. The table below offers a glimpse into the short-term financial health and operating efficiency of each company.