From the Twitter Engineering Blog
- Hadoop filesystem at Twitter Monday, September 28, 2015
Twitter runs multiple large Hadoop clusters that are among the biggest in the world. Hadoop is at the core of our data platform and provides vast storage for analytics of user actions on Twitter. In this post, we will highlight our contributions to ViewFs, the client-side Hadoop filesystem view, and its versatile usage here.
- Building DistributedLog: Twitter’s high-performance replicated log service Wednesday, September 16, 2015
At Twitter, we’ve used replicated logs to address a range of challenging problems in distributed systems. Today we’d like to tell you about DistributedLog, our high-performance replicated log service, and how we used it to address one of these problems.
- Improving communication across distributed teams Friday, September 04, 2015
As a global company, Twitter has offices in numerous locations around the world. Optimizing collaboration and improving productivity between employees across offices is a challenge faced by many companies and something we’ve spent a great deal of time on. For our part, we’ve learned a few lessons about what it takes to make distributed engineering offices successful. Over the next few months, we’ll share what we’ve learned. We’ll also periodically offer workshops about how to do distributed engineering well.
- Diffy: Testing services without writing tests Thursday, September 03, 2015
Today, we’re excited to release Diffy, an open-source tool that automatically catches bugs in Apache Thrift and HTTP-based services. It needs minimal setup and is able to catch bugs without requiring developers to write many tests.