Year: 2014

Bucket list: Catch a fish and eat it

I checked off one of my bucket list items yesterday: catching a fish, cleaning it, and eating it.

This was the last day of a family vacation in Port St. Lucie in Florida. My original plans to go deep sea fishing fell through, so I went to the surprisingly well-stocked local Walmart to pick up some freshwater gear. I was lucky enough to nab a healthy-looking 15″ largemouth bass with a silver Mepps spinner from the lake behind our timeshare.

Continue reading →

Posted by andrew in Personal, 0 comments

Paper review: Facebook f4

It’s been a while since I did one of these! I did a previous review of Facebook Haystack, which was designed as an online blob storage system. f4 is a sister system that works in conjunction with Haystack, and is intended for storage of warm rather than hot blobs. As is usual for Facebook, they came up with a system that is both eminently practical and tailored for their exact use case.

This paper, “f4: Facebook’s Warm BLOB Storage System” by Muralidhar et al., was published at OSDI ’14.

Continue reading →

Posted by andrew in Reviews, 0 comments

In-memory Caching in HDFS: Lower latency, same great taste

My coworker Colin McCabe and I recently gave a talk at Hadoop Summit Amsterdam titled “In-memory Caching in HDFS: Lower latency, same great taste.” I’m very pleased with how this feature turned out, since it was approximately a year-long effort going from initial design to production system. Combined with Impala, we showed up to a 6x performance improvement by running on cached data, and that number will only improve with time.

Slides: pptx

Video: Youtube

Posted by andrew in Talks, 0 comments

Two engineering principles

I received two interesting pieces of advice at the AMP Lab retreat this past week, which concisely state some of my favorite software engineering principles:

  1. Don’t be a zealot. Understand in technical detail why a given language, framework, or design should be preferred, not because of technological fascination or fanboy-ism. The canonical examples here are programming language flamewars, e.g. Java vs. C++.
  2. Ruthlessly optimize for your requirements. This means first, carefully defining said requirements, but then being completely unafraid to buck conventional wisdom if it’s not a good match. This often means intentionally pruning out features, even common ones implemented by other systems.
Posted by andrew, 0 comments