https://www.aaronwest.net/tags/big-data/Recent content in Big Data on Aaron West's technology blogHugoenFri, 13 Apr 2018 14:00:00 -0500https://www.aaronwest.net/blog/what-im-reading-this-week-april-13-2018/Fri, 13 Apr 2018 14:00:00 -0500https://www.aaronwest.net/blog/what-im-reading-this-week-april-13-2018/<p>I spent a lot of time this week reading about how different engineering teams have scaled their Hadoop infrastructure. I also spent time looking at consistent hashing strategies, how GitHub manages DNS and routing, and how individual personalities affect team culture and performance. Read on for all the details.</p> <p><a href="http://awe.st/2gW1R86">Transit and Peering: How your requests reach GitHub</a></p> <p>One of GitHub&rsquo;s infrastructure engineers talks about they use Geographic DNS routing and network <a href="https://en.wikipedia.org/wiki/Peering">peering</a> to impact how their traffic is shaped. Through their efforts they were able to direct 60-70% of GitHub traffic through optimal routes.</p>