https://bobcopeland.com/blog/tags/hbase/ Recent content in Hbase on Insignificant Bits Hugo en-US Wed, 25 Apr 2012 12:58:01 +0000 https://bobcopeland.com/blog/2012/04/graphing-hbase-splits/ Wed, 25 Apr 2012 12:58:01 +0000 http://bobcopeland.com/blog/?p=1813 <p>I was asked for a blog post on this topic. I also do birthdays!</p> <p>HBase achieves balance by splitting regions when they reach a certain size, and by evenly distributing the number of regions among cluster machines. However, the balancer will not run in some cases (e.g. if there are regions stuck in transition), and balancing the number of regions alone may not help if region sizes are not mostly the same size. If a region server is hosting more regions than the others, requests to that server experience higher latency, and batch (map-reduce) jobs take longer to complete due to parallel skew.</p> https://bobcopeland.com/blog/2012/02/parsing-hregioninfo-in-python/ Thu, 16 Feb 2012 13:44:12 +0000 http://bobcopeland.com/blog/?p=1790 <p>I&rsquo;ve been doing a fair amount of HBase work lately at $work, not least of which is <a href="https://github.com/bcopeland/pybase">pybase</a>, a python module that encapsulates Thrift and puts it under an API that looks more or less like the Cassandra wrapper <a href="https://github.com/pycassa/pycassa">pycassa</a> (which we also use).</p> <p>When running an HBase cluster, one must very quickly learn the stack from top to bottom and be ready to fix the metadata when catastrophe strikes. Most of the necessary information about HBase regions is stored in the .META. table; unfortunately some of the values therein are serialized HBase Writables. One usually uses JRuby and directly loads Java classes to deal with the deserialization, but we&rsquo;re a Python shop and doing it all over thrift would be ideal.</p>