HBase HMaster Architecture

HBase architecture follows the traditional master slave model where you have a master which takes decisions and one or more slaves which does the real task. In HBase, the master is called HMaster  and slaves are called HRegionServers (yes..servers). In this post i will zoom in to HMaster and will detail some of the modules and functionality of HMaster.

 

Lets zoom in to HMaster and discuss different modules present in that.

HMaster and its Components

HMaster and its Components

 

Note: Most of the descriptions are taken from javadocs of HBase project. So all credits goes to the developers who wrote it in the first place.

The above diagram shows the internals of HMaster. I have grouped the modules in to different categories for us to make it a little easy.

1 ) External Interfaces

External Interfaces are responsible for interacting with the external world (Hmaster web site, client, Region Servers and other management utilities like JConsole).

Info Server

Info server is an embedded jetty server instance started by HMaster to answer http requests (default port is 60010). The primary goal is to serve up status information for the server. There are three contexts:

  • “/stacks/” -> points to stack trace
  • “/static/” -> points to common static files (src/hbase-webapps/static)
  • “/” -> the jsp server code from (src/hbase-webapps/<name>)

RPC Server

HBase rpc server module instantiates the configured RPC Engine which is responsible for all the rpc communication that the master does. There are atleast two different RPC Engines in hbase now. One is the good old WritableRPCEngine (Which is the default) and the other one is the ProtocolBufferRPCEngine. Whichever is selected, they maintain 3 different interfaces/protocols to which they respond. In master, the supported protocols are MasterMonitorProtocolMasterAdminProtocol and RegionServerStatusProtocol. More about protocols in my earlier post here.

Hbase RPC Server

HBase RPC Server

 

Master MXBean

 Apart from standard HBase metrics, hbase supports Java Management Extension based metric export. You can use any standard JMX compliant browser like JConsole and can view the metrics. To enable JMX based remote metrics monitoring in hbase follow the instructions here

2 ) Executor Services

A generic executor service abstracts a Event Queue where Events of different types can be posted. The events are handled by their respective Runnable handlers which pick threads from a dedicated thread pool. In order to create a new service, create an instance of this class and then do: instance.startExecutorService(“myService”).  When done call shutdown(). In order to use the service created above, call submit(EventHandler). Register pre- and post- processing listeners by registering your implementation of EventHandler.EventHandlerListener with registerListener(EventHandler.EventType, EventHandler.EventHandlerListener).  Be sure to deregister your listener when done via unregisterListener(EventHandler.EventType). The services mentioned below are instantiated using this generic  executor service. The events and Event handlers are described below for each of the service.

Hbase Executor Service

HBase Executor Service

 

Open Region Service (MASTER_OPEN_REGION)

When the master (i.e. Assignment Manager) detects that a region was successfully opened (through zookeeper watch), it posts a event of type RS_ZK_REGION_OPENED to this service. This event is handled by the event handler OpenRegionHandler().

Close Region Service (MASTER_CLOSE_REGION)

When the master (i,e, Assignment Manager) detects that a region was successfully closed (through a watcher), it posts a event of type RS_ZK_REGION_CLOSED to this service. Also when a region open attempt fails an event of type RS_ZK_REGION_FAILED_OPEN is posted. These events are handled by the event handler ClosedRegionHandler().

Server Operations Service (MASTER_SERVER_OPERATIONS)

Master detects a region split through zookeeper watch and posts RS_ZK_REGION_SPLIT which is handled by SplitRegionHandler. Also when a master needs to expire a region server (which does not host ROOT or META) it posts an event M_SERVER_SHUTDOWN which is handled by an event handler ServerShutdownHandler.

Meta Server Operations Service (MASTER_META_SERVER_OPERATIONS)

when a master needs to expire a region server which hosts ROOT or META, it posts an event M_META_SERVER_SHUTDOWN which is handled by an event handler MetaServerShutdownHandler.

Table Operations Service (MASTER_TABLE_OPERATIONS)

 All the table operations originating from the client is handled in this service. Message like C_M_DELETE_TABLE, CM_DISABLE_TABLE, C_M_ENABLE_TABLE, C_M_MODIFY_TABLE and C_M_CREATE_TABLE are posted from the client and handled by the handler DeleteTableHandler, DisableTableHandler, EnableTableHandler, ModifyTableHandler and CreateTableHandler respectivily.

Executor Service

Event Event Handler

Threads

(Default)

Master Open Region
RS_ZK_REGION_OPENED
OpenRegionHandler
5
Master Close Region
RS_ZK_REGION_CLOSED
ClosedRegionHandler
5
Master Server Operations
 
RS_ZK_REGION_SPLIT
M_SERVER_SHUTDOWN
SplitRegionHandler
ServerShutdownHandler
3
Master Meta Server Operations
M_META_SERVER_SHUTDOWN
MetaServerShutdownHandler
5
Master Table Operations
 
C_M_DELETE_TABLE C_M_DISABLE_TABLE C_M_ENABLE_TABLE C_M_MODIFY_TABLE C_M_CREATE_TABLE
DeleteTableHandler DisableTableHandler EnableTableHandler ModifyTableHandler CreateTableHandler
1

3 ) Zookeeper System Trackers

Master and RS uses zookeeper to keep track of certain events and happenings in the cluster. In Master, a centralized class called ZookeeperWatcher acts as a proxy for any event tracker which uses zookeeper. All the common things like connection handling, node management and exceptions are handled here. Any tracker which needs the service of this call must register with this class to get notified of any specific event.

Zookeeper Based Trackers

Zookeeper Based Trackers

Active Master Manager

Handles everything on master side related to master election. This is the place where the backup masters block, until the active master fails or the cluster shuts down. Listens and responds to ZooKeeper notifications on the master znode, both nodeCreated and nodeDeleted. Uses a zNode called “master” under the base zNode.

Region Server Tracker

Watches the zNode called “rs” under the base zNode. If any children is added (i.e. if any Region server comes up) or deleted (if any Region server goes down) this class gets notified. The main function of this class is to maintain a active list of online Region Servers. If any region server fails it triggers the serverExpiry procedure.

Draining Server Tracker

Tracks the list of draining region servers via ZK. This class is responsible for watching for changes to the draining servers list.  It handles adds/deletes in the draining RS list and watches each node. If an RS gets deleted from draining list, we call ServerManager#removeServerFromDrainList(ServerName) If an RS gets added to the draining list, we add a watcher to it and call ServerManager#addServerToDrainList(ServerName). Uses the zNode called “draining” under the base zNode.

Catalog Tracker

Tracks the availability of the catalog tables -ROOT and .META. This class is “read-only” in that the locations of the catalog tables cannot be explicitly set.  Instead, ZooKeeper is used to learn of the availability and location of -ROOT. -ROOT is used to learn of the location of .META. If not available in -ROOT, ZooKeeper is used to monitor for a new location of .META..Call  #start() to start up operation.  Call #stop() to interrupt waits and close up shop.

Cluster Status Tracker

Used to monitor the cluster status using the zNode “shutdown”. It just says wether the cluster is up or down for now.

Assignment Manager

Manages and performs region assignment. Monitors ZooKeeper for events related to regions in transition. Handles existing regions in transition during master failover.

Root Region Tracker

 Tracks the root region server location node in zookeeper (zNode is “root-region-server”). Root region location is set by RootLocationEditor usually called out of RegionServerServices. This class has a watcher on the root location and notices changes. Mainly used to know if the root region is available and where is it hosted.

Load Balancer

Makes decisions about the placement and movement of Regions across RegionServers. Cluster-wide load balancing will occur only when there are no regions in transition and according to a fixed period of a time using  #balanceCluster(Map). Inline region placement with {@link #immediateAssignment} can be used when the Master needs to handle closed regions that it currently does not have a destination set for.  This can happen during master failover. On cluster startup, bulk assignment can be used to determine locations for all Regions in a cluster. This classes produces plans for the {@link AssignmentManager} to execute. The load balancer implementation are pluggable. By default it uses the “DefaultLoadBalancer”. There is a new StochasticsLoadBalancer also which can be used.

Meta Node Tracker

Watches the zNode called “unassigned” used by the META table. Used by CatalogTracker to track the location of the meta table.

Master Address Tracker

Used by Active Master Manager to manage the current location of the master for the Region Servers.

4 ) File System Interfaces

All the services which interacts with the underlying FileSystem to store or manage data pertaining to the control of a Hmaster is lumped under this section

MasterFileSystem

This class abstracts the file system operation for HBase including identifying base directory, log splitting, Delete Region, Delete Table etc.

Log Cleaner

This is a chore (see next section) which runs at some specified interval and attempt to delete the Hlogs in the oldlogs directory. This is a chain of cleaner delegate which can be used to clean any kind of log files. By default, two cleaners: TimeToLiveLogCleaner and ReplicationLogCleaner are called in order. So if other effects are needed, implement your own LogCleanerDelegate and add it to the configuration “hbase.master.logcleaner.plugins”, which is a comma-separated list of fully qualified class names. LogsCleaner will add it to the chain. HBase ships with LogsCleaner as the default implementation.

HFile Cleaner

This is also a chore (see next section) which runs at some specified intervals. This handles the HFile cleaning functions inside the master. By default, only the TimeToLiveHFileCleaner is called. If other effects are needed, implement your own LogCleanerDelegate and add it to the configuration “hbase.master.hfilecleaner.plugins”, which is a comma-separated list of fully qualified class names. The <code>HFileCleaner<code> will build the cleaner chain in  order the order specified by the configuration.

5 ) Chores

Chore is a task performed on a period in hbase.  The chore is run in its own thread. This base abstract class provides while loop and sleeping facility. If an unhandled exception, the threads exit is logged. Implementers just need to add checking if there is work to be done and if so, do it.  Its the base of most of the chore threads in hbase. Don’t subclass Chore if the task relies on being woken up for something to do, such as an entry being added to a queue, etc.

Balancer Chore

The balancer is a tool that balances disk space usage on an HDFS cluster when some datanodes become full or when new empty nodes join the cluster. The tool is deployed as an application program that can be run by the cluster administrator on a live HDFS cluster while applications adding and deleting files.

The threshold parameter is a fraction in the range of (1%, 100%) with a default value of 10%. The threshold sets a target for whether the cluster is balanced. A cluster is balanced if for each datanode, the utilization of the node (ratio of used space at the node to total capacity of the node) differs from the utilization of the (ratio of used space in the cluster to total capacity of the cluster) by no more than the threshold value. The smaller the threshold, the more balanced a cluster will become. It takes more time to run the balancer for small threshold values. Also for a very small threshold the cluster may not be able to reach the balanced state when applications write and delete files concurrently.

The tool moves blocks from highly utilized datanodes to poorly utilized datanodes iteratively. In each iteration a datanode moves or receives no more than the lesser of 10G bytes or the threshold fraction of its capacity. Each iteration runs no more than 20 minutes. At the end of each iteration, the balancer obtains updated datanodes information from the namenode.

A system property that limits the balancer’s use of bandwidth is defined in the default configuration file:

   dfs.balance.bandwidthPerSec

This property determines the maximum speed at which a block will be moved from one datanode to another. The default value is 1MB/s. The higher the bandwidth, the faster a cluster can reach the balanced state, but with greater competition with application processes. If an administrator changes the value of this property in the configuration file, the change is observed when HDFS is next restarted.

Catalog Janitor Chore

A janitor for catalog tables. It scans the META tables for looking for unused regions to garbage collect.

Log Cleaner Chore

Explained in the previous section

HFile Cleaner Chore

Explained in the previous section

6 ) Others

Server Manager

The ServerManager class manages info about region servers.Maintains lists of online and dead servers. Processes the startups, shutdowns, and deaths of region servers. Servers are distinguished in two different ways. A given server has a location, specified by hostname and port, and of which there can only be one online at any given time. A server instance is specified by the location (hostname and port) as well as the startcode (timestamp from when the server was started). This is used to differentiate a restarted instance of a given server from the original instance.

Co-Processor Host

 Provides the common setup framework and runtime services for coprocessor invocation from HBase services.

 

 

 

 

 

 

 

 

 

Protocol Buffers in HBase

For early Hbase developers, it is often a nightmare to understand how the different modules speak among
themselves and what. This comes in handy to understand what each sub-system is responsible for and which informations
do they maintain. This post is about documenting all the different protocols through which the HBase modules
speak among themselves. I will cover the Hbase specific protocols in this post especially protocol buffer ones.

 

HBase Protocols

HBase Protocols

In the Hbase 0.96.0 release, hbase is moving to protocol buffers for communicating with different sub-systems.  There are 5 major  protocols which is used as shown in the figure above.

MasterMonitorProtocol , Protocol that a client uses to communicate with the Master (for monitoring purposes).

  • MasterMonitorService

    • GetSchemaAlterStatusRequest : Used by the client to get the number of regions that have received the updated schema
    • GetSchemaAlterStatusResponse: Indicates the number of regions updated. yetToUpdateRegions is the regions that are yet to be updated. totalRegions is the total number of regions of the table
    • GetTableDescriptorsRequest : Get list of TableDescriptors for requested tables. GetTableDescriptorsRequest that contains tableNames: requested tables, or if empty, all are requested
    • GetTableDescriptorsResponse: The TableSchema (see below for the description of TableSchema) for all the requested tables.
    • GetClusterStatusRequest: Used by the clients to get the current status of the cluster
    • GetClusterStatusResponse: The ClusterStatus object is returned.
  • MasterService

    • IsMasterRunningRequest: Used by clients to see if the master is running or not
    • IsMasterRunningResponse: returns True if the master is available.

MasterAdminProtocol, Protocol that a client uses to communicate with the Master (for admin purposes).

  • MasterAdminService

    • AddColumnRequest: Add a column to the specified table. Req contains tableName: table to modify and column: ColumnFamilySchema.
    • AddColumnResponse: Empty response
    • DeleteColumnRequest: Deletes a column from the specified table. Table must be disabled. Req contains tableName: table to modify and column: column Name
    • DeleteColumnResponse: Empty Response
    • ModifyColumnRequest: Modifies an existing column on the specified table. Req contains tableName: table to modify and column: ColumnFamilySchema.
    • ModifyColumnResponse: Empty Response.
    • MoveRegionRequest: Move a region to a specified destination server. The request that contains region:The RegionSpecifier destServerName: The ServerName of the destination regionserver.  If passed the empty byte array we’ll assign to a random server.
    • MoveRegionResponse: Empty Response
    • AssignRegionRequest: Assign a region to a server chosen at random. req contains the RegionSpecifier.  Will use existing RegionPlan if one found.
    • AssignRegionResponse: Empty Response
    • UnassignRegionRequest: Unassign a region from current hosting regionserver.  Region will then be assigned to a regionserver chosen at random.  Region could be reassigned back to the same server. Req contains a RegionSpecifier (Will clear any existing RegionPlan if one found) and a bool to say wether to force it or not (Will remove region from regions-in-transition too if present as well as from assigned regions — radical!.If results in double assignment use hbck -fix to resolve).
    • UnassignRegionResponse: Empty Response
    • OfflineRegionRequest: Offline a region from the assignment manager’s in-memory state.  The region should be in a closed state and there will be no attempt to automatically reassign the region as in unassign.   This is a special method, and should only be used by experts or hbck. Req  contains the RegionSpecifier to make offline. Will clear any existing RegionPlan if one found.
    • OfflineRegionResponse: Empty Response
    • DeleteTableRequest: Deletes a table. Req contains the table name in bytes.
    • DeleteTableResponse: Empty Response
    • EnableTableRequest: Puts the table on-line (only needed if table has been previously taken offline).Req contains the table name in bytes.
    • EnableTableResponse: Empty Response.
    • DisableTableRequest: Take table offline. Req contains the table name in bytes.
    • DisableTableResponse: Empty Response
    • ModifyTableRequest: Modify a table’s metadata. req contains tableName: table to modify and TableSchema: new descriptor for table
    • ModifyTableResponse: Empty Response
    • CreateTableRequest: Creates a new table asynchronously.  If splitKeys are specified, then the table will be created with an initial set of multiple regions.If splitKeys is null, the table will be created with a single region. Also contains the TableSchema.
    • CreateTableResponse: Empty Response
    • ShutdownRequest: Shutdown an HBase cluster
    • ShutdownResponse: Empty Response
    • StopMasterRequest: Stop HBase Master only. Does not shutdown the cluster.
    • StopMasterResponse: Empty Response
    • BalanceRequest: Run the balancer.  Will run the balancer and if regions to move, it will go ahead and do the reassignments.  Can NOT run for various reasons.  Check logs.
    • BalanceResponse: BalanceResponse that contains: balancerRan: True if balancer ran and was able to tell the region servers to unassign all the regions to balance (the re-assignment itself is async), false otherwise.
    • SetBalancerRunningRequest: Turn the load balancer on or off.  req SetBalancerRunningRequest that contains: on: If true, enable balancer. If false, disable balancer. synchronous: if true, wait until current balance() call, if outstanding, to return.
    • SetBalancerRunningResponse: contains: prevBalanceValue: Previous balancer value
    • CatalogScanRequest: Run a scan of the catalog table
    • CatalogScanResponse: CatalogScanResponse that contains the int return code corresponding to the number of entries cleaned
    • EnableCatalogJanitorRequest: Enable/Disable the catalog janitor.contains: enable: If true, enable catalog janitor. If false, disable janitor.
    • EnableCatalogJanitorResponse: contains: prevValue: true, if it was enabled previously; false, otherwise
    • IsCatalogJanitorEnabledRequest: Query whether the catalog janitor is enabled
    • IsCatalogJanitorEnabledResponse: Contains: value: true, if it is enabled; false, otherwise

AdminProtocol, Protocol that a HBase client uses to communicate with a region server.

  • AdminService

    • GetRegionInfoRequest: Get the information of a region identified with RegionSpecifier. Also say wether to retrieve the compaction state  (NONE, MINOR, MAJOR, MINOR_AND_MAJOR) of the given region.
    • GetRegionInfoResponse: returns the RegionInfo and the comapction state (if requested) of the region.
    • GetStoreFileRequest: Get a list of store files for a set of column families in a particular region identified with RegionSpecifier.If no column family is specified, get the store files for all column families.
    • GetStoreFileResponse: A list of all store file names of the region.
    • GetOnlineRegionRequest: Get information of all the region which are online in a particular region server.
    • GetOnlineRegionResponse: list of RegionInfo of all the regions which are in online state in the given region server.
    • OpenRegionRequest: Open a region/ list of region identified with RegionInfo on the region server
    • OpenRegionResponse: A list with the state of the regions requested (OPENED, ALREADY_OPEN< FAILED_OPENING)
    • CloseRegionRequest: Closes the specified region and will use or not use ZK during the close according to the specified flag. req contains RegionSpecifier, ServerName transitionInZK flag etc.
    • CloseRegionResponse: A flag indicating that the region is closed or not.
    • FlushRegionRequest: Flushes the MemStore of the specified region identified with RegionSpecifier on the region server. This method is synchronous. req has a optional timestamp above which the region should be flushed otherwise not.
    • FlushRegionResponse: Has last flush time and a bool specifying wether the region was flushed or not (this field is not used in the code now).
    • SplitRegionRequest: Split a region identified by the RegionSpecifier on the region server. This method currently flushes the region and then forces a compaction which will then trigger a split. The flush is done synchronously but the compaction is asynchronous.
    • SplitRegionResponse: Empty Response
    • CompactRegionRequest:  Compacts the region identified by RegionSpecifier. Performs a major compaction if the flag is set to true. This method is asynchronous.
    • CompactRegionResponse: Empty Response
    • ReplicateWALEntryRequest: Replicates the given entries. The guarantee is that the given entries will be durable on the slave cluster if this method returns without any exception.”hbase.replication” has to be set to true for this to work.
    • ReplicateWALEntryResponse: Empty response
    • RollWALWriterRequest:  Roll the WAL writer of the region server
    • RollWALWriterResponse: A list of regions to flush.
    • GetServerInfoRequest: Get some information of the region server.
    • GetServerInfoResponse: Response contains the ServerName and the webui port.
    • StopServerRequest: Stop the region server. req contains an reason to stop.
    • StopServerResponse: Empty Response.

ClientProtocol, Protocol that a HBase client uses to communicate with a region server.

  • ClientService

    • GetRequest:  Get data from a table. Perform a single Get operation. Unless existenceOnly is specified, return all the requested data for the row that matches exactly, or the one that immediately precedes it if closestRowBefore is specified. If existenceOnly is set, only the existence will be returned.
    • GetResponse: Contains the row to be fetched / the closest row before. If only existence was requested a bool is sent indicating if the row exists or not.
    • MutateRequest: Mutate data in a table (APPEND, INCREMENT, PUT_DELETE).  Perform a single Mutate operation.Optionally, you can specify a condition. The mutate will take place only if the condition is met. Otherwise,the mutate will be ignored.
    • MutateResponse: In the response result, parameter processed (only in PPEND and INCREMENT) is sent back along with a bool ( for PUT and DELETE) which indicates if the mutate actually happened.
    • ScanRequest:  Instead of get from a table, you can scan it with optional filters. You can specify the row key range, time range, the columns/families to scan and so on. A scan request. Initially, it should specify a scan. Later on, you can use the scanner id returned to fetch result batches with a different scan request. The scanner will remain open if there are more results, and it’s not asked to be closed explicitly. You can fetch the results and ask the scanner to be closed to save a trip if you are not interested in remaining results. req contains a RegionSpecifier.
    • ScanResponse: Contains the result of the scan (rows). If there are no more results, moreResults will be false. If it is not specified, it means there are more.
    • LockRowRequest: Lock a row in a table explicitly from the client. Req contains RegionSpecifier.
    • LockRowResponse: Returns the lock id and TTL (not implemented in code yet)of the lock.
    • UnlockRowRequest: Unlock a locked row in a table. req contains the RegionSpecifier and the lockId.
    • UnlockRowResponse: EmptyResponse
    • BulkLoadHFileRequest: Atomically bulk load multiple HFiles (say from different column families) into an open region. Req contains RegionSpecifier and Column name and the file path.
    • BulkLoadHFileResponse: A bool indicating wether the files are loaded or not.
    • ExecCoprocessorRequest: Executes a single  org.apache.hadoop.hbase.ipc.CoprocessorProtocol method using the registered protocol handlers. Req contains the RegionSpecifier. An individual coprocessor call. You must specify the protocol, the method, and the row to which the call will be executed. You can specify the configuration settings in the property list. The parameter list has the parameters used for the method. A parameter is a pair of parameter name and the binary parameter value. The name is the parameter class name. The value is the binary format of the parameter, for example, protocol buffer encoded value.
    • ExecCoprocessorResponse: Result of the call.
    • MultiRequest: Execute multiple actions on a table: get, mutate, and/or execCoprocessor. You can execute a list of actions on a given region in order. If it is a list of mutate actions, atomic can be set to make sure they can be processed atomically, just like RowMutations.
    • MultiResponse: List of action due to the operations happened.

RegionServerStatusProtocol, Protocol that a RegionServer uses to communicate its status to the Master.

  • RegionServerStatusService

    • RegionServerStartupRequest:  Sent when the region server first starts up. req contains port num, start code, and the server current time in ms.
    • RegionServerStartupResponse: Response from master  contains configuration (list of NameStringPair)for the regionserver to use: e.g. filesystem, hbase rootdir, the hostname to use creating the RegionServer ServerName, etc
    • RegionServerReportRequest: Periodical report sent from region server (ServerName) which contains ServerLoad.
    • RegionServerReportResponse: Empty Response
    • ReportRSFatalErrorRequest: Report the Region Server abort (may be due to Oom or HDFS problems) to master. Req contains the ServerName and error Message.
    • ReportRSFatalErrorResponse: Empty Response ( no one might receive this :-) )

 

Internal Messages or Data Structures in Protocol Buffers

  • ClusterId

    • ClusterId : Content of the ‘/hbase/hbaseid’, cluster id, znode.
  • ClusterStatus

    • RegionState: State of the region while undergoing transition. Has  RegionInfo, State and  the timestamp.
    • RegionInTransition: Identifies all the regions in Transition. Has RegionSpecifier and  RegionState.
    • LiveServerInfo: Describes each Live server with its ServerName and  the current ServerLoad.
    • ClusterStatus: ClusterStatus provides clients with information such as
      • The count and names of region servers in the cluster.
      •  The count and names of dead region servers in the cluster
      • The name of the active master for the cluster.
      •  The name(s) of the backup master(s) for the cluster, if they exist.
      •  The average cluster load.
      •  The number of regions deployed on the cluster.
      •  The number of requests since last report.
      •  Detailed region server loading and resource usage information,
      •  per server and per region.
      •  Regions in transition at master
      •  The unique cluster ID
      • Has HBaseVersionFileContentLiveServerInfoServerName (for dead servers), RegionInTransitionClusterIdCoprocessorServerName (for master), ServerName (for backupMaster) etc.
  • FS

    • HBaseVersionFileContent: The ${HBASE_ROOTDIR}/hbase.version file content
    • Reference: Reference file content used when we split an hfile under a region
  • HBase

    • TableSchema: Describes a table, has Table name and ColumnFamilySchema for each column family present.
    • ColumnFamilySchema: Describes the column family name and other attributes like Bloom filter, inMemory etc.
    • RegionInfo: Protocol buffer version of HRegionInfo. Has information like regionId, table name, start key, end key etc.
    • RegionSpecifier: You can specify region by region name, or the hash of the region name, which is known as encoded region name.
    • RegionLoad: Describes the Load on a given region server. Has informations about store files, req counts, index size and bloom size.
    • ServerLoad: Describes the load on a given server. Has information like req count, memory used, RegionLoads, Coprocessor etc.
    • TimeRange: A range of time. Both from and to are Java time stamp in milliseconds. If you don’t specify a time range, it means all time. By default, if not specified, from = 0, and to = Long.MAX_VALUE
    • KeyValue: The generic key type used in HLog. Has information like row, family, qualifier, timestamp, and the key type for mutable operations.
    • ServerName: Host , port of a server.
    • Coprocessor: Name in string.
    • NameStringPair: A generic name value pair in String format.
    • NameBytesPair: A generic name value pair in bytes format.
  • RPC

    • UserInformation : A real and effective user name
    • ConnectionHeader: Contains User Info beyond what is established at connection establishment and the protocol used. As part of setting up a connection to a server, the client needs to send the ConnectionHeader header. At the data level, this looks like <“hrpc”-bytearray><5-byte><length-of-serialized-ConnectionHeader-obj[int32]><ConnectionHeader-object serialized>
    • RpcRequest: For every RPC that the client makes it needs to send the RpcRequest. At the data level this looks like <length-of-serialized-RpcRequest-obj><RpcRequest-object serialized>
    • RpcException: The Server sends an exception message if the request throws an exception. It contains the class which threw up and the stack.
    • RpcResponse: The server sends back a RpcResponse object as response.At the data level this looks like <protobuf-encoded-length-of-serialized-RpcResponse-obj><RpcResponse-object serialized>. RpcException is optional here.
  • Zookeeper

    • RootRegionServer: Content of the root-region-server znode which contains the ServerName hosting the root region currently.
    • Master: ServerName of the current master store in master znode.
    • ClusterUp: Content of the ‘/hbase/shutdown’, cluster state, znode.  If this znode is present, cluster is up. Currently the data is cluster startDate.
    • RegionTransition: What we write under unassigned up in zookeeper as a region moves through open/close, etc., regions. Details a region in transition. Contains originServerName, regionName, createTime etc.
    • SplitLogTask: WAL SplitLog directory znodes have this for content. Used doing distributed WAL splitting. Holds current state and name of server that originated split. Contains state and SevrverName.
    • Table: The znode that holds state of table (Enabled, Disabled etc). If no znode for a table, its state is presumed enabled.

 

 

 

 

 

 

 

HBase Development Environment

The first step for digging in to the hbase code is to setup the proper development environment. Here are some notes i made when preparing myself for making my hands dirty with hbase. Although this is done in OS X Mountain Lion, this should be fairly same in most of the Unix’ish OS.

 

 Install Protocol Buffers

  1. Download protocol buffers from this link
  2. untar it and cd in to that directory.
  3. Run “./configure”,”make” and “sudo make install” from command line to install “protoc”
  4. In OS X, ./configure will probabaly complain about “C Compiler” and “cmake” missing. You can choose to install it by getting  XCode command line tools and cmake through “brew install cake” if you are a brew user.

 

Download and Compile HBase

  1. git clone git://git.apache.org/hbase.git
  2. mvn clean install -DskipTests
  3. “mvn eclipse:eclipse” to create eclipse related files (.project )
       First time compile takes time as all the required libraries are downloaded from the internet.

 

Setup Eclipse

  1. Download Eclipse (preferably the latest one)
  2. Install the code formatting xml (Look at HBASE-3678 for details)