एचबेस Archiटेक्चर: उपयोग के मामले, घटक और डेटा मॉडल

Apache HBase is a distributed, column-oriented NoSQL database that runs on top of Hadoop and the Hadoop Distributed File System (HDFS). Its architecture combines a coordinating master, region servers, and ZooKeeper to store very large tables and serve fast random reads and writes.

एचबेस Archiटेकचर और इसके महत्वपूर्ण घटक

HBase architecture has the following main components:

• एचएमस्टर
• एचरीजनसर्वर
• एचक्षेत्र
• चिड़ियाघर संचालक
• एचडीएफएस

Below is a detailed architecture of HBase with its components, as shown in the diagram.

एचएमस्टर

HMaster in HBase is the implementation of a Master server in HBase architecture. It acts as a monitoring agent to monitor all Region Server instances present in the cluster and acts as an interface for all the metadata changes. In a distributed cluster environment, the Master runs on the NameNode. The Master runs several background threads.

The following are important roles performed by HMaster in HBase:

• क्लस्टर में नोड्स के प्रदर्शन और रखरखाव के संदर्भ में महत्वपूर्ण भूमिका निभाता है।
• HMaster व्यवस्थापकीय कार्य निष्पादन प्रदान करता है और विभिन्न क्षेत्र सर्वरों को सेवाएं वितरित करता है।
• HMaster क्षेत्र सर्वरों को क्षेत्र निर्दिष्ट करता है।
• HMaster controls load balancing and failover to handle the load over nodes present in the cluster.
• When a client wants to change any schema or any metadata operation, HMaster takes responsibility for these operations.

Some of the methods exposed by the HMaster interface are primarily metadata-oriented methods:

• तालिका (createTable, removeTable, सक्षम, अक्षम)
• कॉलमफ़ैमिली (कॉलम जोड़ें, कॉलम संशोधित करें)
• क्षेत्र (स्थानांतरित करें, असाइन करें)

The client communicates in a bi-directional way with both HMaster and ZooKeeper. For read and write operations, it directly contacts the HRegion servers. HMaster assigns regions to region servers and, in turn, checks the health status of region servers.

In the entire architecture, we have multiple region servers. An HLog is present in the region servers, which stores all the log files.

HBase क्षेत्र सर्वर

When an HBase Region Server receives write and read requests from the client, it assigns the request to a specific region, where the actual column family resides. The client can directly contact the HRegion servers; there is no need for mandatory HMaster permission for the client to communicate with the HRegion servers. The client requires HMaster help only when operations related to metadata and schema changes are required.

HRegionServer is the Region Server implementation. It is responsible for serving and managing regions, or the data that is present in a distributed cluster. The region servers run on the Data Nodes present in the Hadoop cluster.

HMaster can get into contact with multiple HRegion servers and performs the following functions:

• क्षेत्रों की मेजबानी और प्रबंधन
• क्षेत्रों का स्वचालित रूप से विभाजन
• Handling read and write requests
• ग्राहक से सीधे संवाद करना

HBase क्षेत्र

HRegions are the basic building elements of an HBase cluster. They consist of the distribution of tables and are comprised of column families. A region contains multiple stores, one for each column family. It mainly consists of two components: the MemStore and the HFile.

चिड़ियाघर संचालक

एचबेस चिड़ियाघर संचालक is a centralized monitoring server that maintains configuration information and provides distributed synchronization. Distributed synchronization coordinates the distributed applications running across the cluster, providing coordination services between nodes. If the client wants to communicate with regions, the client has to approach ZooKeeper first.

It is an open source project, and it provides many important services.

Services provided by ZooKeeper:

• Maintains configuration information
• वितरित तुल्यकालन प्रदान करता है
• Establishes client communication with region servers
• Provides ephemeral nodes that represent different region servers
• Lets the Master server use these ephemeral nodes to discover available servers in the cluster
• Tracks server failure and network partitions

The Master and HBase slave nodes (region servers) register themselves with ZooKeeper. The client needs access to the ZooKeeper (ZK) quorum configuration to connect with the master and region servers.

During a failure of nodes present in the HBase cluster, the ZooKeeper quorum triggers error messages and starts to repair the failed nodes.

एचडीएफएस

HDFS is the Hadoop Distributed संचिका तंत्र. As the name implies, it provides a distributed environment for storage, and it is a file system designed to run on commodity hardware. It stores each file in multiple blocks, and to maintain fault tolerance, the blocks are replicated across a Hadoop cluster.

HDFS provides a high degree of fault tolerance and runs on cheap commodity hardware. By adding nodes to the cluster and performing processing and storing using cheap commodity hardware, it gives the client better results compared to the existing setup.

Here, the data stored in each block is replicated across 3 nodes, so if any node goes down there will be no loss of data; it has a proper backup and recovery mechanism.

HDFS gets in contact with the HBase components and stores a large amount of data in a distributed manner.

HBase डेटा मॉडल

The HBase Data Model is a set of components that consists of Tables, Rows, Column families, Cells, Columns, and Versions. HBase tables contain column families and rows, with elements defined as primary keys. A column in the HBase data model table represents an attribute of the objects.

The HBase Data Model consists of the following elements:

• तालिकाओं का सेट
• स्तंभ परिवारों और पंक्तियों के साथ प्रत्येक तालिका
• Each table must have an element defined as a primary key.
• The row key acts as a primary key in HBase.
• Any access to HBase tables uses this primary key.
• Each column present in HBase denotes an attribute corresponding to an object.

HBase उपयोग के मामले

The following are examples of HBase use cases with a detailed explanation of the solution HBase provides to various technical problems.

समस्या का विवरण	उपाय
The telecom industry faces the following technical challenges: storing billions of Call Detail Record (CDR) log records generated by the telecom domain; providing real-time access to CDR logs and billing information of customers; and providing a cost-effective solution compared to traditional database systems.	HBase का उपयोग विस्तृत कॉल रिकॉर्ड की अरबों पंक्तियों को संग्रहीत करने के लिए किया जाता है। यदि मौजूदा RDBMS डेटाबेस में प्रति माह 20TB डेटा जोड़ा जाता है, तो प्रदर्शन खराब हो जाएगा। इस उपयोग के मामले में बड़ी मात्रा में डेटा को संभालने के लिए, HBase सबसे अच्छा समाधान है। HBase तेजी से क्वेरी करता है और रिकॉर्ड प्रदर्शित करता है।
The banking industry generates millions of records on a daily basis. In addition, the banking industry also needs an analytics solution that can detect fraud in money transactions.	To store, process, and update vast volumes of data and to perform analytics, an ideal solution is HBase integrated with several Hadoop ecosystem components.

Apart from that, HBase can be used:

• Whenever there is a need for write-heavy applications.
• For performing online log analytics and generating compliance reports.

HBase में भंडारण तंत्र

HBase is a column-oriented database, and data is stored in tables. The tables are sorted by RowId. As shown below, HBase has a RowId, which is the collection of several column families that are present in the table.

The column families that are present in the schema are key-value pairs. If we observe in detail, each column family has multiple columns. The column values are stored on disk memory. Each cell of the table has its own metadata, such as a timestamp and other information.

The column-oriented storage layout, with row keys, column families, and cells, is shown below.

The following are the key terms representing an HBase table schema:

• Table: Collection of rows present.
• Row: Collection of column families.
• Column Family: Collection of columns.
• Column: Collection of key-value pairs.
• Namespace: Logical grouping तालिकाओं का।
• Cell: A {row, column, version} tuple that exactly specifies a cell definition in HBase.

स्तंभ-उन्मुख बनाम पंक्ति-उन्मुख भंडारण

Column-oriented and row-oriented storages differ in their storage mechanism. As we all know, traditional relational models store data in a row-based format, in terms of rows of data. Column-oriented storages store data tables in terms of columns and column families.

The following table gives some key differences between these two storages.

स्तंभ-उन्मुख डेटाबेस	Row-oriented Database
Used when the situation involves processing and analytics, such as Online Analytical Processing and its applications.	Online Transactional Processing, such as banking and finance domains, uses this approach.
The amount of data that can be stored in this model is very large, in terms of petabytes.	इसे छोटी संख्या में पंक्तियों और स्तंभों के लिए डिज़ाइन किया गया है।

HBase डेटा पढ़ने और लिखने की व्याख्या

The read and write operations from the client into the HFile are shown in the diagram below.

Step 1) The client wants to write data and, in turn, first communicates with the Region Server and then the regions.

Step 2) The region contacts the MemStore for storing the data associated with the column family.

Step 3) First, the data is stored in the MemStore, where the data is sorted, and after that it flushes into the HFile. The main reason for using the MemStore is to store data in a distributed file system based on the row key. The MemStore is placed in the Region Server main memory, while HFiles are written into HDFS.

Step 4) The client wants to read data from the regions.

Step 5) In turn, the client can have direct access to the MemStore and can request data.

Step 6) The client approaches the HFiles to get the data. The data is fetched and retrieved by the client.

The MemStore holds in-memory modifications to the store. The hierarchy of objects in HBase regions, from top to bottom, is shown in the table below.

तालिका	HBase क्लस्टर में मौजूद HBase तालिका
क्षेत्र	प्रस्तुत तालिकाओं के लिए HRegions
दुकान	Stores one per column family for each region for the table
मेमस्टोर	MemStore for each store for each region for the table. It sorts data before flushing into HFiles. Write and read performance increase because of sorting.
स्टोरफ़ाइल	तालिका के लिए प्रत्येक क्षेत्र के लिए प्रत्येक स्टोर के लिए स्टोरफ़ाइल्स
खंड	स्टोरफाइल्स के अंदर मौजूद ब्लॉक

HBase बनाम HDFS

HBase runs on top of HDFS and Hadoop. Some key differences between HDFS and HBase are in terms of data operations and processing.

एचबेस	एचडीएफएस
कम विलंबता संचालन	उच्च विलंबता संचालन
यादृच्छिक पढ़ना और लिखना	Write once, read many times
के माध्यम से पहुँचा शैल कमांड, a client API in Java, REST, Avro, or Thrift	Primarily accessed through MapReduce (MR) jobs
Both storage and processing can be performed	It is only for storage areas

Some typical IT industrial applications use HBase operations along with Hadoop. Applications include stock exchange data and online banking data operations, where HBase is the best-suited solution. Once your cluster is ready, you can read and write data in HBase or install HBase on a fresh node.

अक्सर पूछे जाने वाले प्रश्न