Update README
This commit is contained in:
parent
143a0dca25
commit
ea77e70fdf
101
README.md
101
README.md
@ -2,50 +2,107 @@
|
|||||||
|
|
||||||
## An HBase Prometheus Exporter
|
## An HBase Prometheus Exporter
|
||||||
|
|
||||||
Relays JMX metrics for consumption by Prometheus
|
Collects metrics and relays JMX metrics for consumption by Prometheus
|
||||||
|
|
||||||
The exporter parses the (Hortonworks) Hadoop config and uses (Hortonworks)
|
|
||||||
Hadoop internal tooling to determine e.g. the currently active master
|
|
||||||
so that it can run in different environments without requiring any configuration
|
|
||||||
|
|
||||||
Those tools are not yet included when building the app as a container
|
|
||||||
|
|
||||||
Since some important metrics are missing or empty in JMX, we additionally parse the HBase Master UI
|
Since some important metrics are missing or empty in JMX, we additionally parse the HBase Master UI
|
||||||
for e.g. 'Stale regions in transition'
|
for e.g. 'Stale regions in transition'
|
||||||
|
|
||||||
The hbase hbck log is parsed to check for inconsistencies in HBase.
|
The output of the 'hbase hbck' command is parsed to check for inconsistencies in HBase
|
||||||
The log is created independently from the exporter with the help of
|
|
||||||
a systemd-timer unit and a systemd-hbck-service unit
|
|
||||||
|
|
||||||
Unfortunately querying the active namenode requires superuser privileges
|
Marking Hbase unhealthy requires one of the following conditions to be true
|
||||||
|
- There is at least one stale region in transition
|
||||||
|
- The 'hbase hbck' command shows HBase inconsistencies
|
||||||
|
- A write to the predefined table does not succeed
|
||||||
|
- A ZooKeeper leader can not be determined
|
||||||
|
|
||||||
|
|
||||||
|
### Dependencies
|
||||||
For python module requirements see requirements.txt
|
For python module requirements see requirements.txt
|
||||||
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
$ sudo dnf/pkg install python36
|
$ sudo dnf/pkg install python36
|
||||||
```
|
```
|
||||||
|
|
||||||
|
As the user executing the exporter:
|
||||||
```sh
|
```sh
|
||||||
# As the user executing the exporter:
|
|
||||||
$ pip3[.6] install --user -r requirements.txt
|
$ pip3[.6] install --user -r requirements.txt
|
||||||
```
|
```
|
||||||
|
|
||||||
To generate the necessary HBase Python Protobuf bindings, run the Makefile
|
The protobuf compiler is necessary to build the required bindings for Python
|
||||||
The protobuf compiler is necessary to build them
|
|
||||||
|
|
||||||
Install the protobuf compiler
|
Install the protobuf compiler
|
||||||
```
|
```
|
||||||
$ sudo dnf/pkg install protobuf-c
|
$ sudo dnf/pkg install protobuf-c protobuf-devel
|
||||||
```
|
```
|
||||||
|
|
||||||
Run make
|
### Build
|
||||||
|
To generate the necessary HBase Python Protobuf bindings, run make
|
||||||
```
|
```
|
||||||
$ make
|
$ make
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Run
|
||||||
|
The exporter needs to know about the ZooKeeper servers to connect to, so start
|
||||||
|
the exporter with e.g.
|
||||||
|
```
|
||||||
|
$ hbase-exporter --zookeeper-server-address=zk-1.acme.internal
|
||||||
|
--zookeeper-server-address=zk-2.acme.internal
|
||||||
|
--zookeeper-server-address=zk-3.acme.internal
|
||||||
|
--export-refresh-rate=60
|
||||||
|
--hbck-refresh-rate=1200
|
||||||
|
```
|
||||||
|
|
||||||
|
Run 'hbase-exporter --help' for all arguments
|
||||||
|
```
|
||||||
|
$ hbase-exporter --help
|
||||||
|
usage: hbase-exporter [-h] [--hbase-master HBASE_MASTER]
|
||||||
|
[--hdfs-namenode HDFS_NAMENODE]
|
||||||
|
--zookeeper-server-address ZK_SERVER
|
||||||
|
[--zookeeper-use-tls ZK_USE_TLS]
|
||||||
|
[--exporter-port PROM_HTTP_PORT]
|
||||||
|
[--export-refresh-rate PROM_EXPORT_INTERVAL_S]
|
||||||
|
[--hbck-refresh-rate HBASE_HBCK_INTERVAL_S]
|
||||||
|
[--relay-jmx RELAY_JMX] [--logfile LOGFILE]
|
||||||
|
[--loglevel LOGLEVEL]
|
||||||
|
|
||||||
|
optional arguments:
|
||||||
|
-h, --help show this help message and exit
|
||||||
|
--hbase-master HBASE_MASTER
|
||||||
|
HBase master address, can be specified multiple times
|
||||||
|
--hdfs-namenode HDFS_NAMENODE
|
||||||
|
HDFS namenode address, can be specified multiple times
|
||||||
|
--zookeeper-server-address ZK_SERVER
|
||||||
|
ZooKeeper server address, can be specified multiple
|
||||||
|
times
|
||||||
|
--zookeeper-use-tls ZK_USE_TLS
|
||||||
|
Use TLS when connecting to ZooKeeper
|
||||||
|
--exporter-port PROM_HTTP_PORT
|
||||||
|
Listen port for Prometheus export
|
||||||
|
--export-refresh-rate PROM_EXPORT_INTERVAL_S
|
||||||
|
Time between metrics are gathered in seconds
|
||||||
|
--hbck-refresh-rate HBASE_HBCK_INTERVAL_S
|
||||||
|
Minimum time between two consecutive hbck runs in
|
||||||
|
seconds
|
||||||
|
--relay-jmx RELAY_JMX
|
||||||
|
Relay complete JMX data
|
||||||
|
--logfile LOGFILE Path to optional logfile
|
||||||
|
--loglevel LOGLEVEL Loglevel, default: INFO
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deploy
|
||||||
|
Ansible can be used to build and deploy the hbase-exporter
|
||||||
|
```
|
||||||
|
ansible-playbook -v -i inventory/env.yml deploy-hbase-exporter.yml -l host
|
||||||
|
````
|
||||||
|
|
||||||
|
### DEBUG
|
||||||
|
To see the log
|
||||||
|
```
|
||||||
|
$ sudo journalctl -afn100 -uhbase-exporter
|
||||||
|
```
|
||||||
|
|
||||||
## Issues
|
## Issues
|
||||||
Import paths do not work in generated protobufs when used from a subdir
|
Import paths do not work in generated protobufs when used from a subdir<br>
|
||||||
https://github.com/protocolbuffers/protobuf/issues/1491
|
https://github.com/protocolbuffers/protobuf/issues/1491<br>
|
||||||
The solution used here is mentioned in the comments:
|
The solution used here is mentioned in the comments:<br>
|
||||||
https://github.com/protocolbuffers/protobuf/issues/1491#issuecomment-547504972
|
https://github.com/protocolbuffers/protobuf/issues/1491#issuecomment-547504972<br>
|
||||||
|
Unfortunately querying the active namenode requires superuser privileges
|
||||||
|
Loading…
x
Reference in New Issue
Block a user