fulmanski.pl: tutorials

How to install OpenTSDB

Initial version: 2024-10-15
Last update: 2024-10-19

In this tutorial you will learn how to install OpenTSDB.

Table of contents

Introduction
Installation
Working with OpenTSDB – first tests
What you should do after restart your system
Ignore sar's header.

Introduction

OpenTSDB is a distributed, scalable Time Series Database (TSDB) written to address a common need: store, index and serve metrics collected from computer systems (network gear, operating systems, applications) at a large scale, and make this data easily accessible and graphable. It is written on top of HBase. Thanks to HBase's scalability, OpenTSDB allows you to collect thousands of metrics from tens of thousands of hosts and applications, at a high rate (every few seconds). OpenTSDB will never delete or downsample data and can easily store hundreds of billions of data points. GutHub:OpenTSDB

Installation

Runtime requirements

To actually run OpenTSDB, you’ll need to meet the following:

A Linux system (or Windows with manual building),
Java Runtime Environment 1.6 or later,
HBase 0.92 or later,
GnuPlot 4.2 or later.

Step 1: HBase installation

Because OpenTSDB is written on top of HBase this is the first component you should install.

Personally I advice you test with a stand-alone instance of HBase as this is the easiest to get up and running. You can complete this step following for example my tutorial HBase install in 2024 year

When you complete HBase installation make sure that Zookeeper is accessible. One method is to simply telnet to the proper port and execute the stats command:

Start HBase:


nosql@nosql-virtualbox:~$ /home/nosql/Desktop/nosql/hbase-2.5.10/bin/start-hbase.sh

Verify if all HBase prcesses are running:


nosql@nosql-virtualbox:~$ jps
12944 HMaster
13423 Jps

Check if JAVA_HOME environment variable is set (you will need it in a future; this should be visible as it is needed by HBase and is set either by HBase based on its configuration file or in your command interpreter configuration file like .bashrc):


nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ echo $JAVA_HOME
/usr/lib/jvm/java-1.21.0-openjdk-amd64

If not, set this variable as I did for example in HBase install in 2024 year

Telnet to Zookeeper:


nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ telnet localhost 2181
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

Type stats command:


stats
Zookeeper version: 3.8.4-9316c2a7a97e1666d8f4593f34dd6fc36ecc436c, built on 2024-02-12 22:16 UTC
Clients:
 /127.0.0.1:56086[1](queued=0,recved=325,sent=339)
 /127.0.0.1:39510[0](queued=0,recved=1,sent=0)
 /127.0.0.1:43428[1](queued=0,recved=405,sent=426)

Latency min/avg/max: 0/1.5397/72
Received: 732
Sent: 766
Connections: 3
Outstanding: 0
Zxid: 0x2d
Mode: standalone
Node count: 31
Connection closed by foreign host.

If you see information similar to mine above than you can proceed with next steps.

Step 2 [failed]: Compile from source

OpenTSDB may be compiled from source or installed from a package. Releases can be found on Github.

I made some attempts to install OpenTSDB from source, however I failed. Below I give what I did – maybe you will push it to the end; if you will manage to install it this way, please let me know what I did incorrectly. Finally I have installed it from package.

Step 2.1: Meet compile requirements

If you decide to compile it, please note that you require:

A Linux system,
Java Development Kit (1.6 or later),
GnuPlot (4.2 or later),
Autotools,
Make,
Python,
Git (if you want to clone repository).

You can use any package manager (for example Synaptic in Ubuntu) or complete this with command line:


nosql@nosql-virtualbox:~$ sudo apt-get update
nosql@nosql-virtualbox:~$ sudo apt-get upgrade
nosql@nosql-virtualbox:~$ sudo apt-get install autoconf
nosql@nosql-virtualbox:~$ sudo apt-get install automake
nosql@nosql-virtualbox:~$ sudo apt-get install gnuplot
nosql@nosql-virtualbox:~$ sudo apt-get install git

Step 2.2: Get the source and compile it

At the time of writing this tutorial (October, 2024) default version of OpenTSDB is 2.X (2.4.1 dated 2021-09-02):

Figure: OpenTSDB main page

New planned version 3.X is under active development and is not stable – you can find more about this on OpenTSDB 3.0 documentation page

Download the latest 2.X version using git clone command or download a release from the site or Github:

Figure: OpenTSDB download page

Figure: Location where I saved the file

As you may notice I have saved it in my home directory (which is nosql) under the Desktop/install/opentsdb dirctory.

Now you have to move to that directory and extract archive:


nosql@nosql-virtualbox:~$ cd Desktop/install/opentsdb/
nosql@nosql-virtualbox:~/Desktop/install/opentsdb$ tar zxvf opentsdb-2.4.1.tar.gz

Navigate to the extracted directory:


nosql@nosql-virtualbox:~/Desktop/install/opentsdb$ cd opentsdb-2.4.1/
nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$

nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ ls -l
total 232
-rw-rw-r--  1 nosql nosql   813 Sep  3  2021 AUTHORS
-rwxrwxr-x  1 nosql nosql   740 Sep  3  2021 bootstrap
drwxrwxr-x  4 nosql nosql  4096 Sep  3  2021 build-aux
-rw-rw-r--  1 nosql nosql   221 Sep  3  2021 build-bigtable.sh
-rw-rw-r--  1 nosql nosql   222 Sep  3  2021 build-cassandra.sh
-rwxrwxr-x  1 nosql nosql   206 Sep  3  2021 build.sh
-rw-rw-r--  1 nosql nosql  2278 Sep  3  2021 configure.ac
-rw-rw-r--  1 nosql nosql  4551 Sep  3  2021 CONTRIBUTING.md
-rw-rw-r--  1 nosql nosql 35147 Sep  3  2021 COPYING
-rw-rw-r--  1 nosql nosql 26530 Sep  3  2021 COPYING.LESSER
drwxrwxr-x  2 nosql nosql  4096 Sep  3  2021 fat-jar
-rw-rw-r--  1 nosql nosql 37807 Sep  3  2021 Makefile.am
-rw-rw-r--  1 nosql nosql 26578 Sep  3  2021 NEWS
-rw-rw-r--  1 nosql nosql  4125 Sep  3  2021 opentsdb.spec.in
-rw-rw-r--  1 nosql nosql 18997 Sep  3  2021 pom.xml.in
-rw-rw-r--  1 nosql nosql  1024 Sep  3  2021 README
-rw-rw-r--  1 nosql nosql   384 Sep  3  2021 screwdriver.yaml
drwxrwxr-x 17 nosql nosql  4096 Sep  3  2021 src
drwxrwxr-x 19 nosql nosql  4096 Sep  3  2021 test
-rw-rw-r--  1 nosql nosql  3465 Sep  3  2021 THANKS
drwxrwxr-x 26 nosql nosql  4096 Sep  3  2021 third_party
drwxrwxr-x  3 nosql nosql  4096 Sep  3  2021 tools
-rw-rw-r--  1 nosql nosql  3310 Sep  3  2021 tsdb.in

Then just run the build.sh script:


nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ ./build.sh 
+ test -f configure
+ ./bootstrap
autoreconf: export WARNINGS=
autoreconf: Entering directory '.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I build-aux
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: configure.ac: not using Intltool
autoreconf: configure.ac: not using Gtkdoc
autoreconf: running: /usr/bin/autoconf --force
autoreconf: configure.ac: not using Autoheader
autoreconf: running: automake --add-missing --copy --force-missing
configure.ac:19: installing 'build-aux/install-sh'
configure.ac:19: installing 'build-aux/missing'
Makefile.am:933: warning: whitespace following trailing backslash
third_party/validation-api/include.mk:24: warning: variable 'VALIDATION_API_SOURCES' is defined but no program or
third_party/validation-api/include.mk:24: library has 'VALIDATION_API' as canonical name (possible typo)
Makefile.am:995:   'third_party/include.mk' included from here
third_party/include.mk:38:   'third_party/validation-api/include.mk' included from here
autoreconf: Leaving directory '.'
+ test -d build
+ mkdir build
+ cd build
+ test -f Makefile
+ ../configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
[...]

In theory this should be an easy and fast step. However in my case I have encountered a problem:


[...]
../build-aux/gen_build_data.sh src/tools/BuildData.java net.opentsdb.tools 2.4.1
Generating src/tools/BuildData.java
../build-aux/gen_build_data.sh: line 36: python: command not found
make: *** [Makefile:1858: src/tools/BuildData.java] Error 127

As you can see there were some problems with python command:


[...]
../build-aux/gen_build_data.sh: line 36: python: command not found
[...]

The problem was that I have python3 installed but not python:


nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ type python
bash: type: python: not found
nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ type python2
bash: type: python2: not found
nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ type python3
python3 is /usr/bin/python3

I tried to solve this problem making temporarily python3 reachable via alias python:


nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ alias python=python3
nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ type -a python
python is aliased to `python3'

but this approach did not solve this problem.

Another approach was to create a soft link from python to python3:


nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ whereis python3
python3: /usr/bin/python3 /usr/lib/python3 /etc/python3 /usr/share/python3 /usr/share/man/man1/python3.1.gz
nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ sudo ln -s /usr/bin/python3 /usr/bin/python

To verify new soft link use ls command:


nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ ls -l /usr/bin/python /usr/bin/python3
lrwxrwxrwx 1 root root 16 Oct 15 16:22 /usr/bin/python -> /usr/bin/python3
lrwxrwxrwx 1 root root 10 Aug  7 19:44 /usr/bin/python3 -> python3.12

This helped and the installation procedure continued however I faced another one problem:


nosql@nosql-virtualbox:~/Desktop/install/opentsdb/opentsdb-2.4.1$ ./build.sh 
+ test -f configure
+ test -d build
+ cd build
[...]
sion/parser/Token.java ./src/net/opentsdb/query/expression/parser/TokenMgrError.java
warning: [options] bootstrap class path not set in conjunction with -source 6
error: Source option 6 is no longer supported. Use 8 or later.
make[1]: *** [Makefile:1865: .javac-stamp] Error 2
make[1]: Leaving directory '/home/nosql/Desktop/install/opentsdb/opentsdb-2.4.1/build'
make: *** [Makefile:1417: all] Error 2

It seems that installation procedure use to compile some Java sources a third party "ancient" version of Java instead installed in my system quite new OpenJDK in version 21:


tutu sprawdzanie wersji javy: java -version

Step 2 [alternative, successful]: Install from package


nosql@nosql-virtualbox:~/Desktop/install/opentsdb$ sudo dpkg -i opentsdb-2.4.1_all.deb 
[sudo] password for nosql: 
Selecting previously unselected package opentsdb.
(Reading database ... 186017 files and directories currently installed.)
Preparing to unpack opentsdb-2.4.1_all.deb ...
Unpacking opentsdb (2.4.1) ...
Setting up opentsdb (2.4.1) ...
info: Selecting UID from range 100 to 999 ...

info: Adding system user `opentsdb' (UID 120) ...
info: Adding new user `opentsdb' (UID 120) with group `opentsdb' ...
info: Not creating home directory `/usr/share/opentsdb'.

Installation includes an init script at /etc/init.d/opentsdb that can start, stop and restart OpenTSDB. Simply call service opentsdb start to start the TSD and service opentsdb stop to gracefully shutdown. Before you call them, please read note below.

If you install OpenTSDB for the first time on your computer, you’ll need to create the HBase tables using the script located at /usr/share/opentsdb/tools/create_table.sh. Execute:


env COMPRESSION=NONE HBASE_HOME=path/to/hbase-0.94.X ./src/create_table.sh

This will create four tables: tsdb, tsdb-uid, tsdb-tree and tsdb-meta. If you’re just evaluating OpenTSDB, don’t worry about compression for now and leave it as NONE. In production and at scale, make sure you use a valid compression library as it will save on storage tremendously (the COMPRESSION value is either NONE, LZO, GZIP or SNAPPY).

The whole process may look as it is given below:

Check tables present in Hbase


nosql@nosql-virtualbox:~$ Desktop/nosql/hbase-2.5.10/bin/hbase shell
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.5.10, ra3af60980c61fb4be31e0dcd89880f304d01098a, Thu Jul 18 22:45:17 PDT 2024
Took 0.0038 seconds                                                                                                          
hbase:001:0> list
TABLE                                                                                                                        
0 row(s)
Took 0.6053 seconds                                                                                                          
=> []
hbase:002:0>

Open another one terminal and execute create_table.sh script with proper location of your HBase instance and script itself:


nosql@nosql-virtualbox:~/Desktop/install/opentsdb$ env COMPRESSION=NONE HBASE_HOME=/home/nosql/Desktop/nosql/hbase-2.5.10/ /usr/share/opentsdb/tools/create_table.sh
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.5.10, ra3af60980c61fb4be31e0dcd89880f304d01098a, Thu Jul 18 22:45:17 PDT 2024
Took 0.0056 seconds                                                                                                          
hbase:001:0> create 'tsdb-uid',
hbase:002:0*   {NAME => 'id', COMPRESSION => 'NONE', BLOOMFILTER => 'ROW', DATA_BLOCK_ENCODING => 'DIFF'},
hbase:003:0*   {NAME => 'name', COMPRESSION => 'NONE', BLOOMFILTER => 'ROW', DATA_BLOCK_ENCODING => 'DIFF'}
Created table tsdb-uid
Took 1.5205 seconds                                                                                                          
=> Hbase::Table - tsdb-uid
hbase:004:0> 
hbase:005:0> create 'tsdb',
hbase:006:0*   {NAME => 't', VERSIONS => 1, COMPRESSION => 'NONE', BLOOMFILTER => 'ROW', DATA_BLOCK_ENCODING => 'DIFF', TTL => 'FOREVER'}
Created table tsdb
Took 0.6958 seconds                                                                                                          
=> Hbase::Table - tsdb
hbase:007:0>   
hbase:008:0> create 'tsdb-tree',
hbase:009:0*   {NAME => 't', VERSIONS => 1, COMPRESSION => 'NONE', BLOOMFILTER => 'ROW', DATA_BLOCK_ENCODING => 'DIFF'}
Created table tsdb-tree
Took 0.6405 seconds                                                                                                          
=> Hbase::Table - tsdb-tree
hbase:010:0>   
hbase:011:0> create 'tsdb-meta',
hbase:012:0*   {NAME => 'name', COMPRESSION => 'NONE', BLOOMFILTER => 'ROW', DATA_BLOCK_ENCODING => 'DIFF'}
Created table tsdb-meta
Took 0.6488 seconds                                                                                                          
=> Hbase::Table - tsdb-meta
hbase:013:0>

Again list all tables present in HBase:


hbase:002:0> list
TABLE                                                                                                                        
tsdb                                                                                                                         
tsdb-meta                                                                                                                    
tsdb-tree                                                                                                                    
tsdb-uid                                                                                                                     
4 row(s)
Took 0.0871 seconds                                                                                                          
=> ["tsdb", "tsdb-meta", "tsdb-tree", "tsdb-uid"]
hbase:003:0>

As you can see all after-mentioned tables exist.

If you are here there is a high probability that you have in your HBase a required tables; if not, please read note above.

Working with OpenTSDB – first tests

Start OpenTSDB

Now you can safely start OpenTSDB and check if its running:


nosql@nosql-virtualbox:~$ service opentsdb start

You will be asked to enter superuser password:

Figure: Request for superuser password at starting OpenTSDB

You can verify status of OpenTSDB with systemctl status opentsdb command:


nosql@nosql-virtualbox:~$ sudo systemctl status opentsdb
[sudo] password for nosql: 
● opentsdb.service - LSB: Starts OpenTSDB TSD
     Loaded: loaded (/etc/init.d/opentsdb; generated)
     Active: active (running) since Tue 2024-10-15 19:02:57 CEST; 14min ago
       Docs: man:systemd-sysv-generator(8)
    Process: 21608 ExecStart=/etc/init.d/opentsdb start (code=exited, status=0/SUCCESS)
      Tasks: 22 (limit: 4616)
     Memory: 77.1M (peak: 78.4M)
        CPU: 12.285s
     CGroup: /system.slice/opentsdb.service
             └─21621 java -enableassertions -enablesystemassertions -classpath "/usr/share/opentsdb/*.jar:/usr/share/opentsd>

Oct 15 19:02:57 nosql-virtualbox systemd[1]: Starting opentsdb.service - LSB: Starts OpenTSDB TSD...
Oct 15 19:02:57 nosql-virtualbox opentsdb[21608]:  * Starting TSD...
Oct 15 19:02:57 nosql-virtualbox opentsdb[21608]:    ...done.
Oct 15 19:02:57 nosql-virtualbox systemd[1]: Started opentsdb.service - LSB: Starts OpenTSDB TSD.

Once you have a TSD up and running (after following the installation guide) you can follow the steps below to get some data into OpenTSDB.

Create metrics

Every data you put in time series database must be associated with some metric.

For safety reasons you have to register every metric before you can start storing data points for them. This helps to avoid ingesting unwanted data and catch typos. To register one or more metrics, call the mkmetric command. For example command ./tsdb mkmetric object.position object.speed will create two metrics: object.position and object.speed. See example below:


nosql@nosql-virtualbox:/usr/share/opentsdb/bin$ ./tsdb mkmetric object.position object.speed
23:54:13,390 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
23:54:13,390 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
23:54:13,392 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/etc/opentsdb/logback.xml]
23:54:13,531 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
23:54:13,551 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
23:54:13,558 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
23:54:13,652 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
23:54:13,784 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.read.CyclicBufferAppender]
23:54:13,785 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [CYCLIC]
23:54:13,800 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
23:54:13,802 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
23:54:13,812 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@7a765367 - No compression will be used
23:54:13,836 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
23:54:13,838 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: /var/log/opentsdb/opentsdb.log
23:54:13,838 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [/var/log/opentsdb/opentsdb.log]
23:54:13,840 |-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - openFile(/var/log/opentsdb/opentsdb.log,true) call failed. java.io.FileNotFoundException: /var/log/opentsdb/opentsdb.log (Permission denied)
	at java.io.FileNotFoundException: /var/log/opentsdb/opentsdb.log (Permission denied)
	at 	at java.base/java.io.FileOutputStream.open0(Native Method)
	at 	at java.base/java.io.FileOutputStream.open(FileOutputStream.java:289)
	at 	at java.base/java.io.FileOutputStream.(FileOutputStream.java:230)
	at 	at ch.qos.logback.core.recovery.ResilientFileOutputStream.(ResilientFileOutputStream.java:28)
	at 	at ch.qos.logback.core.FileAppender.openFile(FileAppender.java:149)
	at 	at ch.qos.logback.core.FileAppender.start(FileAppender.java:108)
	at 	at ch.qos.logback.core.rolling.RollingFileAppender.start(RollingFileAppender.java:86)
	at 	at ch.qos.logback.core.joran.action.AppenderAction.end(AppenderAction.java:96)
	at 	at ch.qos.logback.core.joran.spi.Interpreter.callEndAction(Interpreter.java:317)
	at 	at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:196)
	at 	at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:182)
	at 	at ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:62)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:149)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:135)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:99)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:49)
	at 	at ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:75)
	at 	at ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:148)
	at 	at org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:85)
	at 	at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:55)
	at 	at org.slf4j.LoggerFactory.bind(LoggerFactory.java:129)
	at 	at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:108)
	at 	at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:302)
	at 	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:276)
	at 	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:288)
	at 	at net.opentsdb.tools.UidManager.(UidManager.java:52)
23:54:13,840 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
23:54:13,840 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [QUERY_LOG]
23:54:13,843 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@2f943d71 - No compression will be used
23:54:13,844 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
23:54:13,844 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY_LOG] - Active log file name: /var/log/opentsdb/queries.log
23:54:13,844 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[QUERY_LOG] - File property is set to [/var/log/opentsdb/queries.log]
23:54:13,844 |-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[QUERY_LOG] - openFile(/var/log/opentsdb/queries.log,true) call failed. java.io.FileNotFoundException: /var/log/opentsdb/queries.log (Permission denied)
	at java.io.FileNotFoundException: /var/log/opentsdb/queries.log (Permission denied)
	at 	at java.base/java.io.FileOutputStream.open0(Native Method)
	at 	at java.base/java.io.FileOutputStream.open(FileOutputStream.java:289)
	at 	at java.base/java.io.FileOutputStream.(FileOutputStream.java:230)
	at 	at ch.qos.logback.core.recovery.ResilientFileOutputStream.(ResilientFileOutputStream.java:28)
	at 	at ch.qos.logback.core.FileAppender.openFile(FileAppender.java:149)
	at 	at ch.qos.logback.core.FileAppender.start(FileAppender.java:108)
	at 	at ch.qos.logback.core.rolling.RollingFileAppender.start(RollingFileAppender.java:86)
	at 	at ch.qos.logback.core.joran.action.AppenderAction.end(AppenderAction.java:96)
	at 	at ch.qos.logback.core.joran.spi.Interpreter.callEndAction(Interpreter.java:317)
	at 	at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:196)
	at 	at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:182)
	at 	at ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:62)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:149)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:135)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:99)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:49)
	at 	at ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:75)
	at 	at ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:148)
	at 	at org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:85)
	at 	at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:55)
	at 	at org.slf4j.LoggerFactory.bind(LoggerFactory.java:129)
	at 	at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:108)
	at 	at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:302)
	at 	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:276)
	at 	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:288)
	at 	at net.opentsdb.tools.UidManager.(UidManager.java:52)
23:54:13,845 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [QueryLog] to OFF
23:54:13,845 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [QueryLog] to false
23:54:13,845 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [QUERY_LOG] to Logger[QueryLog]
23:54:13,845 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.apache.zookeeper] to INFO
23:54:13,845 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.hbase.async] to INFO
23:54:13,845 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [com.stumbleupon.async] to INFO
23:54:13,845 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [net.opentsdb.tools] to INFO
23:54:13,845 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[net.opentsdb.tools]
23:54:13,845 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
23:54:13,845 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [CYCLIC] to Logger[ROOT]
23:54:13,845 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
23:54:13,845 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
23:54:13,853 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@80503 - Registering current configuration as safe fallback point

metrics object.position: [0, 0, 1]
metrics object.speed: [0, 0, 2]

A lot of messages however at he end you can see that metrics were created successfully. Formally, the response of the mkmetric command is the literal metrics followed by the name of the metric (for example: object.position) and a Java formatted byte array representing the UID assigned or retrieved for each metric (for example: [0, 0, 1]), one per line.

If metric does not exist you will receive a message like:


put: unknown metric: No such name for 'metrics': 'object.position'

Put data to OpenTSDB

You can put some simple data with the help of basic script and then nc (netcat) command.

My script was as follow:


from datetime import datetime
from time import sleep

x = 1
x_old = 0
x_new = 0
t_old = 0
delta = 0.1

for i in range(1, 20):
    now = int(datetime.utcnow().timestamp())
    t = 1
    s = abs(x_old - x_new)
    v = s/t
    x = x_new
    y = 10
    print(f'put object.position {now} {x} object_id=1 coordinate=x')
    print(f'put object.position {now} {y} object_id=1 coordinate=y')
    print(f'put object.speed {now} {v} object_id=1 unit=metric')
    x_old = x_new
    delta = delta + delta * 0.9
    x_new = x_new + delta
    sleep(1.1)


for i in range(20, 40):
    now = int(datetime.utcnow().timestamp())
    t = 1
    s = abs(x_old - x_new)
    v = s/t
    x = x_new
    y = 10
    print(f'put object.position {now} {x} object_id=1 coordinate=x')
    print(f'put object.position {now} {y} object_id=1 coordinate=y')
    print(f'put object.speed {now} {v} object_id=1 unit=metric')
    x_old = x_new
    x_new = x_new + delta
    sleep(1.1)

for i in range(40, 80):
    now = int(datetime.utcnow().timestamp())
    t = 1
    s = abs(x_old - x_new)
    v = s/t
    x = x_new
    y = 10
    print(f'put object.position {now} {x} object_id=1 coordinate=x')
    print(f'put object.position {now} {y} object_id=1 coordinate=y')
    print(f'put object.speed {now} {v} object_id=1 unit=metric')
    x_old = x_new
    delta = delta + delta * (-0.1)
    if delta < 0:
        delta = 0
    x_new = x_new + delta
    sleep(1.1)

You can run it or simply download my data it generate.

If you have data, generated any method you want, you can ingest it into OpenTSDB with the help of nc command:


nosql@nosql-virtualbox:~/Desktop/nosql/opentsdb$ nc 127.0.0.1 4242 < test_data.txt

Examine data in GUI

The easiest way to examine your data present in database is to use GUI. Open web browser and type the url 127.0.0.1:4242 – you will see very basic user interface:

Figure: OpenTSDB GUI main page

In the Metric filed you can star typing the name of the metric you want to see and the GUI will give you suggestion what it might be:

Figure: Filling `Metric` field

Figure: `Metric` field with the name of metric you want to see

After selecting metric and the range of the time, you will see plot of data point related with it:

Figure: Plot of the selected metric

If your metric has various tags you can select which of them you want to examine by typing their names in Tags fields:

Figure: Filling `Tags` fields

Figure: Plot of selected tags of the selected metric

OpenTSDB utilize temporary files generated when a graph is requested via the built-in GUI. These files should be purged periodically to free up space. OpenTSDB doesn’t clean up after itself at this time but there is a script clean_cache.sh that should be run as a cron at least once a day located at tools/ (the same place where create_table.sh script is located):


nosql@nosql-virtualbox:~$ ls -l /usr/share/opentsdb/tools/
total 80
-rwxrwxr-x 1 nosql nosql 12243 Sep  3  2021 check_tsd
-rwxrwxr-x 1 nosql nosql 12576 Sep  3  2021 check_tsd_v2
-rwxrwxr-x 1 nosql nosql   248 Sep  3  2021 clean_cache.sh
-rwxrwxr-x 1 nosql nosql  2356 Sep  3  2021 create_table.sh
drwxrwxr-x 2 nosql nosql  4096 Oct 15 17:24 docker
-rw-rw-r-- 1 nosql nosql   828 Sep  3  2021 opentsdb_restart.py
-rw-rw-r-- 1 nosql nosql  4286 Sep  3  2021 osx_full_stack_install.sh
-rwxrwxr-x 1 nosql nosql 12057 Sep  3  2021 repair-tsd
-rwxrwxr-x 1 nosql nosql  5527 Sep  3  2021 tsdb_list_running_queries.py
-rwxrwxr-x 1 nosql nosql  2464 Sep  3  2021 tsddrain.py
-rw-rw-r-- 1 nosql nosql  1271 Sep  3  2021 upgrade_1to2.sh

What you should do after restart your system

After all, when you complete installation, verify it and reboot your system you should do the following to start OpenTSDB:

Step 1: verify path to Java


nosql@nosql-virtualbox:~$ echo $JAVA_HOME
/usr/lib/jvm/java-1.21.0-openjdk-amd64

Step 2: start HBase


nosql@nosql-virtualbox:~$ /home/nosql/Desktop/nosql/hbase-2.5.10/bin/start-hbase.sh 
running master, logging to /home/nosql/Desktop/nosql/hbase-2.5.10/bin/../logs/hbase-nosql-master-nosql-virtualbox.out

Step 3: verify if HBase is running


nosql@nosql-virtualbox:~$ jps
3124 HMaster
3221 Jps

Step 4: verify if Zookeeper is reachable


nosql@nosql-virtualbox:~$ telnet localhost 2181
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
stats
Zookeeper version: 3.8.4-9316c2a7a97e1666d8f4593f34dd6fc36ecc436c, built on 2024-02-12 22:16 UTC
Clients:
 /127.0.0.1:51728[0](queued=0,recved=1,sent=0)
 /127.0.0.1:55830[1](queued=0,recved=112,sent=125)
 /127.0.0.1:45934[1](queued=0,recved=50,sent=60)

Latency min/avg/max: 0/5.1905/66
Received: 212
Sent: 234
Connections: 3
Outstanding: 0
Zxid: 0xa6
Mode: standalone
Node count: 35
Connection closed by foreign host.

Step 5: start HBase shell and verify if all OpenTSDB's table exist


nosql@nosql-virtualbox:~$ /home/nosql/Desktop/nosql/hbase-2.5.10/bin/hbase shell
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.5.10, ra3af60980c61fb4be31e0dcd89880f304d01098a, Thu Jul 18 22:45:17 PDT 2024
Took 0.0193 seconds                                                                             
hbase:001:0> list
TABLE                                                                                           
tsdb                                                                                            
tsdb-meta                                                                                       
tsdb-tree                                                                                       
tsdb-uid                                                                                        
4 row(s)
Took 1.5300 seconds                                                                             
=> ["tsdb", "tsdb-meta", "tsdb-tree", "tsdb-uid"]
hbase:002:0> exit

Step 6: start OpenTSDB


nosql@nosql-virtualbox:~$ sudo service opentsdb start
[sudo] password for nosql:

Step 7: verify if OpenTSDB is running


nosql@nosql-virtualbox:~$ sudo systemctl status opentsdb
● opentsdb.service - LSB: Starts OpenTSDB TSD
     Loaded: loaded (/etc/init.d/opentsdb; generated)
     Active: active (running) since Wed 2024-12-04 23:01:20 CET; 8min ago
       Docs: man:systemd-sysv-generator(8)
      Tasks: 22 (limit: 4616)
     Memory: 109.2M (peak: 111.5M)
        CPU: 4.523s
     CGroup: /system.slice/opentsdb.service
             └─1223 java -enableassertions -enablesystemassertions -classpath "/usr/share/opent>

Dec 04 23:01:20 nosql-virtualbox systemd[1]: Starting opentsdb.service - LSB: Starts OpenTSDB T>
Dec 04 23:01:20 nosql-virtualbox opentsdb[1207]:  * Starting TSD...
Dec 04 23:01:20 nosql-virtualbox opentsdb[1207]:    ...done.
Dec 04 23:01:20 nosql-virtualbox systemd[1]: Started opentsdb.service - LSB: Starts OpenTSDB TS>

Step 8: open web browser, type the url 127.0.0.1:4242 and verify if you will see very basic user interface.

To stop OpenTSDB gracefully do:

Step 1: stop OpenTSDB


nosql@nosql-virtualbox:~$ sudo service opentsdb stop

Step 2: verify if OpenTSDB is not running


nosql@nosql-virtualbox:~$ sudo systemctl status opentsdb
○ opentsdb.service - LSB: Starts OpenTSDB TSD
     Loaded: loaded (/etc/init.d/opentsdb; generated)
     Active: inactive (dead) since Wed 2024-12-04 23:15:04 CET; 13s ago
   Duration: 13min 43.980s
       Docs: man:systemd-sysv-generator(8)
    Process: 5990 ExecStop=/etc/init.d/opentsdb stop (code=exited, status=0/SUCCESS)
        CPU: 8.155s

Dec 04 23:01:20 nosql-virtualbox systemd[1]: Starting opentsdb.service - LSB: Starts OpenTSDB T>
Dec 04 23:01:20 nosql-virtualbox opentsdb[1207]:  * Starting TSD...
Dec 04 23:01:20 nosql-virtualbox opentsdb[1207]:    ...done.
Dec 04 23:01:20 nosql-virtualbox systemd[1]: Started opentsdb.service - LSB: Starts OpenTSDB TS>
Dec 04 23:15:04 nosql-virtualbox systemd[1]: Stopping opentsdb.service - LSB: Starts OpenTSDB T>
Dec 04 23:15:04 nosql-virtualbox opentsdb[5990]:  * Stopping TSD...
Dec 04 23:15:04 nosql-virtualbox opentsdb[5990]:    ...done.
Dec 04 23:15:04 nosql-virtualbox systemd[1]: opentsdb.service: Deactivated successfully.
Dec 04 23:15:04 nosql-virtualbox systemd[1]: Stopped opentsdb.service - LSB: Starts OpenTSDB TS>
Dec 04 23:15:04 nosql-virtualbox systemd[1]: opentsdb.service: Consumed 8.155s CPU time, 156.7M>

Step 3: verify if HBase is running


nosql@nosql-virtualbox:~$ jps
3124 HMaster
6126 Jps

Step 4: stop HBase


nosql@nosql-virtualbox:~$ /home/nosql/Desktop/nosql/hbase-2.5.10/bin/stop-hbase.sh 
stopping hbase............

Step 5: verify if HBase is not running


nosql@nosql-virtualbox:~$ jps
6441 Jps