Zabbix Proxy: Install on Ubuntu 20.04 / 18.04

Zabbix proxy is a service that can collect performance and availability data from the end devices on behalf of the Zabbix server.

You can use Zabbix proxy to monitor remote locations behind the firewall or locations that have unreliable communication and to offload the Zabbix server in large environments.

Proxy architecture example

We will install the latest version 5.0 / 5.2 of the Zabbix proxy on Ubuntu (20.04, 18.04 and 16.04 are supported) and make the key optimizations so that your proxy performs better.

Step 1: Install Zabbix Proxy on Ubuntu

Before you install proxy, keep in mind that your proxy version must match the Zabbix server version!

You can check Zabbix server version with the command:

zabbix_server -V

Setup Zabbix 5 .deb package and install proxy server:

Zabbix 5.0 LTS version (supported until May 31, 2025)

sudo wget https://repo.zabbix.com/zabbix/5.0/ubuntu/pool/main/z/zabbix-release/zabbix-release_5.0-1+$(lsb_release -sc)_all.deb

sudo dpkg -i zabbix-release_5.0-1+$(lsb_release -sc)_all.deb

sudo apt update

sudo apt -y install zabbix-proxy-mysql

OR

Zabbix 5.2 standard version (supported until May 31, 2021)

wget https://repo.zabbix.com/zabbix/5.2/ubuntu/pool/main/z/zabbix-release/zabbix-release_5.2-1+ubuntu$(lsb_release -rs)_all.deb

sudo dpkg -i zabbix-release_5.2-1+ubuntu$(lsb_release -rs)_all.deb

sudo apt update

sudo apt -y install zabbix-proxy-mysql

You can find more information about Zabbix’s life cycle and release policies on the official website.

Step 2: Configure database

In this installation, I will use password rootDBpass as root database password and zabbixDBpass as Zabbix database password. Consider changing your password for security reasons.

a. Install MariaDB

sudo apt -y install mariadb-common mariadb-server mariadb-client

If you receive a error because of unmet dependencies, then install it like this “sudo apt -y install mariadb-common mariadb-server-10.3 mariadb-client-10.3“.

Once the installation is complete, start the MariaDB service and enable it to start on boot using the following commands:

sudo systemctl start mariadb
sudo systemctl enable mariadb

b. Reset root password for database

Secure MySQL/MariaDB by changing the default password for MySQL root:

sudo mysql_secure_installation

Enter current password for root (enter for none): Press the Enter

Set root password? [Y/n]: Y

New password: <Enter root DB password>

Re-enter new password: <Repeat root DB password>

Remove anonymous users? [Y/n]: Y

Disallow root login remotely? [Y/n]: Y

Remove test database and access to it? [Y/n]:  Y

Reload privilege tables now? [Y/n]:  Y

c. Create database

sudo mysql -uroot -p'rootDBpass'
create database zabbix_proxy character set utf8 collate utf8_bin;
grant all privileges on zabbix_proxy.* to zabbix@localhost identified by 'zabbixDBpass';
quit;

d. Import initial database schema

Import database shema for proxy server (could last up to 5 minutes):

zcat /usr/share/doc/zabbix-proxy-mysql*/schema.sql.gz | mysql -uzabbix -p'zabbixDBpass' zabbix_proxy

Step 3: Zabbix proxy configuration

In my setup, I will name proxy as Zabbix proxy 01 and configure it to communicate with the Zabbix that has IP address 10.7.44.235 – please change those accordingly to match your environment.

Open file zabbix_proxy.conf with the command:

sudo nano /etc/zabbix/zabbix_proxy.conf

Add database password and configuration frequency in this format anywhere in the file:

DBPassword=zabbixDBpass
ConfigFrequency=100

“ConfigFrequency” parameter defines how often proxy retrieves configuration data from the Zabbix server in seconds. You can read more about proxy configuration parameters in the official documentation.

Don’t close the file yet! Now, you need to change the current “Server“, “Hostname“, “DBName“, and “DB user” parameter values to something like this:

Server=10.7.44.235

Hostname=Zabbix proxy 01

DBName=zabbix_proxy

DBUser=zabbix

Save and exit file (ctrl+x, followed by y and enter).

Step 4: Start and enable proxy service

Restart proxy service and set it to boot on startup:

sudo systemctl restart zabbix-proxy
sudo systemctl enable zabbix-proxy

Step 5: Register proxy server in the Zabbix frontend

Now that your proxy server is running, you need to register it in the Zabbix frontend using “Proxies” option under the “Administration” section. Click on the “Create proxy” button on the right, then set “Proxy name” to Zabbix proxy 01and “Proxy mode” to “Active“.

Note that for a proxy to work in active mode “Proxy name” must be the same as the “hostname” parameter in “zabbix_proxy.conf” file.

Picture showing how to register an active proxy server in the Zabbix frontend

Picture showing how to register an active proxy server in the Zabbix frontend.

Immediately after adding a proxy to the frontend, you will be informed that the proxy has never been seen.

Status of the proxy server  that has never been seen

However, if you wait for 100 seconds “Last seen (age)” status will display seconds instead “Never” which means that proxy is communicating with the Zabbix server.

Status of the proxy server  that has been seen recently

Why 100 seconds? Remember what we have put under the “ConfigFrequency” parameter in “zabbix_proxy.conf” file? Whatever you put there that’s how long will it take for the proxy to contact the Zabbix server to retrieve configuration data. This rule only applies if the proxy is in active mode.

Note that “Last seen (age)” status should never go above 6 seconds if proxy-server communication is working.

Active proxy mod is recommended because it can work behind a firewall but you can check out the section “Understanding Zabbix proxy Active vs Passive mod” If you need to set proxy to work in a passive mode.

Step 6: Configure hosts in Zabbix to be monitored by a proxy

Zabbix proxy is running and communicating with the Zabbix server, but it is not gathering any data because we didn’t do the last crucial step! We need to configure hosts in Zabbix to be monitored by the proxy server.

Let’s do that together, go to the “Host” option under the “Configuration” section and select hosts that need to monitored by the proxy server.

Configure hosts in Zabbix to be monitored by a proxy server - Step 1

Configure hosts in Zabbix to be monitored by a proxy server – Step 1

After you have select hosts click on the “Mass update” button, then under the “Host” options select our newly added proxy in the “Monitored by proxy” dropdown and finally click the “Update” button just as shown in the image below.

Configure hosts in Zabbix to be monitored by a proxy server - Step 2

Configure hosts in Zabbix to be monitored by a proxy server – Step 2

Step 7: Configuring PSK encryption on the proxy (optional)

Zabbix supports encrypted communications between the Zabbix server and proxy using Transport Layer Security (TLS) protocol v.1.2. You can use certificate-based and pre-shared key-based encryption (PSK), but in this tutorial we will configure the latter.

a. Generate PSK key on proxy server

Generate 256-bit (32 bytes) PSK key with openssl command:

openssl rand -hex 32

The command will generate a key soemthing like this: 382eb0250c365cc207cc4980cdb1bb1814c19d882b7fddcda00281311a8fb955. Copy this code somewhere as you will need it in the next step.

Create and open file “zabbix_proxy.psk” with command:

sudo nano /etc/zabbix/zabbix_proxy.psk

Copy & paste that newly generated key into it.

Save and exit the file (ctrl+x, followed by y and enter) and set the correct file permission:

sudo chown zabbix:zabbix /etc/zabbix/zabbix_proxy.psk
sudo chmod 644 /etc/zabbix/zabbix_proxy.psk

b. Configure Proxy to support PSK encryption

Open zabbix_proxy.conf file with command:

sudo nano /etc/zabbix/zabbix_proxy.conf

Add this configuration anywhere in file:

TLSConnect=psk
TLSAccept=psk
TLSPSKFile=/etc/zabbix/zabbix_proxy.psk
TLSPSKIdentity=ZBX-PSK-01

Save and exit file (ctrl+x, followed by y and enter). Keep in mind that “TLSPSKIdentity” can be anything, so for security reasons set something else – don’t use mine!

Don’t forget to restart proxy after changing the configuration file:

sudo systemctl restart zabbix-proxy

c. Enable PSK encryption on the proxy in Zabbix frontend

Communication between proxy and the Zabbix server is not yet encrypted because we have enabled PSK encryption on the proxy side but not on the server-side.

We need to enable encryption on the server side so go to web frontend and select your proxy on the “Proxies” option under the “Administration” section. Next, on the “Encryption” tab set PSK under “Connections from proxy” option and copy and paste “PSK identity” and “PSK” (key) that you have configured on the proxy serve. If you ever lose your generated PSK key check it with command “cat /etc/zabbix/zabbix_proxy.psk” on the proxy server.

When you are done with configuration click the “Update” button just as shown in the image below.

Configuring PSK encryption on a proxy in the Zabbix frontend

Configuring PSK encryption on a proxy in the Zabbix frontend

If you have done everything right you should see something like this:

Checking proxy encryption status in the web frontend

Checking proxy encryption status in the web frontend

Step 8: Optimizing Zabbix Proxy (optional)

Don’t bother with this optimization if you are monitoring a small number of devices, but if you are planning to monitor a large number of devices then continue with this step.

Open zabbix_proxy.conf file with command: “sudo nano /etc/zabbix/zabbix_proxy.conf” and add this configuration anywhere in file:

StartPollers=100

StartPollersUnreachable=50

StartPingers=50

StartTrappers=10

StartDiscoverers=15

StartHTTPPollers=5

CacheSize=128M

HistoryCacheSize=64M

HistoryIndexCacheSize=32M

Save and exit file (ctrl+x, followed by y and enter).

This is not a perfect configuration, keep in mind that you can optimize it even more. Let’s say if you don’t use ICMP checks then set the “StartPingers” parameter to 1 or if you don’t use active agents then set “StartTrappers” to 1 and so on. You can find out more about the supported parameters in the official documentation.

If you try to start the proxy server you will receive an error “[Z3001] connection to database ‘zabbix_proxy’ failed: [1040] Too many connections” in the log “/var/log/zabbix/zabbix_proxy.log” because we are using more proxy processes than MySQL can handle. We need to increase the maximum permitted number of simultaneous client connections and optimize MySQL – so move to the next step.

Step 9: Optimizing MySQL database (optional)

a. Create custom MySQL configuration file

Create file “10_my_tweaks.cnf” with “sudo nano /etc/mysql/mariadb.conf.d/10_my_tweaks.cnf” and paste this configuration:

[mysqld]

max_connections = 280

innodb_buffer_pool_size = 1G

innodb-log-file-size = 128M

innodb-log-buffer-size = 128M

innodb-file-per-table = 1

innodb_buffer_pool_instances = 8

innodb_old_blocks_time = 1000

innodb_stats_on_metadata = off

innodb-flush-method = O_DIRECT

innodb-log-files-in-group = 2

innodb-flush-log-at-trx-commit = 2

tmp-table-size = 96M

max-heap-table-size = 96M

open_files_limit = 65535

max_connect_errors = 1000000

connect_timeout = 60

wait_timeout = 28800

Save and exit the file (ctrl+x, followed by y and enter) and set the correct file permission:

sudo chown mysql:mysql /etc/mysql/mariadb.conf.d/10_my_tweaks.cnf

sudo chmod 644 /etc/mysql/mariadb.conf.d/10_my_tweaks.cnf

Two things to remember!

Configuration parameter max_connections must be larger than the total number of all the proxy processes plus 50. You can use the command below to automatically check the number of Zabbix processes and add 50 to that number:

root@ubuntu:~ $ egrep “^Start.+=[0-9]” /etc/zabbix/zabbix_proxy.conf | awk -F “=” ‘{s+=$2} END {print s+50}’

280

The second most important parameter is innodb_buffer_pool_size, which determines how much memory can MySQL get for caching InnoDB tables and index data. You should set that parameter to 70% of system memory if only database is installed on server.

However, in this case, we are sharing a server with the proxy server so you should set innodb_buffer_pool_size to 50% of total system memory. That would be 1G because my Ubuntu server has 2G RAM.

I didn’t have any problems with memory, but if your proxy crashes because of lack of memory, reduce “innodb_buffer_pool_size” and restart MySQL server.

b. Restart Proxy and MySQL service

Stop and start the services in the same order as below:

sudo systemctl stop zabbix-proxy

sudo systemctl stop mysql

sudo systemctl start mysql

sudo systemctl start zabbix-proxy

Step 10: How to manage Proxy and MySQL service

Sometimes you will need to check or restart proxy server or MySQL service – use commands below to do that.

Proxy server

sudo systemctl <status/restart/start/stop> zabbix-proxy

MySQL Server

sudo systemctl <status/restart/start/stop> mysql

Step 11: Understanding Zabbix proxy Active vs Passive mod

When you use a proxy in active mode, it will connect to the Zabbix server to retrieve configuration and send data. This is a great feature that enables active proxy server to work behind a firewall.

And if you use a proxy in passive mode Zabbix server will initiate a connection to the proxy to send configuration and retrieve data from the proxy. Diagram

Description automatically generated

Picture showing how active proxy works compared to a passive proxy (source: www.zabbix.com)

Before, in this tutorial, we have configured proxy server to work in active mode. Let me show you how can you configure the proxy to work in passive mode.

Open file zabbix_proxy.conf with the command: “sudo nano /etc/zabbix/zabbix_proxy.conf” and turn on passive mode and define Zabbix server IP address by adding this anywhere in the file (change “Server” parameter to to match your Zabbix server):

ProxyMode=1

Server=10.7.44.235

Save and exit the file (ctrl+x, followed by y and enter). Reboot proxy server with the command “sudo systemctl restart zabbix-proxy“

Now, you need to add the proxy in the Zabbix frontend using the “Proxies” option under the “Administration” section. Click on the “Create proxy” button on the right, then set “Proxy name” to anything you want and “Proxy mode” to “Passive“. Define IP address of the proxy server under the “Interfaces” and click add button if you are done with the configuration.

Picture showing how to register a passive proxy server in  the Zabbix frontend

Picture showing how to register a passive proxy server in the Zabbix frontend

Step 12: Learn about common Zabbix Proxy errors

No data in items because proxy server time differs from Zabbix server time

Starting from version 4.0 Zabbix server will no longer adjust value timestamps in cases when proxy time differs from Zabbix server time. This problem is easy to detect because server-proxy communication will work but there will be no data in the items.

Adjust proxy and the Zabbix server to use the same time server (NTP) or set the time manually.

Communication between Proxy and Server is not working

Zabbix proxy is using port 10051 to communicate with the Zabbix server so make sure that you permit traffic coming from port 10051 in your network.

Note that Zabbix is using compression for communication with Proxies. Compression improves performance because it drastically reduces network traffic.

However, this may cause some problems in environments that have security devices (IDS/IPS or NGFW) installed as they can drop network packets if they detect compression in the network stream. To avoid that problem, make sure the application definitions on the security devices are up-to-date, and if you have any problems, contact your device vendor.

Checking logs on proxy server

If you experience a problem with the proxy you can check last 500 lines of the log with the command:

tail -500 /var/log/zabbix/zabbix_proxy.log


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *