Open in app

Sign in

Write

Sign in

Home Lab Guides: Proxmox 6 — Network and System Monitoring with Grafana, InfluxDB and Telegraf

Elijah Liedtke
6 min readJan 12, 2021

--

https://www.turbogeek.co.uk/wp-content/uploads/2020/09/1200px-Grafana_logo.svg_.png

This guide will take you through the process of creating an InfluxDB database, installing Telegraf to deliver system telemetry, installing Grafana and a system dashboard.

Home Lab Guides: Proxmox 6

Part 1: Basic Setup and Installation
Part 2: PCI(e) Passthrough with NVIDIA
Part 3: IPFire Firewall
Part 4: Pi-hole
Part 5: Network and System Monitoring with Grafana, InfluxDB and Telegraf
Part 6: Automation with Ansible

Table of Contents

1.0 Preparation
2.0 InfluxDB
  2.1 Install InfluxDB
  2.2 Create an InfluxDB Database
3.0 Telegraf
  3.1 Install Telegraf
  3.2 Configure Telegraf
  3.3 Verify telemetry is being sent to InfluxDB and stored in the database
4.0 Grafana
  4.1 Install Grafana
  4.2 Log in to Grafana WebUI
  4.3 Grafana Dashboards
    4.3.1 Connect InfluxDB
    4.3.2 Import a Dashboard
5.0 Connect Other Systems
  5.1 Linux
  5.2 Windows
    5.2.1 Download Telegraf
    5.2.2 Configure Telegraf
    5.2.3 Run Telegraf as a service
6.0 References
7.0 Author's Notes

1.0 Preparation

If you haven’t created a container before, read Home Lab Guides: Proxmox 6 — Pi-hole sections 1.1 Container Templates and 1.2 Create a Container to create a basic Ubuntu 20.04 container.

  1. Create a basic Ubuntu 20.04 container with a static IP.
  2. Start the container and open a Console window.
  3. Log in using the user root and the password you set during the container creation.
  4. Run apt-get update && apt-get upgrade -y to ensure the container is up-to-date.

2.0 InfluxDB

This section will guide you through the installation of InfluxDB and the creation of a database to store delivered system telemetry.

2.1 Install InfluxDB

1. Install the necessary dependencies.

apt-get install gnupg2 software-properties-common -y

2. Import the GPG key.

wget -qO- https://repos.influxdata.com/influxdb.key | tee -a /usr/share/keyrings/influxdb.key

3. Add the signed repository entry.

echo "deb [signed-by=/usr/share/keyrings/influxdb.key] https://repos.influxdata.com/ubuntu focal stable" | tee /etc/apt/sources.list.d/influxdb.list

4. Install and enable InfluxDB.

apt-get update
apt-get install influxdb -y
systemctl enable --now influxdb
systemctl start influxdb

5. Confirm InfluxDB is active.

systemctl status influxdb

2.2 Create an InfluxDB Database

1. Connect to InfluxDB.

influx

2. From within Influx, create a database and user. Feel free to use whatever names and passwords you’d like.

create database grafanadb
create user grafanadbadmin with password 'password'
grant all on grafanadb to grafanadbadmin

3. Disconnect from Influx.

exit

2.3 Bind InfluxDB to Grafana IP

  1. Edit /etc/influxdb/influxdb.conf and add the bind-address
bind-address = "your-grafana-server-ip:8088"

2. Restart InfluxDB

systemctl restart influxdb

3.0 Telegraf

This section will guide you through the installation of Telegraf and configuring a telegraf.conf template.

1. Install the necessary dependencies.

apt-get install gnupg2 software-properties-common -y

2. Import the GPG key.

wget -qO- https://repos.influxdata.com/influxdb.key | tee -a /usr/share/keyrings/influxdb.key

3. Add the signed repository entry.

echo "deb [signed-by=/usr/share/keyrings/influxdb.key] https://repos.influxdata.com/ubuntu focal stable" | tee /etc/apt/sources.list.d/influxdb.list

3.1 Install Telegraf

Because we added the InfluxDB repository, we can go ahead and install Telegraf.

1. Install and enable Telegraf.

apt-get update
apt-get install telegraf -y
systemctl enable --now telegraf
systemctl start telegraf

2. Confirm Telegraf is active.

systemctl status telegraf

3.2 Configure Telegraf

1. Create a basic telegraf.conf file.

telegraf config -input-filter cpu:disk:mem:processes:swap:system -output-filter influxdb > /etc/telegraf/telegraf.conf

2. Edit /etc/telegraf/telegraf.conf for local telemetry. To ensure all the information for the dashboard is being collected, we’ll need to add a few entries. Normally, we’d uncomment these as they already exist, but you can just add them before [[inputs.cpu]] to get the telemetry.

[[outputs.influxdb]]
  urls = ["http://your-grafana-server-ip:8086"]
  database = "grafanadb"
  username = "grafanadbadmin"
  password = "password"[[inputs.conntrack]]
[[inputs.filestat]]
[[inputs.internal]]
[[inputs.interrupts]]
[[inputs.linux_sysctl_fs]]
[[inputs.net]]
[[inputs.net_response]]
  protocol = "tcp"
[[inputs.netstat]]
[[inputs.nstat]]
[[inputs.procstat]]
  pattern = "."
  prefix = "pgrep_serviceprocess"

3. Restart Telegraf to apply the changes.

systemctl restart telegraf

4. Test telegraf.conf and confirm local telemetry is being produced.

telegraf --config /etc/telegraf/telegraf.conf --test

3.3 Verify telemetry is being sent to InfluxDB and stored in the database

influx
use grafanadb
select * from system
exit

4.0 Grafana

Grafana is an opensource proect that allows users to customize and visualize system metrics. It has support for a wide range of time-series databases such as InfluxDB, Prometheus and others. It also has the capability to monitor systems in the Cloud as well as SQL databases such as MySQL and PostgreSQL.

4.1 Install Grafana

1. Import the GPG key.

wget -qO- https://packages.grafana.com/gpg.key | tee -a /usr/share/keyrings/grafana.key

2. Add the signed repository entry.

echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://packages.grafana.com/oss/deb stable main" | tee /etc/apt/sources.list.d/grafana.list

3. Install and enable Grafana.

apt-get update
apt-get install grafana -y
systemctl daemon-reload
systemctl enable --now grafana-server

4. Verify Grafana is active.

systemctl status grafana-server

5. Create a new admin password.

grafana-cli admin reset-admin-password 'mynewpassword'

4.2 Log in to WebUI

1. From a different device, browse to http://grafana-server-ip:3000/ to access the WebUI.

2. Enter the username admin and password you just created.

4.3 Dashboards

Grafana Dashboards are a way of customizing the look and data being collected. It allows for the creation of your own queries and displaying information that is relevant for each device.

4.3.1 Connect InfluxDB data source

1. Click the Configuration cog and choose Data Sources.

2. Click Add data source.

3. Click InfluxDB and change the following:

HTTP
  URL: http://your-grafana-server-ip:8086InfluxDB Details
  Database: grafanadb
  User: grafanadbadmin
  Password: password
  HTTP Method: GET    # if only a few systems/queries
               POST   # if a LOT of systems/queries

4. Click Save & Test to confirm Grafana can connect to the Influx database.

4.3.2 Import a dashboard

1. Click + and choose Import.

2. Enter the ID 11912and press Enter.

Note: There are thousands of templates that exist on Grafana’s repository https://grafana.com/grafana/dashboards

3. Select your InfluxDB data source and then Import.

If things have been setup correctly, telemetry should already exist and the panels should all be displaying information for your Grafana server.

This dashboard has a couple of titles in Spanish, but other than that, the queries should all work fine.

5.0 Connect Other Systems

System metrics gathered by Telegraf and stored in InfluxDB don’t have to be just the system it’s on. You can have every system in your network sending telemetry and metrics to InfluxDB for monitoring by Grafana.

5.1 Linux

For each machine, repeat section 3.0 Telegraf or SCP the Grafana telegraf.conf file to each machine and then restart Telegraf on that machine.

Sometimes root isn’t allowed to SCP, so, if that happens, you’ll need to create a new sudo user.

  1. SCP telegraf.conf to target machines
scp /etc/telegraf/telegraf.conf user@target-machine-ip:/etc/telegraf

5.2 Windows

This section deals with installing and enabling Telegraf ONLY and does not cover changing delivered metrics.

Because Windows stores its data under different measurements than Linux, the dashboard we used 11912 doesn’t display any information regarding Windows systems. Custom queries/panels can be created or a Windows-specific Grafana dashboard can be imported.

5.2.1 Download Telegraf

  1. Download the latest version for Windows https://github.com/influxdata/telegraf/releases
  2. Make directory C:\Program Files\telegraf
  3. Extract Telegraf zip files to C:\Program Files\telegraf

5.2.2 Configure Telegraf

  1. Search for Notepad then right-click and choose Run as Administrator.
  2. Click File then open C:\Program Files\telegraf\telegraf.conf
  3. Make the following changes:
[[outputs.influxdb]]
  urls = ["http://your-grafana-server-ip:8086"]
  database = "grafanadb"
  username = "grafanadbadmin"
  password = "password"

5.2.3 Run Telegraf as a service

  1. Search for PowerShell then right-click and choose Run as Administrator.
  2. Navigate to telegraf.conf
cd "C:\Program Files\telegraf"

3. Run the following command to enable Telegraf to run as a service and start on boot.

.\telegraf --service install --config-directory 'C:\Program Files\telegraf\conf'

4. Search for services.msc then hit Enter to display the Windows Services list.

5. Scroll down the Services until you find Telegraf Data Collector Service. It should have an Automatic startup type so it starts up at boot.

6. Right-click on Telegraf Data Collector Service and select Start to start sending telemetry to the InfluxDB database on the Grafana server.

6.0 References

Grafana Dashboards — discover and share dashboards for Grafana; 11–01–2021; https://grafana.com/grafana/dashboards

Using Telegraf on Windows | Blog | InfluxData; 11–01–2021; https://www.influxdata.com/blog/using-telegraf-on-windows/

7.0 Author’s Notes

  • Thanks for reading this article! If you have any questions, suggestions or comments, feel free to leave a comment.
Proxmox
Homelab
Grafana
Telegraf
Influxdb

--

--

Elijah Liedtke
Elijah Liedtke

Written by Elijah Liedtke

23 Followers
1 Following

No responses yet

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams