Prometheus

Prometheus integration guide

Prometheus is an open-source system monitoring and alerting tool.

You can use Prometheus to monitor standard and custom metrics to get performance data from a variety of sources like machines, Docker, Azure, AWS, and more.

In this guide, we'll show you how to visualize time series data from Prometheus in Clarify. We'll use metrics which is available in a standard Prometheus instance.

Timeline in Clarify with data from PrometheusTimeline in Clarify with data from Prometheus

Timeline in Clarify with data from Prometheus

You can use the Prometheus interface to try queries and find time-series data that you can send to Clarify.

Screenshot of Prometheus with cpu_fraction default metricScreenshot of Prometheus with cpu_fraction default metric

Screenshot of Prometheus with cpu_fraction default metric

Getting started

Use cases

You can use Prometheus to monitor metrics on a variety of hosts and cloud providers. You can e.g. install node exporter on a machine and monitor metrics like CPU usage, memory, disk utilization, filesystem fullness, and network bandwidth. You can also set up Prometheus to monitor docker container metrics and much more.
Check out exporters and integrations for a full list of exporters and integrations.

Prometheus setup

If you are not already running and using Prometheus there are several ways to quickly get started. Check out the installation guides for guides on how to install Prometheus using pre-compiled binaries, source or Docker images.

1. Node-RED setup

At this point you need to have an accessible instance of Prometheus up and running and access to an instance of Node-RED.

If you are new to Node-RED or haven't checked out our Node-RED guides yet, we recommend that you check out Introduction to Node-RED before continuing.

2. Prepare query and request

In Node-RED drag the following nodes and connect them in your flow:

  • Inject node
  • Function node (and name it Prepare query and e.g. use the function code further down)
  • HTTP request node (and configure according to
  • Debug node

Remember to set the output in the Debug node to complete msg object.

Node-RED flow to send a request to Prometheus and debug the responseNode-RED flow to send a request to Prometheus and debug the response

Node-RED flow to send a request to Prometheus and debug the response

Example of Prepare query node function.
In this guide, we'll use go_memstats_gc_cpu_fraction which is the fraction of Prometheus's available CPU time used by the GC (garbage collection) since the program started.

msg.payload = {
    "query": "go_memstats_gc_cpu_fraction[90m:1m]",
};

return msg;

Depending on your setup Prometheus is most likely running on port 9090. Setup the HTTP request node according to the screenshot below.

Prometheus HTTP request node setupPrometheus HTTP request node setup

Prometheus HTTP request node setup

Test query and request
If you click on the left button on the Inject node you can try to request data from Prometheus.

If the flow is working as expected you'll find something similar to the JSON below In the debug window in Node-RED.

payload : {
    "status": "success",
    "data": {
        "resultType": "matrix",
        "result": [
            {
                "metric": {
                    "__name__": "go_memstats_gc_cpu_fraction",
                    "instance": "host.docker.internal:9100",
                    "job": "node"
                },
                "values": [
                    [
                        1621936680,
                        "0"
                    ],
                    [
                        1621936740,
                        "0.0000013283317346844382"
                    ]
                ]
            }
        ]
    }
}

3. Prepare data

Now that we can request data from Prometheus we'll need to format the data so that we can send it to Clarify.

Add another Function node to the flow and connect it between the HTTP request and Debug nodes as you can see in the screenshot below. You can name the node e.g. Prepare data.

Node-RED Prometheus to Clarify integration flowNode-RED Prometheus to Clarify integration flow

Node-RED Prometheus to Clarify integration flow

Example of Prepare data node function code.

delete msg.headers;
delete msg.responseUrl;
delete msg.responseCookies;
delete msg.redirectList;

const DateTime = global.get('luxon').DateTime;
let records = msg.payload.data.result[0].values;

if (records.length > 0) {
    
    let dates = [];
    let values = [];
    
    records.forEach((item)=> {
        if (item[0] && item[1]) {
            let date = DateTime.fromSeconds(item[0], { zone: "Europe/Oslo" });
            dates.push(date);
            values.push(parseFloat(item[1]));
        }
    });
    msg.topic = "prometheus-cpu-fraction";
    msg.signal = {
      name: "Prometheus CPU Fraction",
      labels: {
        "data-source": ["Prometheus"],
      }
    };
    msg.payload = {
      "times": dates,
      "values": values,
    };

    return msg;
}

return null;

We use the NPM package Luxon in the code above to work with date formats. You'll need to install this package or use a different package to handle dates. You can also check out the Node-RED guide.

Remember that you still can test the output by manually clicking on the Inject node to see the request that you are going to send to Clarify in the next step.

4. Send data to Clarify

In this step, we'll send the actual data to Clarify. As a preparation, you need to follow the guides Installing the Clarify nodes and Using the Clarify Insert node to make sure your workflow is able to correctly send data to Clarify.

When you have installed the Clarify nodes and setup credentials you can connect the Prepare data node to the Clarify node. We also recommend that you connect two Debug nodes to the Clarify node. One for Errors and the other for other debug messages.

Complete flow for sending time series data from Prometheus to ClarifyComplete flow for sending time series data from Prometheus to Clarify

Complete flow for sending time series data from Prometheus to Clarify

To send the result to Clarify you can trigger the Inject node manually or set it up to trigger automatically as described below. We recommend you trigger it manually, to begin with, to verify that everything works as expected.

📘

Automatically trigger the flow

The Inject node supports automatic triggering at certain intervalls. You can achieve this by double-clicking on the Inject node and configuring the Repeat setting. For example by setting "repeat interval" with "1 hour" the workflow above will trigger and update data every hour.

5. Visualization and collaboration in Clarify

When you have triggered the flow that now contains the Clarify input node the data becomes available in Clarify.

To visualize the data we need to publish the signals as Items and create a new timeline.

If you follow this guide, you'll have Items similar to the ones below with fresh data.

Prometheus CPU fraction Item available in ClarifyPrometheus CPU fraction Item available in Clarify

Prometheus CPU fraction Item available in Clarify

Drag the Prometheus Item to the timeline and personalize the timeline with labels and colors that suit you and your company.

Timeline in Clarify with time series data from PrometheusTimeline in Clarify with time series data from Prometheus

Timeline in Clarify with time series data from Prometheus

Congratulations

You now have Prometheus time-series data available in Clarify! Add more Items with your data and
invite the rest of your team members to Clarify and create timelines, comment on data and collaborate as fresh new analytics data appear in Clarify.


Did this page help you?