Combining multiple datapoints for a metric

[nabheet]

Is there a way to add two datapoints in a metric? Google and ChatGPT haven’t really provided anything helpful.

In particular, I have a Prometheus receiver scraping the output here https://github.com/mdawar/rq-exporter. I need to add values for two statuses for the same queue name attributes on a datapoint.

Thank you!

[CaptainPickles89]

Hi Nabheet, my understanding is a Prometheus metric can only have one value, I don't think it's possible to add another.

Not sure what your use case is but I think you'd be better off sending both values separately and then combining them when you query from your data store

[nabheet]

I was hoping to add two different datapoints of a metric to create a new metric which would be the sum of the original two? Is that possible? Or even a new datapoint on the same metric which is a sum of the first two?

[CaptainPickles89]

Again is that not something you can do at the visualisation end? So for example in Grafana you can add transformations to metrics to combine them.

If not and you have a use case for needing to combine them then you could potentially do that within the metrics transform processor I think, it has the ability to aggregate values based on your match type, so if you have 2 metrics with the same name, you can match and aggregate them into a new value

[nabheet]

The limitation is in GCP Alerts where their monitoring query does not allow the same field to be specified multiple times.

Lets say you have a metric with these datapoints:

1. queue_name=do_task_1, status=good, value=1
2. queue_name=do_task_2, status=bad, value=1
3. queue_name=do_task_1, status=meh, value=5

in GCP, I cannot say:

WHERE status = 'good' OR status = 'meh'

because it is repeating a field in the query. So I was hoping that maybe in the collector I could combine (1) and (3) datapoints into a new one with a value of 6.

I realize I can also control this at the generation of these metrics but this is more work because we don't control the software the is generating these metrics. So I am evaluating which way is easiest or just bite the bullet and implement the metrics collection ourselves.

[CaptainPickles89]

Ok thanks for sharing the use case here. So in theory you should be able to do something like this in the collector:

processors:
  metricstransform:
    transforms:
      - include: <your_metric_name>
        match_type: strict
        action: update
        operations:
          - action: aggregate_label_values
            label: status
            aggregated_values: [ good, meh ]
            new_value: <new_label> 
            aggregation_type: sum

I haven't tested that, but in theory it should grab any of the metrics (I am assuming the actual name is the same for all those) then if they have a status tag with a value of good or meh, sum the values.