prometheus

Commit Graph

Author	SHA1	Message	Date
George Krajcsovits	5b7ff92d95	fix(promql): histogram_quantile and histogram_fraction NaN observed in native histogram (#16724 ) * fix(promql): histogram_quantile NaN observed in native histogram Fixes: #16578 See the issue for detailed explanation. When a histogram had only NaN observations and no normal observations, we returned 0 from the quantile, which is completely wrong. If there were normal observations but we went over them, we returned the upper bound of the existing buckets, however that contradicts expectations on histogram_fraction. Now we return NaN if the quantile is calculated to be over all normal observations, falling into NaNs (in a virtual +Inf bucket). We also return info level annotations if we see any NaN observations. The annotation calls out if we returned NaN or even if we took the virtual +Inf bucket into account. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * fix(promql): histogram_fraction NaN observed in native histogram Fixes: #16580 According to the specification we should not take NaN observations into account when calculating the fraction. This commit fixes that and adds an info level annotation to let the user know about this. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 days ago
🌲 Harry 🌊 John 🏔	59bf3d442b	PromQL: Fix native histogram last_over_time with offset Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>	1 week ago
Michael Hoffmann	ec98e7f069	promql: fix ts_of_last_over_time for histogram vectors This PR fixes a bug in ts_of_last_over_time where the float samples where used when computing the last timestamp of the histogram samples. Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>	2 weeks ago
Michael Hoffmann	a5fa9431d8	promql: add ts_of_(max,min,last)_over_time functions This commit adds the ts_of_(max,min,last)_over_time functions behind the experimental feature flag. Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>	2 weeks ago
beorn7	68b1bcc648	promql: typo fix in comment Signed-off-by: beorn7 <beorn@grafana.com>	2 weeks ago
beorn7	ff67596a82	promql: Simplify avg aggregation and avg_over_time As it turns out, if we combine Kahan summation and incremental mean calculation properly, it works quite well and we do not need to switch between simple mean calculation and incremental calculation based on overflow. This simplifies the code quite a bit. Signed-off-by: beorn7 <beorn@grafana.com>	3 weeks ago
Bartlomiej Plotka	8e6b008608	feature: type-and-unit-labels (PROM-39 implementation) (#16228 ) * feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> * Fix compilation errors Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Lint Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Revert change made to protobuf 'Accept' header Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Fix compilation errors for 'dedupelabels' tag Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Rectored into schema.Metadata Signed-off-by: bwplotka <bwplotka@gmail.com> * texparse: Added tests for PromParse Signed-off-by: bwplotka <bwplotka@gmail.com> * add OM tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add proto tests Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * add schema label tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * fix tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add promql tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * lint Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>	1 month ago
Neeraj Gartia	591242901a	promql: Refactor some functions to make them more DRY (#16532 ) Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2 months ago
Charles Korn	2c0f02a702	promql: don't emit a value from `histogram_fraction` or `histogram_quantile` if classic and native histograms are present at the same timestamp Signed-off-by: Charles Korn <charles.korn@grafana.com>	2 months ago
Aman	26bddcf068	promql: histogram_stddev and histogram_stdvar should use arithmetic mean for custom buckets Signed-off-by: amanycodes <amanycodes@gmail.com>	2 months ago
Michael Hoffmann	d6d9f97bac	promql: histogram_fraction for bucket histograms (#16095 ) * promql: histogram_fraction for bucket histograms This PR extends the histogram_fraction function to also work with classic bucket histograms. This is beneficial because it allows expressions like sum(increase(my_bucket{le="0.5"}[10m]))/sum(increase(my_total[10m])) to be written without knowing the actual values of the "le" label, easing the transition to native histograms later on. It also feels natural since histogram_quantile also can deal with classic histograms. Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> * promql: histogram_fraction for bucket histograms * Add documentation and reduce code duplication * Fix a bug in linear interpolation between bucket boundaries * Add more PromQL tests Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> * Update docs/querying/functions.md Co-authored-by: Björn Rabenstein <github@rabenste.in> Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> --------- Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2 months ago
Neeraj Gartia	a511164615	[REFACTOR] PromQL: DRY `stddev` and `stdvar` functions (#16451 ) * DRY stddev and stdvar --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2 months ago
Charles Korn	a2aeb813af	promql: return NaN from `irate()` if second-last sample is NaN (#16199 ) promql: return NaN from `irate()` if either of last two samples is NaN Signed-off-by: Charles Korn <charles.korn@grafana.com> --------- Signed-off-by: Charles Korn <charles.korn@grafana.com>	3 months ago
Charles Korn	c88d0b0e0a	promql: return NaN from `irate()` if second-last sample is NaN (#16199 ) promql: return NaN from `irate()` if either of last two samples is NaN Signed-off-by: Charles Korn <charles.korn@grafana.com> --------- Signed-off-by: Charles Korn <charles.korn@grafana.com>	4 months ago
Matthieu MOREL	c7d4b53ec1	chore: enable unused-parameter from revive Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	4 months ago
Charles Korn	0964b6e584	promql: emit correct annotation in `quantile_over_time` when run over a range with histograms and floats Signed-off-by: Charles Korn <charles.korn@grafana.com>	4 months ago
Neeraj Gartia	20371118b6	[FIX] PromQL: Ignore histograms in `scalar`, `sort` and `sort_desc` functions (#15964 ) PromQL: Ignore histograms in scalar, sort and sort_desc functions --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	5 months ago
Joshua Hesketh	5a5fdea7ad	Fix duplicate output vector if delayed name removal is disabled (#15975 ) Fix duplicate output vector if delayed name removal is disabled This error is emitted in cleanupMetricLabels, but is skipped if enableDelayedNameRemoval is false. This makes it consistent with label_replace Signed-off-by: Joshua Hesketh <josh@nitrotech.org> --------- Signed-off-by: Joshua Hesketh <josh@nitrotech.org> Signed-off-by: Björn Rabenstein <github@rabenste.in> Co-authored-by: Björn Rabenstein <github@rabenste.in>	5 months ago
Joshua Hesketh	e4037b3ec3	Allow UTF-8 labels in label_replace This makes it consistent with label_join. Signed-off-by: Joshua Hesketh <josh@nitrotech.org>	5 months ago
Neeraj Gartia	130cd024e0	[FEATURE] PromQL: Implements `idelta` and `irate` with histograms (#15853 ) Add native histogram support to idelta and irate functions --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	5 months ago
beorn7	2581c7d057	promql: fix rate calculation with a counter reset after the 1st histogram If a rate (or increase) is calculated on native histograms, and there is a counter reset between the 1st and 2nd histogram, we never have to touch the 1st histogram, so it doesn't even matter if it has an incompatible bucket layout. So we should not error out in that case. This simply nulls out the 1st histogram in that case. Signed-off-by: beorn7 <beorn@grafana.com>	5 months ago
Joshua Hesketh	665d1ba0cc	Ensure metric name is present on histogram_quantile annotations (#15828 ) Ensure metric name is present on histogram_quantile annotations Previously the series __name__ label was dropped before the annotation was emitted causing the annotation message to have a blank value. This fix also allows delayed name removal for classic histograms. This also adds a test for malformed le label This annotation could be split into two to be clearer for the user (one for missing, and one for malformed). This is outside of the scope of this change. Fixes: #15411 --------- Signed-off-by: Joshua Hesketh <josh@nitrotech.org> Signed-off-by: Björn Rabenstein <github@rabenste.in> Co-authored-by: Björn Rabenstein <github@rabenste.in>	5 months ago
Neeraj Gartia	b3e30d52ce	[BUGFIX] PromQL: Fix `<aggr_over_time>` functions with histograms (#15711 ) fix aggr_over_time with histograms Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	6 months ago
Neeraj Gartia	0e99ca3e8c	[BUGFIX] PromQL: Fix `deriv`, `predict_linear` and `double_exponential_smoothing` with histograms (#15686 ) PromQL: Fix deriv, predict_linear and double_exponential_smoothing with histograms Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	6 months ago
beorn7	ebfa1dd822	promql: Purge Holt-Winters from a doc comment `funcDoubleExponentialSmoothing` did not get its doc comment updated when we renamed it from the confusing `funcHoltWinters`. Signed-off-by: beorn7 <beorn@grafana.com>	7 months ago
Neeraj Gartia	4aeed2c4b1	fix resets function for histograms Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	7 months ago
Neeraj Gartia	0de340c1b4	ignore histogram in dateWrapper Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	7 months ago
Neeraj Gartia	36e0897f0f	[BUGFIX] PromQL: Fix behaviour of `changes()` for mix of histograms and floats (#15469 ) PromQL: Fix behaviour of changes() for mix of histograms and floats --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	7 months ago
Joshua Hesketh	8e3301eb44	Export quantile functions (#15190 ) Export quantile functions For use in Mimir's query engine, it would be helpful if these functions were exported. Co-authored-by: Björn Rabenstein <github@rabenste.in> Signed-off-by: Joshua Hesketh <josh@hesketh.net.au> --------- Signed-off-by: Joshua Hesketh <josh@nitrotech.org> Signed-off-by: Joshua Hesketh <josh@hesketh.net.au> Co-authored-by: Björn Rabenstein <github@rabenste.in>	7 months ago
Neeraj Gartia	abfc3145a2	apply DRY on clamp (#15441 ) Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	7 months ago
Ben Ye	b7aca45de7	fix round function ignoring enableDelayedNameRemoval feature flag Signed-off-by: Ben Ye <benye@amazon.com>	8 months ago
Joshua Hesketh	14ef1ce8ab	Round function should ignore native histograms As per the documentation, native histograms are skipped. This is in line with other simpleFunc's. Signed-off-by: Joshua Hesketh <josh@nitrotech.org>	8 months ago
Arve Knudsen	de16f5e387	[FEATURE] PromQL: Add experimental info function MVP (#14495 ) The `info` function is an experiment to improve UX around including labels from info metrics. `info` has to be enabled via the feature flag `--enable-feature=promql-experimental-functions`. This MVP of info simplifies the implementation by assuming: * Only support for the target_info metric * That target_info's identifying labels are job and instance Also: * Encode info samples' original timestamp as sample value * Deduce info series select hints from top-most VectorSelector --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Ying WANG <ying.wang@grafana.com> Co-authored-by: Augustin Husson <augustin.husson@amadeus.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Björn Rabenstein <github@rabenste.in> Co-authored-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Joshua Hesketh	31d19381f6	Clamp functions should ignore native histograms As per the documentation, native histograms are skipped. This is in line with other `simpleFunc`'s. Signed-off-by: Joshua Hesketh <josh@nitrotech.org>	8 months ago
Bryan Boreham	b8e5b7cda9	[REFACTOR] PromQL: remove label_join and label_replace stubs These functions operate on whole series, not on samples, so they do not fit into the table of functions that return a Vector. Remove the stub entries that were left to help downstream users of the code identify what changed. We cannot remove the entries from the `FunctionCalls` map without breaking `TestFunctionList`, so put some nils in to keep it happy. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	7f99d2930d	[BUGFIX] PromQL: make sort_by_label stable Go's sorting functions can re-order equal elements, so the strategy of sorting by the fallback ordering first does not always work. Pulling the fallback into the main comparison function is more reliable and more efficient. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Jan Fajerski	96e5a94d29	promql: rename holt_winters to double_exponential_smoothing Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	9 months ago
Mario Fernandez	5814920601	Fix: optimize .* regexp performance Shortcut for `.*` matches newlines as well. Add preamble change ^(?s: Add test dotAll flag por al regex Add and fix regex tests Signed-off-by: Mario Fernandez <mariofer@redhat.com>	9 months ago
beorn7	2304be9dd5	promql: Improve the rate extrapolation explanation Signed-off-by: beorn7 <beorn@grafana.com>	10 months ago
Bryan Boreham	8742077498	[BUGFIX] PromQL: pass Context so spans parent correctly Assigning to `evaluator.ctx` in `eval()` broke the parent-child relationship. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	10 months ago
Jorge Creixell	e9e3d64b7c	PromQL engine: Delay deletion of __name__ label to the end of the query evaluation (#14477 ) PromQL engine: Delay deletion of __name__ label to the end of the query evaluation - This change allows optionally preserving the `__name__` label via the `label_replace` and `label_join` functions, and helps prevent the dreaded "vector cannot contain metrics with the same labelset" error. - The implementation extends the `Series` and `Sample` structs with a boolean flag indicating whether the `__name__` label should be deleted at the end of the query evaluation. - The `label_replace` and `label_join` functions can still access the value of the `__name__` label, even if it has been previously marked for deletion. If `__name__` is used as target label, it won't be dropped at the end of the query evaluation. - Fixes https://github.com/prometheus/prometheus/issues/11397 - See https://github.com/jcreixell/prometheus/pull/2 for previous discussion, including the decision to create this PR and benchmark it before considering other alternatives (like refactoring `labels.Labels`). - See https://github.com/jcreixell/prometheus/pull/1 for an alternative implementation using a special label instead of boolean flags. - Note: a feature flag `promql-delayed-name-removal` has been added as it changes the behavior of some "weird" queries (see https://github.com/prometheus/prometheus/issues/11397#issuecomment-1451998792) Example (this always fails, as `__name__` is being dropped by `count_over_time`): ``` count_over_time({__name__!=""}[1m]) => Error executing query: vector cannot contain metrics with the same labelset ``` Before: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => Error executing query: vector cannot contain metrics with the same labelset ``` After: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => count_go_gc_cycles_automatic_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 count_go_gc_cycles_forced_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 ... ``` Signed-off-by: Jorge Creixell <jcreixell@gmail.com> --------- Signed-off-by: Jorge Creixell <jcreixell@gmail.com> Signed-off-by: Björn Rabenstein <github@rabenste.in>	10 months ago
suntala	8e82ac8d2b	Add comments to the sort by label functions Signed-off-by: suntala <arati.rana@grafana.com>	10 months ago
Arve Knudsen	3a78e76282	Upgrade golangci-lint to v1.60.1 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	10 months ago
suntala	fd2f44af7f	Fall back to comparing by label set when sorting by label desc Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com> Signed-off-by: suntala <arati.rana@grafana.com>	11 months ago
suntala	94ad489328	Fall back to comparing by label set when sorting by label Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com> Signed-off-by: suntala <arati.rana@grafana.com>	11 months ago
Charles Korn	424cefcf5e	Fix "cannot reduce resolution to custom buckets schema" panic in `rate` over native histograms with mix of custom and exponential buckets Signed-off-by: Charles Korn <charles.korn@grafana.com>	11 months ago
Charles Korn	aadec25faf	promql: Fix issue where some native histogram-related annotations are not emitted by `rate` (#14575 ) Signed-off-by: Charles Korn <charles.korn@grafana.com>	11 months ago
beorn7	cff0429b1a	promql: make avg_over_time faster and more precise Same idea as for the avg aggregator before: Most of the time, there is no overflow, so we don't have to revert to the more expensive and less precise incremental calculation of the mean value. Signed-off-by: beorn7 <beorn@grafana.com>	12 months ago
beorn7	3a908d8e08	promql: Improve Kahan usage in avg_over_time The calculation of the mean value in avg_over_time is performed in an incremental fashion. This introduces additional numerical errors that even Kahan summation cannot compensate, but at least we can use the Kahan-corrected mean value when we use the intermediate mean value in the calculation. Signed-off-by: beorn7 <beorn@grafana.com>	12 months ago
Charles Korn	fd6bdf5230	Fix issue where summation of +/- infinity returns NaN instead of infinity Signed-off-by: Charles Korn <charles.korn@grafana.com>	1 year ago

1 2 3 4 5

243 Commits (main)