prometheus

Commit Graph

Author	SHA1	Message	Date
Bartlomiej Plotka	8e6b008608	feature: type-and-unit-labels (PROM-39 implementation) (#16228 ) * feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> * Fix compilation errors Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Lint Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Revert change made to protobuf 'Accept' header Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Fix compilation errors for 'dedupelabels' tag Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Rectored into schema.Metadata Signed-off-by: bwplotka <bwplotka@gmail.com> * texparse: Added tests for PromParse Signed-off-by: bwplotka <bwplotka@gmail.com> * add OM tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add proto tests Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * add schema label tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * fix tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add promql tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * lint Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>	7 months ago
Matthieu MOREL	5fa1146e21	chore: enable gci linter (#16245 ) Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	9 months ago
Bartlomiej Plotka	dc85d677d8	textparse: Optimize CreatedTimestamp; It returns int64 value now. (#16072 ) ..instead of *int64. This is as an optimization and ease of use. We already accepted in many places (proto histograms, PRW) that CT (or any timestamp really) 0 means not set. Signed-off-by: bwplotka <bwplotka@gmail.com>	10 months ago
Bartlomiej Plotka	733a5e9eb4	textparse: Optimized protobuf parser with custom streaming unmarshal. (#15731 ) * textparse: Optimized protobuf parser with custom streaming decoder. Signed-off-by: bwplotka <bwplotka@gmail.com> Update model/textparse/protobufparse.go Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Addressing comments. Signed-off-by: bwplotka <bwplotka@gmail.com> decoder: reuse histograms and summaries. Signed-off-by: bwplotka <bwplotka@gmail.com> optimize help returning (5% of mem utilization). Signed-off-by: bwplotka <bwplotka@gmail.com> Apply suggestions from code review Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Update prompb/io/prometheus/client/decoder.go Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Fix build. Signed-off-by: bwplotka <bwplotka@gmail.com> * Update model/textparse/protobufparse.go Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	10 months ago
Bartlomiej Plotka	00b69efabb	model/textparse: Change parser interface Metric(...) string to Labels(...) (#16012 ) * model/textparse: Change parser interface Metric(...) string to Labels(...) Simplified the interface given no one is using the return argument. Renamed for clarity too. Found and discussed https://github.com/prometheus/prometheus/pull/15731#discussion_r1950916842 Signed-off-by: bwplotka <bwplotka@gmail.com> * Fixed comments; optimized not needed copy for om and text. Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com>	10 months ago
machine424	18b81ad79d	feat: ProtobufParse.formatOpenMetricsFloat: improve float formatting by using strconv.AppendFloat instead of fmt.Sprint Signed-off-by: machine424 <ayoubmrini424@gmail.com>	1 year ago
Bartlomiej Plotka	f6e110d588	textparse: Refactored main testing utils for reusability; fixed proto Units. (#15095 ) Signed-off-by: bwplotka <bwplotka@gmail.com>	1 year ago
Nathan Baulch	50cd453c8f	chore: Fix typos (#14868 ) * Fix typos --------- Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>	1 year ago
Bryan Boreham	d599c4b28c	[BUGFIX] Protobuf scraping: reset exemplar position Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
riskrole	406bf775aa	chore: fix some comments Signed-off-by: riskrole <yuhang@before.tech>	1 year ago
Bryan Boreham	eff3a13e19	model/textparse: parsers take a labels SymbolTable This allows strings to be interned to save memory. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Ziqi Zhao	a93859a52f	Prometheus support parse exemplars from native histogram (#13488 ) scrape: support parsing exemplars from native histogram --------- Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com> Signed-off-by: Björn Rabenstein <github@rabenste.in> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2 years ago
Arianna Vespri	8f07f9dd90	Avoid creating string for suffix, consider counters without _total suffix Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Arianna Vespri	9a664b515a	Correct order in error message Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Arianna Vespri	51e78d9a32	Delete debugging lines, amend error message for unit Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Bryan Boreham	8065bef172	Move metric type definitions to common/model They are used in multiple repos, so common is a better place for them. Several packages now don't depend on `model/textparse`, e.g. `storage/remote`. Also remove `metadata` struct from `api.go`, since it was identical to a struct in the `metadata` package. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Arianna Vespri	b65021d7a5	Get conditional right Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Arianna Vespri	9fb1e9715c	Go on adding protobuf parsing for unit Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Arianna Vespri	5d5acf5744	Add unit protobuf parser Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Arthur Silva Sens	5082655392	Append Created Timestamps (#12733 ) * Append created timestamps. Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Log when created timestamps are ignored Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Proposed changes to Append CT PR. Changes: * Changed textparse Parser interface for consistency and robustness. * Changed CT interface to be more explicit and handle validation. * Simplified test, change scrapeManager to allow testability. * Added TODOs. Signed-off-by: bwplotka <bwplotka@gmail.com> * Updates. Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * Refactor head_appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Fix linter issues Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Use model.Sample in head appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> --------- Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: bwplotka <bwplotka@gmail.com>	2 years ago
zenador	32ee1b15de	Fix error on ingesting out-of-order exemplars (#13021 ) Fix and improve ingesting exemplars for native histograms. See code comment for a detailed explanation of the algorithm. Note that this changes the current behavior for all kind of samples slightly: We now allow exemplars with the same timestamp as during the last scrape if the value or the labels have changed. Also note that we now do not ingest exemplars without timestamps for native histograms anymore. Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Björn Rabenstein <github@rabenste.in> --------- Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Signed-off-by: zenador <zenador@users.noreply.github.com> Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2 years ago
Matthieu MOREL	fe057fc60d	use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
beorn7	b428416f06	textparse: Update comment about timestamp_ms protobuf parsing By now, we know better what the plan is. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Arthur Silva Sens	ef8e6ae780	Parse created timestamps from Prometheus Protobuf (#12973 ) Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>	2 years ago
beorn7	65ccf4460a	textparse: Fix endless loop #12731 PR #12557 introduced the possibility of parsing multiple exemplars per native histograms. It did so by requiring the `Exemplar` method of the parser to be called repeatedly until it returns false. However, the protobuf parser code wasn't correctly updated for the old case of a single exemplar for a classic bucket (if actually parsed as a classic bucket) and a single exemplar on a counter. In those cases, the method would return `true` forever, yielding the same exemplar again and again, leading to an endless loop. With this fix, the state is now tracked and the single exemplar is only returned once. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
György Krajcsovits	e846736134	Fix typo Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
beorn7	c58e20ad0e	histogram: Identify native histograms even without observations Native histograms without observations and with a zero threshold of zero look the same as classic histograms in the protobuf exposition format. According to https://github.com/prometheus/client_golang/issues/1127 , the idea is to add a no-op span to those histograms to mark them as native histograms. This commit enables Prometheus to detect that no-op span and adds a doc comment to the proto spec describing the behavior. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	071f4bbea4	histograms: Fix parsing float histograms without zero bucket If a float histogram has a zero bucket with a threshold of zero _and_ an empty zero bucket, it wasn't identified as a native histogram because the `isNativeHistogram` helper function only looked at integer buckets. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	da047c6857	histograms: Fix bug #12552 The problem was the following: When trying to parse native histograms and classic histograms in parallel, the parser would first parse the histogram proto messages as a native histogram and then parse the same message again, but now as a classic histogram. Afterwards, it would forget that it was dealing with a metric family that contains native histograms and would parse the rest of the metric family as classic histograms only. The fix is to check again after being done with a classic histogram. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	9e500345f3	textparse/scrape: Add option to scrape both classic and native histograms So far, if a target exposes a histogram with both classic and native buckets, a native-histogram enabled Prometheus would ignore the classic buckets. With the new scrape config option `scrape_classic_histograms` set, both buckets will be ingested, creating all the series of a classic histogram in parallel to the native histogram series. For example, a histogram `foo` would create a native histogram series `foo` and classic series called `foo_sum`, `foo_count`, and `foo_bucket`. This feature can be used in a migration strategy from classic to native histograms, where it is desired to have a transition period during which both native and classic histograms are present. Note that two bugs in classic histogram parsing were found and fixed as a byproduct of testing the new feature: 1. Series created from classic _gauge_ histograms didn't get the _sum/_count/_bucket prefix set. 2. Values of classic _float_ histograms weren't parsed properly. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	c26891043f	textparse: Set CounterResetHint If a (float or integer) histogram is a gauge histogram, set the CounterResetHint accordingly. (The default value is fine for the normal counter histograms.) Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	b5d4a94e9d	textparse: Add gauge histogram support to protobuf parsing With this commit, the parser stops to see a gauge histogram (whether native or conventional) as an unexpected metric type. It ingests it normally, it even sets the `GaugeHistogram` type in the metadata (as it has already done for a conventional gauge histogram scraped using OpenMetrics), but it otherwise treats it as a normal counter-like histogram. Once #11783 is merged, though, it should be very easy to utilize the type information. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	e9d9bb1b08	textparse: Handle unknown metric types in protobuf gracefully So far, the parser hasn't validated that the type is valid in the `Next()` call. Later, in the `Series()` call, however, it assumes that we will only see valid types and therefore panics with `encountered unexpected metric type, this is a bug`. This commit fixes said bug by adding validation to the `Next()` call. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Bryan Boreham	1f04899ae3	Update package model/textparse for new labels.Labels type Parse metrics using labels.ScratchBuilder, so we reduce assumptions about internals of Labels. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
beorn7	a7c519930e	histograms: Add Compact method to the normal integer Histogram And use the new method to call to compact Histograms during parsing. This happens for both `Histogram` and `FloatHistogram`. In this way, if targets decide to optimize the exposition size by merging spans with empty buckets in between, we still get a normalized results. It will also normalize away any valid but weird representations like empty spans, spans with offset zero, and empty buckets at the start or end of a span. The implementation seemed easy at first as it just turns the `compactBuckets` helper into a generic function (which now got its own file). However, the integer Histograms have delta buckets instead of absolute buckets, which had to be treated specially in the generic `compactBuckets` function. To make sure it works, I have added plenty of explicit tests for `Histogram` in addition to the `FloatHistogram` tests. I have also updated the doc comment for the `Compact` method. Based on the insights now expressed in the doc comment, compacting with a maxEmptyBuckets > 0 is rarely useful. Therefore, this commit also sets the value to 0 in the two cases we were using 3 so far. We might still want to reconsider, so I don't want to remove the maxEmptyBuckets parameter right now. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Marc Tudurí	f7df3b86ba	histograms: parse float histograms from proto definition (#11149 ) * histograms: parse float histograms from proto definition Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Improve comment Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Ignore float buckets Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Refactor Histogram() function Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Fix test_float_histogram Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Update model/textparse/protobufparse.go Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update protobufparse.go Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update scrape.go Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update scrape/scrape.go Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> Signed-off-by: Marc Tuduri <marctc@protonmail.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>	3 years ago
beorn7	c40b105efd	histograms: Move to new exposition protobuf format This is an incompatible protobuf change. Instrumented targets must include https://github.com/prometheus/client_golang/pull/1092 to make this work. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago

41 Commits (aa1d46a9da0f9710ca1528bef7623f1b4fb93f53)