prometheus

Commit Graph

Author	SHA1	Message	Date
Manik Rana	032ca9ef96	[PERF] textparse: further optimzations for OM `CreatedTimestamps` (#15097 ) * feat: Added more tests; some changes/optimizations when pair-programming with Bartek. Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: imports Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: gofumpt Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: use an efficient replacement to p.Metric Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: reduce mem allocs + comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: gofumpt Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: use single quotes Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * refac: rename Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * refac: rename to seriesHash Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: switch condition order Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * refac: switch condition order Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * feat: stronger checking Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * chore: fmt Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: pass pointer of buf into seriesHash() Signed-off-by: Manik Rana <manikrana54@gmail.com> --------- Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	1 year ago
Bartlomiej Plotka	f6e110d588	textparse: Refactored main testing utils for reusability; fixed proto Units. (#15095 ) Signed-off-by: bwplotka <bwplotka@gmail.com>	1 year ago
Bartlomiej Plotka	8d281c3491	textparse: Refactored benchmark (#15083 ) * textparse: Refactored benchmark. Signed-off-by: bwplotka <bwplotka@gmail.com> * Kill sinks, tested, nothing is inlined. Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com>	1 year ago
Matthieu MOREL	ab64966e9d	fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" (#15094 ) * fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" --------- Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	1 year ago
Manik Rana	47aeca9663	feat: naive fixes and optimzations for `CreatedTimestamp` function (#14965 ) * enhance: wip ct parse optimizations Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: further work on optimization Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: further improvements and remove unused code Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: improve optimizations and fix some CT parse errors Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: check for LsetHash along with name Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: cleanup and documentation Signed-off-by: Manik Rana <manikrana54@gmail.com> * enhance: improve comments and add cleaner functions Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: improve comments and add cleaner functions Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: rename to resetCTParseValues Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: post-merge fixes Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: add all possible reserved suffixes Signed-off-by: Manik Rana <manikrana54@gmail.com> * test: separate CT values for each metric Signed-off-by: Manik Rana <manikrana54@gmail.com> --------- Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com>	1 year ago
Manik Rana	f1c57a95ed	change: No longer ingest OM _created as timeseries if feature-flag 'enable-ct-zero-ingestion' is enabled; fixed OM text CT conversion bug (#14738 ) * chore: revert TypeRequiresCT to private Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: init NewOpenMetricsParser with skipCT true Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: allow opt-in to OM CT ingestion Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: use textparse interface to set om options Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: set skipOMSeries in test Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: gofumpt Signed-off-by: Manik Rana <manikrana54@gmail.com> * wip: add tests for OM CR parse Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: merge ct tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: add cases for OM text Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: check correct test cases Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: use both scrape protocols in config Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: fix inputs and output tests for OM Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: cleanup Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: rename skipOMSeries to skipOMCTSeries Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * fix: finish refac Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: move setup code outside test Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: verify _created lines create new metric in certain cases Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: post merge fixes Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * manager: Fixed CT OMText conversion bug; Refactored tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: gofumpt Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: imports Signed-off-by: Manik Rana <manikrana54@gmail.com> --------- Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: bwplotka <bwplotka@gmail.com>	1 year ago
Manik Rana	98cd80b2e2	feat: add microbenchmarks for OM CT parsing (#14933 ) * test: benchmark OM CT parsing Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: move OM ct benchmark to promparse_test Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: stricter comparison Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * feat: use richer OM test data Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: move parse-ct test outside of inner loop Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: separate benchmarks for om and prom parsers Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: remove unused code Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: remove more unused code Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: rename to BenchmarkOMParseCreatedTimestamp Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> --------- Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	1 year ago
Bryan Boreham	31c5760551	Neater string vs byte-slice conversions (#14425 ) unsafe.Slice and unsafe.StringData were added in Go 1.20 Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Manik Rana	315165e49d	refac: make typeRequiresCT private Signed-off-by: Manik Rana <manikrana54@gmail.com>	1 year ago
Nathan Baulch	50cd453c8f	chore: Fix typos (#14868 ) * Fix typos --------- Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>	1 year ago
Bryan Boreham	d599c4b28c	[BUGFIX] Protobuf scraping: reset exemplar position Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Bryan Boreham	3aaf2c3c9d	[TESTS] Add second native histogram with exemplars in TestProtobufParse This test fails. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Bryan Boreham	e8ecc11816	[TESTS] Fix up item numbers in TestProtobufParse If an error is thrown by the test, these numbers help you see which item is wrong. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
riskrole	406bf775aa	chore: fix some comments Signed-off-by: riskrole <yuhang@before.tech>	1 year ago
beorn7	0f760f63dd	lint: Revamp our linting rules, mostly around doc comments Several things done here: - Set `max-issues-per-linter` to 0 so that we actually see all linter warnings and not just 50 per linter. (As we also set `max-same-issues` to 0, I assume this was the intention from the beginning.) - Stop using the golangci-lint default excludes (by setting `exclude-use-default: false`. Those are too generous and don't match our style conventions. (I have re-added some of the excludes explicitly in this commit. See below.) - Re-add the `errcheck` exclusion we have used so far via the defaults. - Exclude the signature requirement `govet` has for `Seek` methods because we use non-standard `Seek` methods a lot. (But we keep other requirements, while the default excludes completely disabled the check for common method segnatures.) - Exclude warnings about missing doc comments on exported symbols. (We used to be pretty adamant about doc comments, but stopped that at some point in the past. By now, we have about 500 missing doc comments. We may consider reintroducing this check, but that's outside of the scope of this commit. The default excludes of golangci-lint essentially ignore doc comments completely.) - By stop using the default excludes, we now get warnings back on malformed doc comments. That's the most impactful change in this commit. It does not enforce doc comments (again), but _if_ there is a doc comment, it has to have the recommended form. (Most of the changes in this commit are fixing this form.) - Improve wording/spelling of some comments in .golangci.yml, and remove an outdated comment. - Leave `package-comments` inactive, but add a TODO asking if we should change that. - Add a new sub-linter `comment-spacings` (and fix corresponding comments), which avoids missing spaces after the leading `//`. Signed-off-by: beorn7 <beorn@grafana.com>	1 year ago
Manik Rana	02c465bf58	textparse: Implement `CreatedTimestamp()` in `openmetricsparse.go` (#14356 ) * feat: initial implement of createedTimestamp() with tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: return ct after finding it Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: remove unneeded test Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: add comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: multiple changes - implement changes from pair programming session - use newParse.val() - advance parser p if ct is found Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: check if err from p.Next() Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: advance parser and parse histograms + summary Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: restore previous tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: retore failing tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: remove unneeded comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: return nil when mtype doesn't match Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: update go fmt version Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: cleanup Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: document deepcopyparser Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: cover edgecase of `gauge_created` in CreatedTimestamp() Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: readability updates Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: dedeuplicate labeldiff checks Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: add tests for new label functions Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: document CreatedTimestamp func Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: optimize `CreatedTimestamp()` - Use refactored CreatedTimestamp function with bug fixes - Remove unused code in labels.go - Improve code documentation Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: add tests and lint fixes Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: remove mName Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: add tests for CT parse failures and deepCopy Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: edit expectCT struct Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: add new label in deepCopy Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: use p.builder in deepCopy Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: add NewMetricsParserWithOpts Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * chore: comments Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * chore: rename var Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: add condition for OM fuzzing Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: build tags Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: default skipCT to false Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: rename skipCT to skipCTSeries Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: formatting Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments and readability updates Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * refac: remove NewOpenMetricsParserWithOpts Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: extract skipCTSeries logic from parseMetricSuffix Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: inline create a NewOpenMetricsParser Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * refac: improve error handling Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: return error instead of nil Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: remove skipCT check from tBraceOpen Signed-off-by: Manik Rana <manikrana54@gmail.com> * Pair programming with Manik, Arthur and Daniel. Signed-off-by: bwplotka <bwplotka@gmail.com> * chore: comments and use helper funcs Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> --------- Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	1 year ago
Arthur Silva Sens	07355c9199	Bump client_golang to 1.19 Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>	2 years ago
Ondrej Kokes	2e30f1231b	docs: textparse.Parser return type mismatch The docs suggest the Next method returns a bool, but that's not the case (`Entry` is an int). ``` // Next advances the parser to the next sample. It returns false if no // more samples were read or an error occurred. Next() (Entry, error) ``` The docs were first added in `d80a3de235` in 2017. Back then the signature was indeed `func (p *Parser) Next() bool`. But then it got refactored in `76a4a46cb0` and the signature changed with it, yet docs stayed the same - and eventually made their way into the `Parser` interface. However, the Protobuf parser does have the right wording: `5de2df752f` ``` // Next advances the parser to the next "sample" (emulating the behavior of a // text format parser). It returns (EntryInvalid, io.EOF) if no samples were // read. ``` Changing all other implementations (and the interface itself) to match this doc. Signed-off-by: Ondrej Kokes <ondrej.kokes@gmail.com>	2 years ago
Bryan Boreham	ba6aa95222	refactor: metrics parsing: extract common checking code Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	eff3a13e19	model/textparse: parsers take a labels SymbolTable This allows strings to be interned to save memory. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	93b72ec5dd	tsdb: create SymbolTables for labels as required Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Łukasz Mierzwa	5597020a60	Use github.com/klauspost/compress for gzip and zlib klauspost/compress is a high quality drop-in replacement for common Go compression libraries. Since Prometheus sends out a lot of HTTP requests that often return compressed output having improved compression libraries helps to save cpu & memory resources. On a test Prometheus server I was able to see cpu reduction from 31 to 30 cores. Benchmark results: name old time/op new time/op delta TargetScraperGzip/metrics=1-8 69.4µs ± 4% 69.2µs ± 3% ~ (p=0.122 n=50+50) TargetScraperGzip/metrics=100-8 84.3µs ± 2% 80.9µs ± 2% -4.02% (p=0.000 n=48+46) TargetScraperGzip/metrics=1000-8 296µs ± 1% 274µs ±14% -7.35% (p=0.000 n=47+45) TargetScraperGzip/metrics=10000-8 2.06ms ± 1% 1.66ms ± 2% -19.34% (p=0.000 n=47+45) TargetScraperGzip/metrics=100000-8 20.9ms ± 2% 17.5ms ± 3% -16.50% (p=0.000 n=49+50) name old alloc/op new alloc/op delta TargetScraperGzip/metrics=1-8 6.06kB ± 0% 6.07kB ± 0% +0.24% (p=0.000 n=48+48) TargetScraperGzip/metrics=100-8 7.04kB ± 0% 6.89kB ± 0% -2.17% (p=0.000 n=49+50) TargetScraperGzip/metrics=1000-8 9.02kB ± 0% 8.35kB ± 1% -7.49% (p=0.000 n=50+50) TargetScraperGzip/metrics=10000-8 18.1kB ± 1% 16.1kB ± 2% -10.87% (p=0.000 n=47+47) TargetScraperGzip/metrics=100000-8 1.21MB ± 0% 1.01MB ± 2% -16.69% (p=0.000 n=36+50) name old allocs/op new allocs/op delta TargetScraperGzip/metrics=1-8 71.0 ± 0% 72.0 ± 0% +1.41% (p=0.000 n=50+50) TargetScraperGzip/metrics=100-8 81.0 ± 0% 76.0 ± 0% -6.17% (p=0.000 n=50+50) TargetScraperGzip/metrics=1000-8 92.0 ± 0% 83.0 ± 0% -9.78% (p=0.000 n=50+50) TargetScraperGzip/metrics=10000-8 93.0 ± 0% 91.0 ± 0% -2.15% (p=0.000 n=50+50) TargetScraperGzip/metrics=100000-8 111 ± 0% 135 ± 1% +21.89% (p=0.000 n=40+50) Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	2 years ago
Owen Williams	a28d7865ad	UTF-8: Add support for parsing UTF8 metric and label names This adds support for the new grammar of `{"metric_name", "l1"="val"}` to promql and some of the exposition formats. This grammar will also be valid for non-UTF-8 names. UTF-8 names will not be considered valid unless model.NameValidationScheme is changed. This does not update the go expfmt parser in text_parse.go, which will be addressed by https://github.com/prometheus/common/issues/554/. Part of https://github.com/prometheus/prometheus/issues/13095 Signed-off-by: Owen Williams <owen.williams@grafana.com>	2 years ago
Ziqi Zhao	a93859a52f	Prometheus support parse exemplars from native histogram (#13488 ) scrape: support parsing exemplars from native histogram --------- Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com> Signed-off-by: Björn Rabenstein <github@rabenste.in> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2 years ago
Bryan Boreham	17f48f2b3b	Tests: use replacement DeepEquals in more places Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Arianna Vespri	8f07f9dd90	Avoid creating string for suffix, consider counters without _total suffix Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Arianna Vespri	9a664b515a	Correct order in error message Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Arianna Vespri	51e78d9a32	Delete debugging lines, amend error message for unit Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Bryan Boreham	c83e1fc574	textparse: remove MetricType alias No backwards-compatibility; make a clean break. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	8065bef172	Move metric type definitions to common/model They are used in multiple repos, so common is a better place for them. Several packages now don't depend on `model/textparse`, e.g. `storage/remote`. Also remove `metadata` struct from `api.go`, since it was identical to a struct in the `metadata` package. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Arianna Vespri	b65021d7a5	Get conditional right Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Arianna Vespri	9fb1e9715c	Go on adding protobuf parsing for unit Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Arianna Vespri	5d5acf5744	Add unit protobuf parser Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	2 years ago
Arthur Silva Sens	5082655392	Append Created Timestamps (#12733 ) * Append created timestamps. Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Log when created timestamps are ignored Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Proposed changes to Append CT PR. Changes: * Changed textparse Parser interface for consistency and robustness. * Changed CT interface to be more explicit and handle validation. * Simplified test, change scrapeManager to allow testability. * Added TODOs. Signed-off-by: bwplotka <bwplotka@gmail.com> * Updates. Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * Refactor head_appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Fix linter issues Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Use model.Sample in head appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> --------- Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: bwplotka <bwplotka@gmail.com>	2 years ago
Matthieu MOREL	9c4782f1cc	golangci-lint: enable testifylint linter (#13254 ) Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
zenador	32ee1b15de	Fix error on ingesting out-of-order exemplars (#13021 ) Fix and improve ingesting exemplars for native histograms. See code comment for a detailed explanation of the algorithm. Note that this changes the current behavior for all kind of samples slightly: We now allow exemplars with the same timestamp as during the last scrape if the value or the labels have changed. Also note that we now do not ingest exemplars without timestamps for native histograms anymore. Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Björn Rabenstein <github@rabenste.in> --------- Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Signed-off-by: zenador <zenador@users.noreply.github.com> Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2 years ago
Matthieu MOREL	fe057fc60d	use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
beorn7	b428416f06	textparse: Update comment about timestamp_ms protobuf parsing By now, we know better what the plan is. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Arthur Silva Sens	ef8e6ae780	Parse created timestamps from Prometheus Protobuf (#12973 ) Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>	2 years ago
Matthieu MOREL	0a513f827d	ci(lint): enable errorlint linter on model Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
beorn7	65ccf4460a	textparse: Fix endless loop #12731 PR #12557 introduced the possibility of parsing multiple exemplars per native histograms. It did so by requiring the `Exemplar` method of the parser to be called repeatedly until it returns false. However, the protobuf parser code wasn't correctly updated for the old case of a single exemplar for a classic bucket (if actually parsed as a classic bucket) and a single exemplar on a counter. In those cases, the method would return `true` forever, yielding the same exemplar again and again, leading to an endless loop. With this fix, the state is now tracked and the single exemplar is only returned once. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	3d9a830f2f	textparse: Expose #12731 in protobufparse_test.go Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
György Krajcsovits	e846736134	Fix typo Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
beorn7	c58e20ad0e	histogram: Identify native histograms even without observations Native histograms without observations and with a zero threshold of zero look the same as classic histograms in the protobuf exposition format. According to https://github.com/prometheus/client_golang/issues/1127 , the idea is to add a no-op span to those histograms to mark them as native histograms. This commit enables Prometheus to detect that no-op span and adds a doc comment to the proto spec describing the behavior. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	071f4bbea4	histograms: Fix parsing float histograms without zero bucket If a float histogram has a zero bucket with a threshold of zero _and_ an empty zero bucket, it wasn't identified as a native histogram because the `isNativeHistogram` helper function only looked at integer buckets. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	2ea8df4734	histogram: Expose #12305 Native histograms without a zero threshold aren't federated properly. This adds a test to prove the specific failure mode, which is that histograms with a zero threshold of zero are federated as classic histograms. The underlying reason is that the protobuf parser identifies a native histogram by detecting a zero bucket or by detecting integer buckets. Therefore, a float histogram with a zero threshold of zero and an unpopulated zero bucket falls through the cracks (no integer buckets, no zero bucket). This commit also addse a test case for the latter. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	0e3f35324b	scrape: Enable ingestion of multiple exemplars per sample This has become a requirement for native histograms, as a single histogram sample commonly has many buckets, so that providing many exemplars makes sense. Since OM text doesn't support native histograms yet, the test had to be expanded to also support protobuf test cases. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	da047c6857	histograms: Fix bug #12552 The problem was the following: When trying to parse native histograms and classic histograms in parallel, the parser would first parse the histogram proto messages as a native histogram and then parse the same message again, but now as a classic histogram. Afterwards, it would forget that it was dealing with a metric family that contains native histograms and would parse the rest of the metric family as classic histograms only. The fix is to check again after being done with a classic histogram. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	1c3bd04bea	histograms: Modify test to expose bug #12552 Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	9e500345f3	textparse/scrape: Add option to scrape both classic and native histograms So far, if a target exposes a histogram with both classic and native buckets, a native-histogram enabled Prometheus would ignore the classic buckets. With the new scrape config option `scrape_classic_histograms` set, both buckets will be ingested, creating all the series of a classic histogram in parallel to the native histogram series. For example, a histogram `foo` would create a native histogram series `foo` and classic series called `foo_sum`, `foo_count`, and `foo_bucket`. This feature can be used in a migration strategy from classic to native histograms, where it is desired to have a transition period during which both native and classic histograms are present. Note that two bugs in classic histogram parsing were found and fixed as a byproduct of testing the new feature: 1. Series created from classic _gauge_ histograms didn't get the _sum/_count/_bucket prefix set. 2. Values of classic _float_ histograms weren't parsed properly. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago

1 2

91 Commits (b8867f8eada6f604365eb62f416e6d95cd2cb07f)