prometheus

Commit Graph

Author	SHA1	Message	Date
Julien Pivotto	a5671a002f	API: Add a /api/v1/features endpoint Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>	4 weeks ago
Clark McCauley	9d508a4888	util: add +Inf bucket in MetricFamiliesToWriteRequest when not present (#15864 ) * Add +Inf bucket in MetricFamiliesToWriteRequest if not present Signed-off-by: Clark McCauley <clarkmccauley@gmail.com>	2 months ago
Ben Kochie	204249fcb5	Update golangci-lint (#17478 ) * Update golangci-lint to v2.6.0 * Fixup various linting issues. * Fixup deprecations. * Add exception for `labels.MetricName` deprecation. Signed-off-by: SuperQ <superq@gmail.com>	2 months ago
Ben Kochie	48956f60d7	Update modernize (#17471 ) Apply additional Go modernize tool improvements. Signed-off-by: SuperQ <superq@gmail.com>	2 months ago
Linas Medžiūnas	44df626620	promql (histograms): reconcile mismatched NHCB bounds (#17278 ) Fixes #17255. The implementation happens mostly in the Add and Sub method, but the reconciliation works for all relevant operations. For example, you can now `rate` over a range wherein the custom bucket boundaries are changing. Any custom bucket reconciliation is flagged with an info-level annotation. --------- Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com> Signed-off-by: Linas Medžiūnas <linasm@users.noreply.github.com>	3 months ago
beorn7	ad7d1aed99	Phase out native histogram feature flag The detailed plan for this is laid out in https://github.com/prometheus/prometheus/issues/16572 . This commit adds a global and local scrape config option `scrape_native_histograms`, which has to be set to true to ingest native histograms. To ease the transition, the feature flag is changed to simply set the default of `scrape_native_histograms` to true. Further implications: - The default scrape protocols now depend on the `scrape_native_histograms` setting. - Everywhere else, histograms are now "on by default". Documentation beyond the one for the feature flag and the scrape config are deliberately left out. See https://github.com/prometheus/prometheus/pull/17232 for that. Signed-off-by: beorn7 <beorn@grafana.com>	3 months ago
György Krajcsovits	d11ee103ac	perf(tsdb): reuse map of sample types to speed up head appender While investigating +10% CPU in v3.7 release, found that ~5% is from expanding the types map. Try reuse. Also fix a linter error. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	3 months ago
György Krajcsovits	c26a5390aa	perf(tsdb): reuse map of sample types to speed up head appender While investigating +10% CPU in v3.7 release, found that ~5% is from expanding the types map. Try reuse. Also fix a linter error. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	3 months ago
Jan Fajerski	c9e0e36701	Add comments clarifying why promql.Querylogger exists (#17231 ) And why we only have one implementation in this code base. Fixes: https://github.com/prometheus/prometheus/issues/15869 Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	3 months ago
machine424	365409d3be	chore: allow seamless use of testing/synctest for >=go1.24 Signed-off-by: machine424 <ayoubmrini424@gmail.com>	4 months ago
beorn7	2628320292	promql: Fix when to emit a `HistogramCounterResetCollisionWarning` So far, we emitted a `HistogramCounterResetCollisionWarning` when encountering conflicting counter resets in the calculation of (i)rate and friends. We even tested for that. However, in the rate calculation, we are not interested in those collisions. They are actually expected. On the other hand, we did not warn about those collisions when doing a `sum` aggregation, where such a warning would be appropriate. This commit removes the warning in the former case and adds it in the latter. Sadly, we cannot really test this as we still remove the counter reset hint for the first sample in a chunk. (And that's the only sample where we could get a `NotCounterReset` hint.) Signed-off-by: beorn7 <beorn@grafana.com>	4 months ago
Owen Williams	6ee965c255	common: Update to prom/common v0.66.0, fix TextParser creation (#17139 ) TextParser as of prom/common v0.66.0 requires an explicit validation scheme. Signed-off-by: Owen Williams <owen.williams@grafana.com>	4 months ago
beorn7	747c5ee2b1	Apply analyzer "modernize" to the whole codebase See https://pkg.go.dev/golang.org/x/tools/gopls/internal/analysis/modernize for details. This ran into a few issues (arguably bugs in the modernize tool), which I will fix in the next commit, so that we have transparency what was done automatically. Beyond those hiccups, I believe all the changes applied are legitimate. Even where there might be no tangible direct gain, I would argue it's still better to use the "modern" way to avoid micro discussions in tiny style PRs later. Signed-off-by: beorn7 <beorn@grafana.com>	4 months ago
Julius Hinze	cdf7208478	annotations: histogram counter reset warning includes operation Signed-off-by: Julius Hinze <julius.hinze@grafana.com>	5 months ago
Julius Hinze	77b5c3f217	Histograms: set annotation when adding or subtracting histograms that have `not_reset` and `reset` hints. Signed-off-by: Julius Hinze <julius.hinze@grafana.com>	5 months ago
Arve Knudsen	0a40df33fb	Make metric/label name validation scheme explicit (#16928 ) * Parameterize metric/label name validation scheme Parameterized metric/label name validation scheme --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Julius Hinze <julius.hinze@grafana.com>	5 months ago
Neeraj Gartia	2c0de4e7c2	Fix `histogram_quantile` annotation in range query when delayed name removal is disabled (#16794 ) Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	5 months ago
Matthieu MOREL	cef219c31c	chore: enable unused-receiver rule from revive Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	5 months ago
Carrie Edwards	44b0fbba1e	No info annotation for rate/increase when type is histogram (#16915 ) Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>	6 months ago
George Krajcsovits	5b7ff92d95	fix(promql): histogram_quantile and histogram_fraction NaN observed in native histogram (#16724 ) * fix(promql): histogram_quantile NaN observed in native histogram Fixes: #16578 See the issue for detailed explanation. When a histogram had only NaN observations and no normal observations, we returned 0 from the quantile, which is completely wrong. If there were normal observations but we went over them, we returned the upper bound of the existing buckets, however that contradicts expectations on histogram_fraction. Now we return NaN if the quantile is calculated to be over all normal observations, falling into NaNs (in a virtual +Inf bucket). We also return info level annotations if we see any NaN observations. The annotation calls out if we returned NaN or even if we took the virtual +Inf bucket into account. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * fix(promql): histogram_fraction NaN observed in native histogram Fixes: #16580 According to the specification we should not take NaN observations into account when calculating the fraction. This commit fixes that and adds an info level annotation to let the user know about this. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	7 months ago
Arthur Silva Sens	8d0a747237	PROM-39: Provide PromQL info annotations when rate()/increase() over series without `__type__`="counter" label (#16632 ) * Provide PromQL info annotations when rate()/increase() over series without counter label Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Address comments Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> --------- Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	7 months ago
Innokentii Konstantinov	cd5bd11bd8	Export QuerySamples StartTimestamp and Interval to enable custom statistics implementations to populate these fields as needed. Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com>	7 months ago
Julius Volz	13564c03ef	Standardize doc page title handling See https://groups.google.com/g/prometheus-developers/c/cwL3cM66Em8 Signed-off-by: Julius Volz <julius.volz@gmail.com>	7 months ago
jub0bs	4bc8df0f54	util/httputil: Always add Vary header in SetCORS Closes #15406 Signed-off-by: jub0bs <jcretel-infosec+github@protonmail.com>	8 months ago
Arve Knudsen	e7e3ab2824	Fix linting issues found by golangci-lint v2.0.2 (#16368 ) * Fix linting issues found by golangci-lint v2.0.2 --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	8 months ago
Arthur Silva Sens	95f49dd84b	Bump prometheus/common to v0.63.0 (#16210 ) * Bump prometheus/common to v0.63.0 Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * nolint usage of deprecated model.NameValidationScheme Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> --------- Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	10 months ago
Bartlomiej Plotka	7a7bc65237	Add util/compression package to consolidate snappy/zstd use in Prometheus. (#16156 ) # Conflicts: # tsdb/db_test.go Apply suggestions from code review tmp Addressed comments. Update util/compression/buffers.go Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>	10 months ago
Matthieu MOREL	c7d4b53ec1	chore: enable unused-parameter from revive Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	11 months ago
jub0bs	329ec6831a	util/httputil: reduce heap allocations in newCompressedResponseWriter (#16001 ) * util/httputil: Benchmark newCompressedResponseWriter This benchmark illustrates that newCompressedResponseWriter incurs a prohibitive amount of heap allocations when handling a request containing a malicious Accept-Encoding header.¬ Signed-off-by: jub0bs <jcretel-infosec+github@protonmail.com> * util/httputil: Improve newCompressedResponseWriter This change dramatically reduces the heap allocations (in bytes) incurred when handling a request containing a malicious Accept-Encoding header. Below are some benchmark results; for conciseness, I've omitted the name of the benchmark function (BenchmarkNewCompressionHandler_MaliciousAcceptEncoding): ``` goos: darwin goarch: amd64 pkg: github.com/prometheus/prometheus/util/httputil cpu: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz │ old │ new │ │ sec/op │ sec/op vs base │ 18.60m ± 2% 13.54m ± 3% -27.17% (p=0.000 n=10) │ old │ new │ │ B/op │ B/op vs base │ 16785442.50 ± 0% 32.00 ± 0% -100.00% (p=0.000 n=10) │ old │ new │ │ allocs/op │ allocs/op vs base │ 2.000 ± 0% 1.000 ± 0% -50.00% (p=0.000 n=10) ``` Signed-off-by: jub0bs <jcretel-infosec+github@protonmail.com> --------- Signed-off-by: jub0bs <jcretel-infosec+github@protonmail.com>	11 months ago
Neeraj Gartia	8be416a67c	[FIX] PromQL: Updates annotation for bin op between incompatible histograms (#15895 ) PromQL: Updates annotation for bin op between incompatible histograms --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	11 months ago
crystalstall	616914abe2	Signed-off-by: crystalstall <crystalruby@qq.com> refactor: using slices.Contains to simplify the code Signed-off-by: crystalstall <crystalruby@qq.com>	1 year ago
Neeraj Gartia	0e99ca3e8c	[BUGFIX] PromQL: Fix `deriv`, `predict_linear` and `double_exponential_smoothing` with histograms (#15686 ) PromQL: Fix deriv, predict_linear and double_exponential_smoothing with histograms Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	1 year ago
Joel Beckmeyer	bad1c75514	fix alignment of atomic uint64 on 32-bit Signed-off-by: Joel Beckmeyer <joel@beckmeyer.us>	1 year ago
TJ Hoplock	437362a7a7	fix(deduper): use ptr to sync.RWMutex, fix panic during concurrent use Resolves: #15559 As accurately noted in the issue description, the map is shared among child loggers that get created when `WithAttr()`/`WithGroup()` are called on the underlying handler, which happens via `log.With()` and `log.WithGroup()` respectively. The RW mutex was a value in the previous implementation that used go-kit/log, and I should've updated it to use a pointer when I converted the deduper. Also adds a test. Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	1 year ago
TJ Hoplock	e0104a6b7e	ref: JSONFileLogger slog handler, add scrape.FailureLogger interface Improves upon #15434, better resolves #15433. This commit introduces a few changes to ensure safer handling of the JSONFileLogger: - the JSONFileLogger struct now implements the slog.Handler interface, so it can directly be used to create slog Loggers. This pattern more closely aligns with upstream slog usage (which is generally based around handlers), as well as making it clear that devs are creating a whole new logger when interacting with it (vs silently modifying internal configs like it did previously). - updates the `promql.QueryLogger` interface to be a union of the methods of both the `io.Closer` interface and the `slog.Handler` interface. This allows for plugging in/using slog-compatible loggers other than the JSONFileLogger, if desired (ie, for downstream projects). - introduces new `scrape.FailureLogger` interface; just like `promql.QueryLogger`, it is a union of `io.Closer` and `slog.Handler` interfaces. Similar logic applies to reasoning. - updates tests where needed; have the `FakeQueryLogger` from promql's engine_test implement the `slog.Handler`, improve JSONFileLogger test suite, etc. Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	1 year ago
György Krajcsovits	eb9ce70c3e	Set hasCount after setting count to be consistent Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
György Krajcsovits	a48d05912d	nhcb: optimize, do not recalculate suffixes multiple times Reduce string manipulation by just cutting off the histogram suffixes from the series name label once. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
György Krajcsovits	41f051e9a4	Small improvement in handling cases without count Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
TJ Hoplock	3e24e84172	fix!: stop unbounded memory usage from query log Resolves: #15433 When I converted prometheus to use slog in #14906, I update both the `QueryLogger` interface, as well as how the log calls to the `QueryLogger` were built up in `promql.Engine.exec()`. The backing logger for the `QueryLogger` in the engine is a `util/logging.JSONFileLogger`, and it's implementation of the `With()` method updates the logger the logger in place with the new keyvals added onto the underlying slog.Logger, which means they get inherited onto everything after. All subsequent calls to `With()`, even in later queries, would continue to then append on more and more keyvals for the various params and fields built up in the logger. In turn, this causes unbounded growth of the logger, leading to increased memory usage, and in at least one report was the likely cause of an OOM kill. More information can be found in the issue and the linked slack thread. This commit does a few things: - It was referenced in feedback in #14906 that it would've been better to not change the `QueryLogger` interface if possible, this PR proposes changes that bring it closer to alignment with the pre-3.0 `QueryLogger` interface contract - reverts `promql.Engine.exec()`'s usage of the query logger to the pattern of building up an array of args to pass at once to the end log call. Avoiding the repetitious calls to `.With()` are what resolve the issue with the logger growth/memory usage. - updates the scrape failure logger to use the update `QueryLogger` methods in the contract. - updates tests accordingly - cleans up unused methods Builds and passes tests successfully. Tested locally and confirmed I could no longer reproduce the issue/it resolved the issue. Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	1 year ago
TJ Hoplock	16a4354b48	chore(deduper): add compile check that slog.Handler int is satisfied Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	1 year ago
Neeraj Gartia	789c9b1a5e	[BUGFIX] PromQL: Corrects the behaviour of some operator and aggregators with Native Histograms (#15245 ) PromQL: Correct the behaviour of some operator and aggregators with Native Histograms --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	1 year ago
Julien	5c05c89b8b	Update prometheus/common Signed-off-by: Julien <roidelapluie@o11y.eu>	1 year ago
Julien	28f22b4d80	Update prometheus/common Signed-off-by: Julien <roidelapluie@o11y.eu>	1 year ago
György Krajcsovits	eafe72a0d0	perf(nhcb): optimize away most allocations in convertnhcb In general aim for the happy case when the exposer lists the buckets in ascending order. Use Compact(2) to compact the result of nhcb convert. This is more in line with how client_golang optimizes spans vs buckets. `aef8aedb4b/prometheus/histogram.go (L1485)` Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
TJ Hoplock	4f9e4dc016	ref: remove unused deduper log wrapper methods I used these wrapper methods during initial development of the custom handler that the deduper now implements. Since the deduper implements slog.Handler and can be used directly as a logger, these wrapper methods are no longer needed. Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	1 year ago
TJ Hoplock	b602393473	fix: avoid data race in log deduper This change should have been included in the initial prometheus slog conversion, but I must've lost track of it in all the rebases involved in that PR. This changes the dedupe logger so that the only method that needs to use the lock is the `Handle()` method that actually interacts with the deduplication map. Ex: ``` ================== WARNING: DATA RACE Write at 0x00c000518bc0 by goroutine 29481: github.com/prometheus/prometheus/util/logging.(Deduper).WithAttrs() /home/tjhop/go/src/github.com/prometheus/prometheus/util/logging/dedupe.go:89 +0xef log/slog.(Logger).With() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/log/slog/logger.go:132 +0x106 github.com/prometheus/prometheus/storage/remote.NewQueueManager() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/queue_manager.go:483 +0x7a9 github.com/prometheus/prometheus/storage/remote.(WriteStorage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/write.go:201 +0x102c github.com/prometheus/prometheus/storage/remote.(Storage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:92 +0xfd github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.func1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:172 +0x3e4 github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.gowrap1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:174 +0x41 Previous read at 0x00c000518bc0 by goroutine 31261: github.com/prometheus/prometheus/util/logging.(Deduper).Handle() /home/tjhop/go/src/github.com/prometheus/prometheus/util/logging/dedupe.go:82 +0x2b1 log/slog.(Logger).log() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/log/slog/logger.go:257 +0x228 log/slog.(Logger).Error() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/log/slog/logger.go:230 +0x3d4 github.com/prometheus/prometheus/tsdb/wlog.(Watcher).loop() /home/tjhop/go/src/github.com/prometheus/prometheus/tsdb/wlog/watcher.go:254 +0x2db github.com/prometheus/prometheus/tsdb/wlog.(Watcher).Start.gowrap1() /home/tjhop/go/src/github.com/prometheus/prometheus/tsdb/wlog/watcher.go:227 +0x33 Goroutine 29481 (running) created at: github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:164 +0xe4 testing.tRunner() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/testing/testing.go:1690 +0x226 testing.(T).Run.gowrap1() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/testing/testing.go:1743 +0x44 Goroutine 31261 (running) created at: github.com/prometheus/prometheus/tsdb/wlog.(Watcher).Start() /home/tjhop/go/src/github.com/prometheus/prometheus/tsdb/wlog/watcher.go:227 +0x177 github.com/prometheus/prometheus/storage/remote.(QueueManager).Start() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/queue_manager.go:934 +0x304 github.com/prometheus/prometheus/storage/remote.(WriteStorage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/write.go:232 +0x151b github.com/prometheus/prometheus/storage/remote.(Storage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:92 +0xfd github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.func1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:172 +0x3e4 github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.gowrap1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:174 +0x41 ================== --- FAIL: TestWriteStorageApplyConfigsDuringCommit (2.26s) testing.go:1399: race detected during execution of test FAIL FAIL github.com/prometheus/prometheus/storage/remote 68.321s ``` Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	1 year ago
Vanshika	cccbe72514	TSDB: Fix some edge cases when OOO is enabled (#14710 ) Fix some edge cases when OOO is enabled Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> Signed-off-by: Vanshika <102902652+Vanshikav123@users.noreply.github.com> Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com> Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>	1 year ago
Neeraj Gartia	d4b1f9eb33	Corrects the behaviour of binary opperators between histogram and float (#14726 ) promql: corrects binary operators functioning for mixed sample with histogram and float For invalid pairings of sample types, an annotation is added now. Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	1 year ago
György Krajcsovits	78de9bd10f	convertnhcb: use CutSuffix instead of regex replace for histogram name This is much quicker. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
George Krajcsovits	522149a2ae	model: move classic NHCB conversion into its own file (#15156 ) * model: move classic to NHCB conversion into its own file In preparation for #14978. Author: Jeanette Tan <jeanette.tan@grafana.com> 2024-07-03 11:56:48 Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Jeanette Tan <jeanette.tan@grafana.com> Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Better naming from review comment Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Add doc strings. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> --------- Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Jeanette Tan <jeanette.tan@grafana.com>	1 year ago

1 2 3 4 5 ...

310 Commits (3698cd044c33be89fcc0dfdcb1b279f9d30c8d53)