prometheus

Commit Graph

Author	SHA1	Message	Date
Artur Melanchyk	30112f6ed7	promtool: optimize labels slice allocation Signed-off-by: Artur Melanchyk <arturmelanchyk@imail.name>	11 months ago
Matthieu MOREL	dd5ab743ea	chore(deps): use version.PrometheusUserAgent Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	11 months ago
Giedrius Statkevičius	92218ecb9b	promtool: add --ignore-unknown-fields Add --ignore-unknown-fields that ignores unknown fields in rule group files. There are lots of tools in the ecosystem that "like" to extend the rule group file structure but they are currently unreadable by promtool if there's anything extra. The purpose of this flag is so that we could use the "vanilla" promtool instead of rolling our own. Some examples of tools/code: https://github.com/grafana/mimir/blob/main/pkg/mimirtool/rules/rwrulefmt/rulefmt.go `8898eb3cc5/pkg/rules/rules.go (L18-L25)` Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	11 months ago
Arve Knudsen	5df6ea3042	promtool: Support linting of scrape interval (#15719 ) * PromTool: Support Scrape Interval Lint Checking --------- Signed-off-by: zhaowang <zhaowang@apac.freewheel.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: zhaowang <zhaowang@apac.freewheel.com>	11 months ago
sh0rez	5303e515af	remote/otlp: convert delta to cumulative (#15165 ) What Adds support for OTLP delta temporality to the OTLP endpoint. This is done by calling the deltatocumulative processor from the OpenTelemetry collector during OTLP conversion. Why Delta conversion is a naturally stateful process, which requires careful request routing when operated inside a collector. Prometheus is already stateful and doing the conversion in-server reduces the operational burden on the ingest architecture by only having one stateful component. How deltatocumulative is a OTel collector component that works as follows: * pmetric.Metrics come from a receiver or in this case from the HTTP client * It operates as an in-place update loop: * for each sample, if not delta, leave unmodified * if delta, do: * state += sample, where state is the in-memory sum of all previous samples * sample = state, sample value is now cumulative * this is supported for sums (counters), gauges, histograms (old histograms) and exponential histograms (native histograms) If a series receives no new samples for 5m, its state is removed from memory Performance Delta performance is a stateful operation and the OTel code is not highly optimized yet, e.g. it locks the entire processor for each request. Nonetheless, care has been taken to mitigate those effects: delta conversion is behind a feature flag. If disabled, no conversion code is ever invoked if enabled, conversion is not invoked if request not actually contains delta samples. This leads to no measureable performance difference between default-cumulative to convert-cumulative (only cumulative, feature on/off) Signed-off-by: sh0rez <me@shorez.de>	12 months ago
Arve Knudsen	f030894c2c	Fix issues raised by staticcheck (#15722 ) Fix issues raised by staticcheck We are not enabling staticcheck explicitly, though, because it has too many false positives. --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	12 months ago
machine424	9823a93c42	fix(main.go): avoid closing the query engine until it is guaranteed to no longer be in use. partially reverts https://github.com/prometheus/prometheus/pull/14064 fixes https://github.com/prometheus/prometheus/issues/15232 supersedes https://github.com/prometheus/prometheus/pull/15533 reusing Engine.Close() outside of tests will require more consideration. Signed-off-by: machine424 <ayoubmrini424@gmail.com>	1 year ago
Bryan Boreham	7b03796d0f	Scraping: stop storing discovered labels (#15261 ) Instead of storing discovered labels on every target, recompute them if required. The `Target` struct now needs to hold some more data required to recompute them, such as ScrapeConfig. This moves the load from every Prometheus all of the time, to just when someone views Service Discovery in the UI. The way `PopulateLabels` is used changes; you are no longer expected to call it with a part-populated `labels.Builder`. The signature of `Target.Labels` changes to take a `labels.Builder` instead of a `ScratchBuilder`, for consistency with `DiscoveredLabels`. This will save a lot of work when many targets are filtered out in relabeling. Combine with `keep_dropped_targets` to avoid ever computing most labels for dropped targets. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Arthur Silva Sens	3b97a6397c	Put PRWv2 created timestamp ingestion behing feature-flag Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	1 year ago
bwplotka	f44bba31b3	config: Remove validation GetScrapeConfigs; require using config.Load. We should never modify (or even shallow copy) Config after config.Load; added comments and modified GetScrapeConfigs to do so. For GetScrapeConfigs the validation (even repeated) was likely doing writes (because global fields was 0). We GetScrapeConfigs concurrently in tests and ApplyConfig causing test races. In prod there were races but likelyt only to replace 0 with 0, so not too severe. I removed validation since I don't see anyone using our config.Config without Load. I had to refactor one test that was doing it, all others use yaml config. Fixes #15538 Previous attempt: https://github.com/prometheus/prometheus/pull/15634 Signed-off-by: bwplotka <bwplotka@gmail.com>	1 year ago
Ben Ye	40c3ba9fd4	fix promtool analyze block shows metric name with 0 cardinality (#15438 ) Signed-off-by: Ben Ye <benye@amazon.com>	1 year ago
Ben Kochie	869addfec8	Enable auto-gomaxprocs by default (#15378 ) Enable the `auto-gomaxprocs` feature flag by default. * Add command line flag `--no-auto-gomaxprocs` to disable. Signed-off-by: SuperQ <superq@gmail.com>	1 year ago
Julien	b93aec077c	Fix auto reload when a config file with a syntax error is reverted When we had a syntax error but restored the old file, we did not re-trigger the config reload, so the config reload metric was showing that config reload was unsucessful. I made magic to handle logs in cmd/prometheus. For now it is a separate file so we can backport this easily. I will generalize the helper in another PR. Signed-off-by: Julien <roidelapluie@o11y.eu>	1 year ago
Ben Kochie	0f0deb77a2	Enable auto-gomemlimit by default (#15372 ) Enable the `auto-gomemlimit` feature flag by default. * Add command line flag `--no-auto-gomemlimit` to disable. Signed-off-by: SuperQ <superq@gmail.com>	1 year ago
Ben Kochie	f701f4e236	Enable auto-gomaxprocs by default (#15376 ) Enable the `auto-gomaxprocs` feature flag by default. * Add command line flag `--no-auto-gomaxprocs` to disable. Signed-off-by: SuperQ <superq@gmail.com>	1 year ago
Ben Kochie	fb69a38e67	Enable auto-gomemlimit by default (#15373 ) * Enable auto-gomemlimit by default Enable the `auto-gomemlimit` feature flag by default. * Add command line flag `--no-auto-gomemlimit` to disable. --------- Signed-off-by: SuperQ <superq@gmail.com>	1 year ago
Bryan Boreham	de35a40ed5	[REFACTOR] Small improvement to top-level naming Previously, top-level initialization (including WAL reading) had this at the top of the stack: github.com/oklog/run.(*Group).Run.func1:38 I found this confusing, and thought it had something to do with logging. Introducing another local function should make this clearer. Also make the error message user-centric not code-centric. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Ben Ye	140f4aa9ae	feat: Allow customizing TSDB postings decoder (#13567 ) * allow customizing TSDB postings decoder --------- Signed-off-by: Ben Ye <benye@amazon.com>	1 year ago
Aniket Kaulavkar	650abaccce	Parallelize tests in cmd/promtool/ Signed-off-by: Aniket Kaulavkar <aniket.kaulavkar@gmail.com>	1 year ago
🌲 Harry 🌊 John 🏔	f9bc50b247	storage: Implement limit in mergeGenericQuerier Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>	1 year ago
Matthieu MOREL	af1a19fc78	enable errorf rule from perfsprint linter Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	1 year ago
Oleg Zaytsev	50e83e8f98	Export 'go_sync_mutex_wait_total_seconds_total' metric This metric can be useful to debug mutex contention. Ref: https://github.com/golang/go/issues/49881 Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	1 year ago
Julien	9d9e86b333	Fix auto reload when a config file with a syntax error is reverted When we had a syntax error but restored the old file, we did not re-trigger the config reload, so the config reload metric was showing that config reload was unsucessful. I made magic to handle logs in cmd/prometheus. For now it is a separate file so we can backport this easily. I will generalize the helper in another PR. Signed-off-by: Julien <roidelapluie@o11y.eu>	1 year ago
Arthur Silva Sens	88818c9cb3	chore: Remove dead code Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	1 year ago
Alban Hurtaud	4b56af7eb8	Add hidden flag for the delayed compaction random time window (#14919 ) * Add hidden flag for the delayed compaction random time window Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Update cmd/prometheus/main.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> * Update cmd/prometheus/main.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> * Update tsdb/db.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> * Fix flag name according to review - add test for delay Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Fix afer main rebase Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Implement review comments Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Update generatedelaytest to try with limit values Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> --------- Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>	1 year ago
Vanshika	cccbe72514	TSDB: Fix some edge cases when OOO is enabled (#14710 ) Fix some edge cases when OOO is enabled Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> Signed-off-by: Vanshika <102902652+Vanshikav123@users.noreply.github.com> Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com> Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>	1 year ago
George Krajcsovits	1b4e7f74e6	feat(tools): add debug printouts to rules unit testing (#15196 ) * promtool: Add debug flag for rule tests This makes it print out the tsdb state (both input_series and rules that are run) at the end of a test, making reasoning about tests much easier. Signed-off-by: David Leadbeater <dgl@dgl.cx> * Reuse generated test name from junit testing Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> --------- Signed-off-by: David Leadbeater <dgl@dgl.cx> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: David Leadbeater <dgl@dgl.cx>	1 year ago
machine424	cf128a0472	test(cmd/prometheus): speed up test execution by t.Parallel() when possible turn some loops into subtests to make use of t.Parallel() requires Go 1.22 to make use of https://go.dev/blog/loopvar-preview Signed-off-by: machine424 <ayoubmrini424@gmail.com>	1 year ago
Alan Protasio	c78d5b94af	Disallowing configure AM with the v1 api (#13883 ) * Stop supporting Alertmanager v1 * Disallowing configure AM with the v1 api Signed-off-by: alanprot <alanprot@gmail.com> * Update config/config_test.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> * Update config/config.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> * Addressing coments Signed-off-by: alanprot <alanprot@gmail.com> * Update notifier/notifier.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> * Update config/config_test.go Co-authored-by: Jan Fajerski <jan--f@users.noreply.github.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> --------- Signed-off-by: alanprot <alanprot@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Co-authored-by: Jan Fajerski <jan--f@users.noreply.github.com>	1 year ago
Yi	2cabd1b707	config: remove expand-external-labels flag in release 3.0 (#14657 ) remove expand-external-labels feature flag and enabled env arg expansion for external labels by default. Signed-off-by: jyz0309 <45495947@qq.com>	1 year ago
TJ Hoplock	6ebfbd2d54	chore!: adopt log/slog, remove go-kit/log For: #14355 This commit updates Prometheus to adopt stdlib's log/slog package in favor of go-kit/log. As part of converting to use slog, several other related changes are required to get prometheus working, including: - removed unused logging util func `RateLimit()` - forward ported the util/logging/Deduper logging by implementing a small custom slog.Handler that does the deduping before chaining log calls to the underlying real slog.Logger - move some of the json file logging functionality to use prom/common package functionality - refactored some of the new json file logging for scraping - changes to promql.QueryLogger interface to swap out logging methods for relevant slog sugar wrappers - updated lots of tests that used/replicated custom logging functionality, attempting to keep the logical goal of the tests consistent after the transition - added a healthy amount of `if logger == nil { $makeLogger }` type conditional checks amongst various functions where none were provided -- old code that used the go-kit/log.Logger interface had several places where there were nil references when trying to use functions like `With()` to add keyvals on the new *slog.Logger type Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	1 year ago
Matthieu MOREL	ab64966e9d	fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" (#15094 ) * fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" --------- Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	1 year ago
Ben Ye	6c3d11629b	add missing flag storage.tsdb.allow-overlapping-compaction Signed-off-by: Ben Ye <benye@amazon.com>	1 year ago
Julien	9d275c23de	cmd/prometheus: Fix flakiness of QueryLogTest Now we check that a rule execution has taken place. This also reduces the time to run the rules tests from 45s to 25s. Signed-off-by: Julien <roidelapluie@o11y.eu>	1 year ago
Julien	21e0f83b68	Move notifications in utils Signed-off-by: Julien <roidelapluie@o11y.eu>	1 year ago
Julien	b6158e8956	Notify web UI when starting up and shutting down Signed-off-by: Julien <roidelapluie@o11y.eu>	1 year ago
Jesus Vazquez	77d3b3aff3	OTLP: Remove experimental word form OTLP receiver (#14894 ) The OTLP receiver can now considered stable. We've had it for longer than a year in main and has received constant improvements. Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>	1 year ago
Bryan Boreham	6b247c50d2	Revert "Merge pull request #14769 from roidelapluie/autoreload" This reverts commit `50f5327f83`, reversing changes made to `eb4004c344`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Julien	f9bbad1148	Limit the number of SSE Subscribers to 16 by default Signed-off-by: Julien <roidelapluie@o11y.eu>	1 year ago
Julien	6cde0096e2	Add notifications to the web UI when configuration reload fails. This commit introduces a new `/api/v1/notifications/live` endpoint that utilizes Server-Sent Events (SSE) to stream notifications to the web UI. This is used to display alerts such as when a configuration reload has failed. I opted for SSE over WebSockets because SSE is simpler to implement and more robust for our use case. Since we only need one-way communication from the server to the client, SSE fits perfectly without the overhead of establishing and maintaining a two-way WebSocket connection. When the SSE connection fails, we go back to a classic /api/v1/notifications API endpoint. This commit also contains the required UI changes for the new Mantine UI. Signed-off-by: Julien <roidelapluie@o11y.eu>	1 year ago
Bryan Boreham	410fcce6f0	Remove unnecessary pprof import (#14988 ) The pattern of `import _ "net/http/pprof"` adds handlers to the default http handler, but Prometheus does not use that. There are explicit handlers in `web/web.go`. So, we can remove this line with no impact to behaviour. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Arthur Silva Sens	6bd9b1a7cc	Histogram CT Zero ingestion Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	1 year ago
Alex Johnson	be0f10054e	Remove no-default-scrape-port featureFlag Signed-off-by: Alex Johnson <alex.kattathra.johnson@gmail.com>	1 year ago
Julien	146b22d196	fix(autoreload): Reload invalid yaml files When a YAML file is invalid, trigger auto-reload anyway so that user is aware that the configuration file is incorrect. Failing to do so does not change the reload status in metrics and api. Signed-off-by: Julien <roidelapluie@o11y.eu>	1 year ago
Julien	ac5377873f	mantine UI: Distinguish between Not Ready and Stopping Signed-off-by: Julien <roidelapluie@o11y.eu>	1 year ago
Carrie Edwards	14e3c05ce8	tsdb: Add support for ingestion of out-of-order native histogram samples (#14546 ) Add support for ingesting OOO native histograms * Add flag for enabling and disabling OOO native histogram ingestion * Update OOO querying tests to include native histogram samples * Add OOO head tests * Add test for OOO native histogram counter reset headers Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored by: Carrie Edwards <edwrdscarrie@gmail.com> Co-authored by: Jeanette Tan <jeanette.tan@grafana.com> Co-authored by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored by: Fiona Liao <fiona.liao@grafana.com>	1 year ago
Augustin Husson	6febfbb3be	put back feature flag 'delayed-compaction' and 'old-ui' (#14909 ) Signed-off-by: Augustin Husson <husson.augustin@gmail.com>	1 year ago
Nathan Baulch	50cd453c8f	chore: Fix typos (#14868 ) * Fix typos --------- Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>	1 year ago
Owen Williams	88bb05c3e8	utf8: enable utf-8 support by default This change causes Prometheus to allow all UTF-8 characters in metric and label names. This means that names that were previously invalid and would have been previously rejected will be allowed through. Signed-off-by: Owen Williams <owen.williams@grafana.com>	1 year ago
machine424	cc40b65ab4	fix(promtool): use the final database path for --sandbox-dir-root instead of the default value as it may be overridden add a regression test for that. Signed-off-by: machine424 <ayoubmrini424@gmail.com>	1 year ago

1 2 3 4 5 ...

880 Commits (bwplotka-patch-3)