prometheus

Commit Graph

Author	SHA1	Message	Date
beorn7	aa82fe198f	tsdb: Fix histogram validation So far, `ValidateHistogram` would not detect if the count did not include the count in the zero bucket. This commit fixes the problem and updates all the tests that have been undetected offenders so far. Note that this problem would only ever create false negatives, so we never falsely rejected to store a histogram because of it. On the other hand, `ValidateFloatHistogram` has been to strict with the count being at least as large as the sum of the counts in all the buckets. Float precision issues could create false positives here, see products of PromQL evaluations, it's actually quite hard to put an upper limit no the floating point imprecision. Users could produce the weirdest expressions, maxing out float precision problems. Therefore, this commit simply removes that particular check from `ValidateFloatHistogram`. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Michael Hoffmann	4d8e380269	promql: allow tests to be imported (#12050 ) Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>	2 years ago
Bryan Boreham	1e3fef6ab0	scraping: limit detail on dropped targets, to save memory (#12647 ) It's possible (quite common on Kubernetes) to have a service discovery return thousands of targets then drop most of them in relabel rules. The main place this data is used is to display in the web UI, where you don't want thousands of lines of display. The new limit is `keep_dropped_targets`, which defaults to 0 for backwards-compatibility. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Goutham Veeramachaneni	ad4f514e66	Add OTLP Ingestion endpoint (#12571 ) * Add OTLP Ingestion endpoint We copy files from the otel-collector-contrib. See the README in `storage/remote/otlptranslator/README.md`. This supersedes: https://github.com/prometheus/prometheus/pull/11965 Signed-off-by: gouthamve <gouthamve@gmail.com> * Return a 200 OK It is what the OTEL Golang SDK expect :( https://github.com/open-telemetry/opentelemetry-go/issues/4363 Signed-off-by: Goutham <gouthamve@gmail.com> --------- Signed-off-by: gouthamve <gouthamve@gmail.com> Signed-off-by: Goutham <gouthamve@gmail.com>	2 years ago
Julien Pivotto	16c645a6dd	Release 2.46.0 Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
Bryan Boreham	dcadb32eb1	web/api: use stream encoder for embedded labels This is much more efficient. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	bb528d4a55	Add jsoniter encoder for Labels Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	54e1046616	web/api: extend BenchmarkRespond with more types of data Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Julien Pivotto	c37af1eda5	Release 2.46.0-rc.0 Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
beorn7	2ea8df4734	histogram: Expose #12305 Native histograms without a zero threshold aren't federated properly. This adds a test to prove the specific failure mode, which is that histograms with a zero threshold of zero are federated as classic histograms. The underlying reason is that the protobuf parser identifies a native histogram by detecting a zero bucket or by detecting integer buckets. Therefore, a float histogram with a zero threshold of zero and an unpopulated zero bucket falls through the cracks (no integer buckets, no zero bucket). This commit also addse a test case for the latter. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Bryan Boreham	ce153e3fff	Replace sort.Sort with faster slices.SortFunc The generic version is more efficient. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Marco Pracucci	7cc4292328	Export MinTime and MaxTime Signed-off-by: Marco Pracucci <marco@pracucci.com>	3 years ago
Giedrius Statkevičius	3f230fc9f8	promql: convert QueryOpts to interface Convert QueryOpts to an interface so that downstream projects like https://github.com/thanos-community/promql-engine could extend the query options with engine specific options that are not in the original engine. Will be used to enable query analysis per-query. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	3 years ago
cui fliter	484a9e4071	fix some typos (#12498 ) Signed-off-by: cui fliter <imcusg@gmail.com>	3 years ago
Matthias Loibl	686482ab34	Remove Add(0) Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>	3 years ago
Jesus Vazquez	8ef767e396	Release 2.45.0 (#12486 ) Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>	3 years ago
Jesus Vazquez	c858049744	Create 2.45.0-rc.1 (#12478 ) Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>	3 years ago
Matthias Loibl	8bc2a19469	web: Initialize requestCounter metrics to 0 with handler and 200k labels. Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>	3 years ago
Arthur Silva Sens	1ea477f4bc	Add feature flag to squash metadata from /api/v1/metadata (#12391 ) Signed-off-by: ArthurSens <arthursens2005@gmail.com>	3 years ago
Jesus Vazquez	edfc97a77e	Bump UI version (#12440 ) Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>	3 years ago
Jesus Vazquez	bfa466d00f	Create release candidate 2.45.0-rc.0 (#12435 ) Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>	3 years ago
cui fliter	6e7ac76981	fix problematic link (#12405 ) Signed-off-by: cui fliter <imcusg@gmail.com>	3 years ago
Baskar Shanmugam	905a0bd63a	Added 'limit' query parameter support to /api/v1/status/tsdb endpoint (#12336 ) * Added 'topN' query parameter support to /api/v1/status/tsdb endpoint Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com> * Updated query parameter for tsdb status to 'limit' Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com> * Corrected Stats() parameter name from topN to limit Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com> * Fixed p.Stats CI failure Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com> --------- Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>	3 years ago
Bryan Boreham	1ac5131f69	Release 2.44.0 (#12364 ) Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
beorn7	9e500345f3	textparse/scrape: Add option to scrape both classic and native histograms So far, if a target exposes a histogram with both classic and native buckets, a native-histogram enabled Prometheus would ignore the classic buckets. With the new scrape config option `scrape_classic_histograms` set, both buckets will be ingested, creating all the series of a classic histogram in parallel to the native histogram series. For example, a histogram `foo` would create a native histogram series `foo` and classic series called `foo_sum`, `foo_count`, and `foo_bucket`. This feature can be used in a migration strategy from classic to native histograms, where it is desired to have a transition period during which both native and classic histograms are present. Note that two bugs in classic histogram parsing were found and fixed as a byproduct of testing the new feature: 1. Series created from classic _gauge_ histograms didn't get the _sum/_count/_bucket prefix set. 2. Values of classic _float_ histograms weren't parsed properly. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Bryan Boreham	94d9367bbf	Create 2.44.0-rc.2 (#12341 ) Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Bryan Boreham	e278195e39	Cherry-pick bugfix #12322 and create v2.43.1 (#12324 ) * labels: respect Set after Del in Builder (#12322) The implementations are not symmetric between `Set()` and `Del()`, so we must be careful. Add tests for this, both in labels and in relabel where the issue was reported. Also make the slice implementation consistent re `slices.Contains`. * Create v2.43.1 with bugfix Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Co-authored-by: Julius Volz <julius.volz@gmail.com>	3 years ago
Bryan Boreham	3d26faade4	Create 2.44.0-rc.1 (#12323 ) Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Bryan Boreham	aeccf9e770	Bump version to 2.44.0-rc0 Including CHANGELOG. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Vladimir Varankin	d281ebb178	web: display GOMEMLIMIT in runtime info Signed-off-by: Vladimir Varankin <vladimir@varank.in>	3 years ago
gotjosh	e78be38cc0	don't show empty groups Signed-off-by: gotjosh <josue.abreu@gmail.com>	3 years ago
beorn7	5b53aa1108	style: Replace `else if` cascades with `switch` Wiser coders than myself have come to the conclusion that a `switch` statement is almost always superior to a statement that includes any `else if`. The exceptions that I have found in our codebase are just these two: * The `if else` is followed by an additional statement before the next condition (separated by a `;`). * The whole thing is within a `for` loop and `break` statements are used. In this case, using `switch` would require tagging the `for` loop, which probably tips the balance. Why are `switch` statements more readable? For one, fewer curly braces. But more importantly, the conditions all have the same alignment, so the whole thing follows the natural flow of going down a list of conditions. With `else if`, in contrast, all conditions but the first are "hidden" behind `} else if `, harder to spot and (for no good reason) presented differently from the first condition. I'm sure the aforemention wise coders can list even more reasons. In any case, I like it so much that I have found myself recommending it in code reviews. I would like to make it a habit in our code base, without making it a hard requirement that we would test on the CI. But for that, there has to be a role model, so this commit eliminates all `if else` occurrences, unless it is autogenerated code or fits one of the exceptions above. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	c3c7d44d84	lint: Adjust to the lint warnings raised by current versions of golint-ci We haven't updated golint-ci in our CI yet, but this commit prepares for that. There are a lot of new warnings, and it is mostly because the "revive" linter got updated. I agree with most of the new warnings, mostly around not naming unused function parameters (although it is justified in some cases for documentation purposes – while things like mocks are a good example where not naming the parameter is clearer). I'm pretty upset about the "empty block" warning to include `for` loops. It's such a common pattern to do something in the head of the `for` loop and then have an empty block. There is still an open issue about this: https://github.com/mgechev/revive/issues/810 I have disabled "revive" altogether in files where empty blocks are used excessively, and I have made the effort to add individual `// nolint:revive` where empty blocks are used just once or twice. It's borderline noisy, though, but let's go with it for now. I should mention that none of the "empty block" warnings for `for` loop bodies were legitimate. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
gotjosh	96b6463f25	review comments Signed-off-by: gotjosh <josue.abreu@gmail.com>	3 years ago
gotjosh	f3394bf7a1	Rules API: Allow filtering by rule name Introduces support for a new query parameter in the `/rules` API endpoint that allows filtering by rule names. If all the rules of a group are filtered, we skip the group entirely. Signed-off-by: gotjosh <josue.abreu@gmail.com>	3 years ago
Ben Ye	fd3630b9a3	add ctx to QueryEngine interface Signed-off-by: Ben Ye <benye@amazon.com>	3 years ago
Matthieu MOREL	fb3eb21230	enable gocritic, unconvert and unused linters Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	3 years ago
beorn7	817a2396cb	Name float values as "floats", not as "values" In the past, every sample value was a float, so it was fine to call a variable holding such a float "value" or "sample". With native histograms, a sample might have a histogram value. And a histogram value is still a value. Calling a float value just "value" or "sample" or "V" is therefore misleading. Over the last few commits, I already renamed many variables, but this cleans up a few more places where the changes are more invasive. Note that we do not to attempt naming in the JSON APIs or in the protobufs. That would be quite a disruption. However, internally, we can call variables as we want, and we should go with the option of avoiding misunderstandings. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	630bcb494b	storage: Use separate sample types for histogram vs. float Previously, we had one “polymorphous” `sample` type in the `storage` package. This commit breaks it up into `fSample`, `hSample`, and `fhSample`, each still implementing the `tsdbutil.Sample` interface. This reduces allocations in `sampleRing.Add` but inflicts the penalty of the interface wrapper, which makes things worse in total. This commit therefore just demonstrates the step taken. The next commit will tackle the interface overhead problem. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	c0879d64cf	promql: Separate `Point` into `FPoint` and `HPoint` In other words: Instead of having a “polymorphous” `Point` that can either contain a float value or a histogram value, use an `FPoint` for floats and an `HPoint` for histograms. This seemingly small change has a _lot_ of repercussions throughout the codebase. The idea here is to avoid the increase in size of `Point` arrays that happened after native histograms had been added. The higher-level data structures (`Sample`, `Series`, etc.) are still “polymorphous”. The same idea could be applied to them, but at each step the trade-offs needed to be evaluated. The idea with this change is to do the minimum necessary to get back to pre-histogram performance for functions that do not touch histograms. Here are comparisons for the `changes` function. The test data doesn't include histograms yet. Ideally, there would be no change in the benchmark result at all. First runtime v2.39 compared to directly prior to this commit: ``` name old time/op new time/op delta RangeQuery/expr=changes(a_one[1d]),steps=1-16 391µs ± 2% 542µs ± 1% +38.58% (p=0.000 n=9+8) RangeQuery/expr=changes(a_one[1d]),steps=10-16 452µs ± 2% 617µs ± 2% +36.48% (p=0.000 n=10+10) RangeQuery/expr=changes(a_one[1d]),steps=100-16 1.12ms ± 1% 1.36ms ± 2% +21.58% (p=0.000 n=8+10) RangeQuery/expr=changes(a_one[1d]),steps=1000-16 7.83ms ± 1% 8.94ms ± 1% +14.21% (p=0.000 n=10+10) RangeQuery/expr=changes(a_ten[1d]),steps=1-16 2.98ms ± 0% 3.30ms ± 1% +10.67% (p=0.000 n=9+10) RangeQuery/expr=changes(a_ten[1d]),steps=10-16 3.66ms ± 1% 4.10ms ± 1% +11.82% (p=0.000 n=10+10) RangeQuery/expr=changes(a_ten[1d]),steps=100-16 10.5ms ± 0% 11.8ms ± 1% +12.50% (p=0.000 n=8+10) RangeQuery/expr=changes(a_ten[1d]),steps=1000-16 77.6ms ± 1% 87.4ms ± 1% +12.63% (p=0.000 n=9+9) RangeQuery/expr=changes(a_hundred[1d]),steps=1-16 30.4ms ± 2% 32.8ms ± 1% +8.01% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=10-16 37.1ms ± 2% 40.6ms ± 2% +9.64% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=100-16 105ms ± 1% 117ms ± 1% +11.69% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=1000-16 783ms ± 3% 876ms ± 1% +11.83% (p=0.000 n=9+10) ``` And then runtime v2.39 compared to after this commit: ``` name old time/op new time/op delta RangeQuery/expr=changes(a_one[1d]),steps=1-16 391µs ± 2% 547µs ± 1% +39.84% (p=0.000 n=9+8) RangeQuery/expr=changes(a_one[1d]),steps=10-16 452µs ± 2% 616µs ± 2% +36.15% (p=0.000 n=10+10) RangeQuery/expr=changes(a_one[1d]),steps=100-16 1.12ms ± 1% 1.26ms ± 1% +12.20% (p=0.000 n=8+10) RangeQuery/expr=changes(a_one[1d]),steps=1000-16 7.83ms ± 1% 7.95ms ± 1% +1.59% (p=0.000 n=10+8) RangeQuery/expr=changes(a_ten[1d]),steps=1-16 2.98ms ± 0% 3.38ms ± 2% +13.49% (p=0.000 n=9+10) RangeQuery/expr=changes(a_ten[1d]),steps=10-16 3.66ms ± 1% 4.02ms ± 1% +9.80% (p=0.000 n=10+9) RangeQuery/expr=changes(a_ten[1d]),steps=100-16 10.5ms ± 0% 10.8ms ± 1% +3.08% (p=0.000 n=8+10) RangeQuery/expr=changes(a_ten[1d]),steps=1000-16 77.6ms ± 1% 78.1ms ± 1% +0.58% (p=0.035 n=9+10) RangeQuery/expr=changes(a_hundred[1d]),steps=1-16 30.4ms ± 2% 33.5ms ± 4% +10.18% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=10-16 37.1ms ± 2% 40.0ms ± 1% +7.98% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=100-16 105ms ± 1% 107ms ± 1% +1.92% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=1000-16 783ms ± 3% 775ms ± 1% -1.02% (p=0.019 n=9+9) ``` In summary, the runtime doesn't really improve with this change for queries with just a few steps. For queries with many steps, this commit essentially reinstates the old performance. This is good because the many-step queries are the one that matter most (longest absolute runtime). In terms of allocations, though, this commit doesn't make a dent at all (numbers not shown). The reason is that most of the allocations happen in the sampleRingIterator (in the storage package), which has to be addressed in a separate commit. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Ben Ye	fb67d368a2	use consistent error for instant and range query 400 Signed-off-by: Ben Ye <benye@amazon.com>	3 years ago
Xiaochao Dong (@damnever)	2b7202c4cc	Validate the metric names and labels in the remote write handler Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>	3 years ago
Hayk Davtyan	408f31f786	[WebUI/ScrapePoolList] Case-insensitive search of "Scrape Pools" (#12207 ) Signed-off-by: hayk96 <hayko5999@gmail.com>	3 years ago
Alison	3ac49d4ae2	Set SourceMap to false to fix UI in MCR builds (#12175 ) * Set sourceMap to false Signed-off-by: Alison Burgess <alburgess@microsoft.com> * add sourcemap=false to package.json build Signed-off-by: Alison Burgess <alburgess@microsoft.com> * set sourcemap to true in tsconfig Signed-off-by: Alison Burgess <alburgess@microsoft.com> --------- Signed-off-by: Alison Burgess <alburgess@microsoft.com> Co-authored-by: Alison Burgess <alburgess@microsoft.com>	3 years ago
Julien Pivotto	1070c9b06c	Release 2.43.0 Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	3 years ago
Julien Pivotto	2c6168be5f	Release 2.43.0-rc.1 Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	3 years ago
pbudner	46683eadf7	fix: advertise correct flag to enable remote write receiver Signed-off-by: pbudner <mail@pascalbudner.de>	3 years ago
Julien Pivotto	b6d91e8bf8	Release 2.43.0-rc.0 Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	3 years ago
Charles Korn	46a28899a0	Implement fully-featured content negotiation for API requests, and allow overriding the default API codec. Signed-off-by: Charles Korn <charles.korn@grafana.com>	3 years ago
Julien Pivotto	db2d759b81	Add support for lookbackdelta per query via the API Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	3 years ago

1 2 3 4 5 ...

1527 Commits (8ef7dfdeebf0a7491973303c7fb6b68ec5cc065b)