prometheus

Commit Graph

Author	SHA1	Message	Date
machine424	0996b78326	remote_write: add a unit test to make sure the write client sends the extra http headers as expected This will help letting prometheus off the hook from situations like https://github.com/prometheus/prometheus/issues/13030 Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2 years ago
Linas Medziunas	1f8aea11d6	Move histogram validation code to model/histogram Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>	2 years ago
Linas Medziunas	1cd6c1cde5	ValidateHistogram: strict Count check in absence of NaNs Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>	2 years ago
Matthieu MOREL	fe057fc60d	use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Charles Korn	5184368db6	Fix issue where `chainSampleIterator` can obscure errors (#13006 ) * Fix issue where `chainSampleIterator` can obscure errors Signed-off-by: Charles Korn <charles.korn@grafana.com> * Address PR feedback. Signed-off-by: Charles Korn <charles.korn@grafana.com> --------- Signed-off-by: Charles Korn <charles.korn@grafana.com>	2 years ago
Oleksandr Redko	fa90ca46e5	ci(lint): enable godot; append dot at the end of comments Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>	2 years ago
beorn7	4696b46dd5	storage: Fix mixed samples handling in sampleRing Two issues are fixed here, that lead to the same problem: 1. If `newSampleRing` is called with an unknown ValueType including ValueNone, we have initialized the interface buffer (`iBuf`). However, we would still use a specialized buffer for the first sample, opportunistically assuming that we might still not encounter mixed samples and we should go down the more efficient road. 2. If the `sampleRing` is `reset`, we leave all buffers alone, including `iBuf`, which is generally fine, but not for `iBuf`, see below. In both cases, `iBuf` already contains values, but we will fill one of the specialized buffers first. Once we then actually encounter mixed samples, the content of the specialized buffer is copied into `iBuf` using `append`. That's by itself the right idea because `iBuf` might be `nil`, and even if not, it might or might not have the right capacity. However, this approach assumes that `iBuf` is empty, or more precisely has a length of zero. This commit makes sure that `iBuf` does not get needlessly initialized in `newSampleRing` and that it is emptied upon `reset`. A test case is added to demonstrate both issues above. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Oleksandr Redko	8e5f0387a2	ci(lint): enable nolintlint and remove redundant comments (#12926 ) Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>	2 years ago
Matthieu MOREL	1ec6e407d0	ci(lint): enable errorlint on storage (#12935 ) Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Levi Harrison	454a0a2c1b	Update dependencies for 2.48 (#12964 ) Signed-off-by: Levi Harrison <git@leviharrison.dev>	2 years ago
Levi Harrison	dcaca86958	Update dependencies for 2.48 (#12964 )	2 years ago
Bryan Boreham	a5a4eab679	Storage: reduce memory allocations when merging series sets (#12938 ) Instead of setting to nil and allocating a new slice every time the merge is advanced, re-use the previous slice. This is safe because the `currentSets` member is only used inside member functions, and explicitly copied in `At()`, the only place it leaves the struct. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
rakshith210	cdad64002a	Added Azure OAuth support (#12572 ) * Added Azure OAuth support Signed-off-by: rakshith210 <rakshith.me@gmail.com> * Added missing comment Signed-off-by: rakshith210 <rakshith.me@gmail.com> * Addressing comment Signed-off-by: rakshith210 <rakshith.me@gmail.com> * Fixed lint issue Signed-off-by: rakshith210 <rakshith.me@gmail.com> * Fix test Signed-off-by: rakshith210 <rakshith.me@gmail.com> * Addressing comments Signed-off-by: rakshith210 <rakshith.me@gmail.com> * Added documentation and updated unit tests Signed-off-by: rakshith210 <rakshith.me@gmail.com> * Addressing comments Signed-off-by: rakshith210 <rakshith.me@gmail.com> --------- Signed-off-by: rakshith210 <rakshith.me@gmail.com>	2 years ago
Goutham Veeramachaneni	86729d4d7b	Update exp package (#12650 )	2 years ago
William Dumont	ce6ad15422	remote-write: TestClientRetryAfter status code 500 and compare the retryAfter values. Signed-off-by: William Dumont <william.dumont@grafana.com>	2 years ago
William Dumont	febd62a23e	remote-write: refactor TestClientRetryAfter The new version features a set of test cases that simplify the addition of new HTTP status codes. Signed-off-by: William Dumont <william.dumont@grafana.com>	2 years ago
Bryan Boreham	9b85354acd	remote-write: respect Retry-After header on 5xx errors If the server sent it to us, we should assume it knows better than we do and respect it. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Paschalis Tsilias	c173cd57c9	Add a header to count retried remote write requests (#12729 ) Header name is `Retry-Attempt`, only set when >0. Signed-off-by: Marc Tuduri <marctc@protonmail.com> Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>	2 years ago
George Krajcsovits	3512b2d678	storage: make histogram reset handling consistent in chainSampleIterator (#12779 ) storage: make histogram reset handling consistent in chainSampleIterator --------- Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
zenador	69edd8709b	Add warnings (and annotations) to PromQL query results (#12152 ) Return annotations (warnings and infos) from PromQL queries This generalizes the warnings we have already used before (but only for problems with remote read) as "annotations". Annotations can be warnings or infos (the latter could be false positives). We do not treat them different in the API for now and return them all as "warnings". It would be easy to distinguish them and return infos separately, should that appear useful in the future. The new annotations are then used to create a lot of warnings or infos during PromQL evaluations. Partially these are things we have wanted for a long time (e.g. inform the user that they have applied `rate` to a metric that doesn't look like a counter), but the new native histograms have created even more needs for those annotations (e.g. if a query tries to aggregate float numbers with histograms). The annotations added here are not yet complete. A prominent example would be a warning about a range too short for a rate calculation. But such a warnings is more tricky to create with good fidelity and we will tackle it later. Another TODO is to take annotations into account when evaluating recording rules. --------- Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2 years ago
Arve Knudsen	156222cc50	Add context argument to LabelQuerier.LabelValues (#12665 ) Add context argument to LabelQuerier.LabelValues and LabelQuerier.SortedLabelValues. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
Arve Knudsen	a964349e97	Add context argument to LabelQuerier.LabelNames (#12666 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
beorn7	0521ec12af	storage: remove obsolete TODO This was solved one layer deeper with #11687. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Arve Knudsen	6daee89e5f	Add context argument to Querier.Select (#12660 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
Gregor Zeitlinger	f01718262a	Unit tests for native histograms (#12668 ) promql: Extend testing framework to support native histograms This includes both the internal testing framework as well as the rules unit test feature of promtool. This also adds a bunch of basic tests. Many of the code level tests can now be converted to tests within the framework, and more tests can be added easily. --------- Signed-off-by: Harold Dost <h.dost@criteo.com> Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com> Signed-off-by: Stephen Lang <stephen.lang@grafana.com> Co-authored-by: Harold Dost <h.dost@criteo.com> Co-authored-by: Stephen Lang <stephen.lang@grafana.com> Co-authored-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>	2 years ago
Justin Lei	8ef7dfdeeb	Add a chunk size limit in bytes (#12054 ) Add a chunk size limit in bytes This creates a hard cap for XOR chunks of 1024 bytes. The limit for histogram chunk is also 1024 bytes, but it is a soft limit as a histogram has a dynamic size, and even a single one could be larger than 1024 bytes. This also avoids cutting new histogram chunks if the existing chunk has fewer than 10 histograms yet. In that way, we are accepting "jumbo chunks" in order to have at least 10 histograms in a chunk, allowing compression to kick in. Signed-off-by: Justin Lei <justin.lei@grafana.com>	2 years ago
beorn7	aa82fe198f	tsdb: Fix histogram validation So far, `ValidateHistogram` would not detect if the count did not include the count in the zero bucket. This commit fixes the problem and updates all the tests that have been undetected offenders so far. Note that this problem would only ever create false negatives, so we never falsely rejected to store a histogram because of it. On the other hand, `ValidateFloatHistogram` has been to strict with the count being at least as large as the sum of the counts in all the buckets. Float precision issues could create false positives here, see products of PromQL evaluations, it's actually quite hard to put an upper limit no the floating point imprecision. Users could produce the weirdest expressions, maxing out float precision problems. Therefore, this commit simply removes that particular check from `ValidateFloatHistogram`. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Michael Hoffmann	4d8e380269	promql: allow tests to be imported (#12050 ) Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>	2 years ago
Bryan Boreham	a018a7ef53	storage: simplify Seek on BufferedSeriesIterator Small tweak to call a simpler method Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	d2ae8dc3cb	remote-write: add http.resend_count tracing attribute As recommended by the OpenTelemetry semantic conventions. https://opentelemetry.io/docs/specs/otel/trace/semantic_conventions/http/#http-client Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Goutham Veeramachaneni	ad4f514e66	Add OTLP Ingestion endpoint (#12571 ) * Add OTLP Ingestion endpoint We copy files from the otel-collector-contrib. See the README in `storage/remote/otlptranslator/README.md`. This supersedes: https://github.com/prometheus/prometheus/pull/11965 Signed-off-by: gouthamve <gouthamve@gmail.com> * Return a 200 OK It is what the OTEL Golang SDK expect :( https://github.com/open-telemetry/opentelemetry-go/issues/4363 Signed-off-by: Goutham <gouthamve@gmail.com> --------- Signed-off-by: gouthamve <gouthamve@gmail.com> Signed-off-by: Goutham <gouthamve@gmail.com>	2 years ago
George Krajcsovits	6cd2d1621f	Hide histogram chunk append and reset header internals (#12352 ) tsdb: Hide histogram chunk append and reset header internals Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>	2 years ago
LHHDZ	7d8f9b0978	remote-write receiver: reuse 'ref' to optimize multiple samples for same series (#12580 ) reuse 'ref' to optimize multi samples processing efficiency Signed-off-by: changlin.shi <changlin.shi@ly.com>	2 years ago
György Krajcsovits	d4e355243a	tsdbutil/ChunkFromSamplesGeneric should not panic Add error handling instead. Prepares for #12352 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
Bryan Boreham	ce153e3fff	Replace sort.Sort with faster slices.SortFunc The generic version is more efficient. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Bryan Boreham	5255bf06ad	Replace sort.Slice with faster slices.SortFunc The generic version is more efficient. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
rakshith210	b1675e23af	Add Azure AD package for remote write (#11944 ) * Add Azure AD package for remote write * Made AzurePublic default and updated configuration.md * Updated config structure and removed getToken at initialization * Changed passing context from request Signed-off-by: Rakshith Padmanabha <rapadman@microsoft.com> Signed-off-by: rakshith210 <rakshith.me@gmail.com>	3 years ago
Callum Styan	0d2108ad79	[tsdb] re-implement WAL watcher to read via a "notification" channel (#11949 ) * WIP implement WAL watcher reading via notifications over a channel from the TSDB code Signed-off-by: Callum Styan <callumstyan@gmail.com> * Notify via head appenders Commit (finished all WAL logging) rather than on each WAL Log call Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix misspelled Notify plus add a metric for dropped Write notifications Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update tests to handle new notification pattern Signed-off-by: Callum Styan <callumstyan@gmail.com> * this test maybe needs more time on windows? Signed-off-by: Callum Styan <callumstyan@gmail.com> * does this test need more time on windows as well? Signed-off-by: Callum Styan <callumstyan@gmail.com> * read timeout is already a time.Duration Signed-off-by: Callum Styan <callumstyan@gmail.com> * remove mistakenly commited benchmark data files Signed-off-by: Callum Styan <callumstyan@gmail.com> * address some review feedback Signed-off-by: Callum Styan <callumstyan@gmail.com> * fix missed changes from previous commit Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix issues from wrapper function Signed-off-by: Callum Styan <callumstyan@gmail.com> * try fixing race condition in test by allowing tests to overwrite the read ticker timeout instead of calling the Notify function Signed-off-by: Callum Styan <callumstyan@gmail.com> * fix linting Signed-off-by: Callum Styan <callumstyan@gmail.com> --------- Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
George Krajcsovits	f5fcaa3872	Fix setting reset header to gauge histogram in seriesToChunkEncoder (#12329 ) Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	3 years ago
Justin Lei	7bbf24b707	Make MemoizedSeriesIterator not implement chunkenc.Iterator Signed-off-by: Justin Lei <justin.lei@grafana.com>	3 years ago
beorn7	b0272255b7	storage: optimise sampleRing Replace many checks for the lengths of slices with a single tracking variable. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Justin Lei	6985dcbe73	Optimize and test MemoizedSeriesIterator Signed-off-by: Justin Lei <justin.lei@grafana.com>	3 years ago
Filip Petkovski	0d049feac7	Fix encoding samples in ChunkSeries (#12185 ) The storage.ChunkSeries iterator assumes that a histogram sample can always be appended to the currently open chunk. This is not the case when there is a counter reset, or when appending a stale sample to a chunk with non-stale samples. In addition, the open chunk sometimes needs to be recoded before a sample can be appended. This commit addresses the issue by implementing a RecodingAppender which can recode incoming samples in a transparent way. It also detects cases when a sample cannot be appended at all and returns `false` so that the caller can open a new chunk. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Jeanette Tan	2ad39baa72	Treat bucket limit like sample limit and make it fail the whole scrape and return an error Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	3 years ago
Jeanette Tan	4d21ac23e6	Implement bucket limit for native histograms Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	3 years ago
Jeanette Tan	1102ffd188	Fix according to code review Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	3 years ago
Jeanette Tan	e9a1e26ab7	Perform integer/float histogram type checking on conversions, and use a consistent method for determining integer vs float histogram Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	3 years ago
beorn7	5b53aa1108	style: Replace `else if` cascades with `switch` Wiser coders than myself have come to the conclusion that a `switch` statement is almost always superior to a statement that includes any `else if`. The exceptions that I have found in our codebase are just these two: * The `if else` is followed by an additional statement before the next condition (separated by a `;`). * The whole thing is within a `for` loop and `break` statements are used. In this case, using `switch` would require tagging the `for` loop, which probably tips the balance. Why are `switch` statements more readable? For one, fewer curly braces. But more importantly, the conditions all have the same alignment, so the whole thing follows the natural flow of going down a list of conditions. With `else if`, in contrast, all conditions but the first are "hidden" behind `} else if `, harder to spot and (for no good reason) presented differently from the first condition. I'm sure the aforemention wise coders can list even more reasons. In any case, I like it so much that I have found myself recommending it in code reviews. I would like to make it a habit in our code base, without making it a hard requirement that we would test on the CI. But for that, there has to be a role model, so this commit eliminates all `if else` occurrences, unless it is autogenerated code or fits one of the exceptions above. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	c3c7d44d84	lint: Adjust to the lint warnings raised by current versions of golint-ci We haven't updated golint-ci in our CI yet, but this commit prepares for that. There are a lot of new warnings, and it is mostly because the "revive" linter got updated. I agree with most of the new warnings, mostly around not naming unused function parameters (although it is justified in some cases for documentation purposes – while things like mocks are a good example where not naming the parameter is clearer). I'm pretty upset about the "empty block" warning to include `for` loops. It's such a common pattern to do something in the head of the `for` loop and then have an empty block. There is still an open issue about this: https://github.com/mgechev/revive/issues/810 I have disabled "revive" altogether in files where empty blocks are used excessively, and I have made the effort to add individual `// nolint:revive` where empty blocks are used just once or twice. It's borderline noisy, though, but let's go with it for now. I should mention that none of the "empty block" warnings for `for` loop bodies were legitimate. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Sebastian Rabenhorst	5d4ec08a1f	Fixed sampleRingIterator for mixed histograms Signed-off-by: Sebastian Rabenhorst <sebastian.rabenhorst@shopify.com> Fixed sampleRingIterator for mixed histograms Signed-off-by: Sebastian Rabenhorst <sebastian.rabenhorst@shopify.com> Fixed lint	3 years ago

1 2 3 4 5 ...

1352 Commits (1bfb3ed062e99bd3c74e05d9ff9a7fa4e30bbe21)