prometheus

Commit Graph

Author	SHA1	Message	Date
Nathan Baulch	50cd453c8f	chore: Fix typos (#14868 ) * Fix typos --------- Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>	10 months ago
György Krajcsovits	d3f4e7c223	Remove unnecessary conversion Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	10 months ago
György Krajcsovits	60ab1cc5a5	BUGFIX: TSDB: panic in chunk querier Followup to #14831 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	10 months ago
George Krajcsovits	536d9f9ce9	BUGFIX: TSDB: panic in query during truncation with OOO head (#14831 ) Check if headQuerier is nil before trying to use it. * TestQueryOOOHeadDuringTruncate: unit test to check query during truncate Regression test for #14822 * Simulate race between query and Compact() Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	10 months ago
Arthur Silva Sens	442f24e099	chore: Simplify TestHeadAppender_AppendCTZeroSample (#14812 ) Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	10 months ago
Oleg Zaytsev	ce7d830f1f	Bring back BenchmarkLoadRealWLs (#14757 ) This was part of #14525 which was reverted. I still think that having this benchmark committed in to the repo is useful. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	10 months ago
Marco Pracucci	ef649d5968	Revert " Store `mmMaxTime` in same field as `seriesShard`" Signed-off-by: Marco Pracucci <marco@pracucci.com>	10 months ago
Arve Knudsen	3a78e76282	Upgrade golangci-lint to v1.60.1 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	10 months ago
cuiweiyuan	1800af54f0	chore: fix some function names Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>	11 months ago
Oleg Zaytsev	0833d2a230	Fix appendable: check whether last val was a histogram (#14613 ) * Fix appendable: check whether last val was a histogram When appending a float, we were checking whether lastValue was equal to current value, but we didn't check whether last value was a float value. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	11 months ago
Oleg Zaytsev	b7f2f3c3ac	Add BenchmarkLoadRealWLs This benchmark runs on real WLs rather than fake generated ones. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	11 months ago
Carrie Edwards	79b53bd3de	Refactor TestWBLReplay to use scenarios Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Co-authored by: Fiona Liao <fiona.liao@grafana.com>:	12 months ago
Carrie Edwards	2e0e4e9ce9	Add support for handling multiple chunks in OOO head Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Co-authored by: Jeanette Tan <jeanette.tan@grafana.com>: Co-authored by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>: Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Co-authored by: Fiona Liao <fiona.liao@grafana.com>:	12 months ago
Carrie Edwards	06550883c1	Clean up of tests and test utils Co-authored by: Fiona Liao <fiona.liao@grafana.com>: Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>	12 months ago
Carrie Edwards	45a32a29ef	Update tsdb tests to use test utils. Co-authored-by: Fiona Liao <fiona.liao@grafana.com> Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>	12 months ago
Oleg Zaytsev	fd1a89b7c8	Pass affected labels to `MemPostings.Delete()` (#14307 ) * Pass affected labels to MemPostings.Delete As suggested by @bboreham, we can track the labels of the deleted series and avoid iterating through all the label/value combinations. This looks much faster on the MemPostings.Delete call. We don't have a benchmark on stripeSeries.gc() where we'll pay the price of iterating the labels of each one of the deleted series. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	1 year ago
Arve Knudsen	b2396c0c8f	Upgrade to golangci-lint v1.59.0 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	1 year ago
Alan Protasio	8894d65cd6	Fix head stats and hooks when replaying a corrupted snapshot (#14079 ) * Fixing head stats and hooks when replaying a corrupted snapshot Signed-off-by: alanprot <alanprot@gmail.com> * Fixing create/removed series metrics Signed-off-by: alanprot <alanprot@gmail.com> * Refactoring to have common code between gc and flush method Signed-off-by: alanprot <alanprot@gmail.com> * Update tsdb/head.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> * refactor Signed-off-by: alanprot <alanprot@gmail.com> * Update tsdb/head_test.go Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> * Update tsdb/head_test.go Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> --------- Signed-off-by: alanprot <alanprot@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	1 year ago
Oleksandr Redko	f10c3454e9	Enable perfsprint linter and fix up code Signed-off-by: Oleksandr Redko <oleksandr.red+github@gmail.com>	1 year ago
Matthieu MOREL	6f595c6762	golangci-lint: enable whitespace linter (#13905 ) Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	1 year ago
Nick Pillitteri	481f14e1c0	TSDB: Don't rely on integer overflow in head compaction check (#13755 ) * TSDB: Don't compact the head block when empty Don't compact the Head block if there have not yet been any samples appended. Previously, the logic for determining if the head should be compacted relied on the default values for min and max time and integer overflow when they were checked in `Head.compactable()`. The check in `Head.compactable()` effectively did `math.MinInt64 - math.MaxInt64` which overflowed and wrapped to `1`. Since `1` is less than `1.5` times the chunk range, compaction did not happen. This was the correct behavior but relying on overflow wrapping is surprising. This change add a method for checking if the min and max time for the head is unset and uses it to short-circuit compaction in that case. It also replaces several explicit checks for the default value to determine if the head has not yet had any samples added. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	1 year ago
Bryan Boreham	925134e6de	tsdb tests: make work with labels SymbolTable Need to initialize decoders with SymbolTable. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Bryan Boreham	c0e36e6bb3	Standardise exemplar label as "trace_id" This is consistent with the OpenTelemetry standard, and an example in OpenMetrics. https://github.com/open-telemetry/opentelemetry-specification/blob/89aa01348139/specification/metrics/data-model.md#exemplars https://github.com/OpenObservability/OpenMetrics/blob/138654493130/specification/OpenMetrics.md#exemplars-1 Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Bryan Boreham	12cac5bd5c	tsdb tests: use go-cmp instead of DeepEquals Also one simpler call checking nil. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Bryan Boreham	39af788dbd	Tests: use replacement DeepEquals using go-cmp Use DeepEqual replacement using go-cmp, which is more flexible. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Paweł Szulik	5961f78186	Refactor tsdb tests to use testify. Signed-off-by: Paweł Szulik <paul.szulik@gmail.com>	1 year ago
Marco Pracucci	501bc6419e	Add ShardedPostings() support to TSDB (#10421 ) This PR is a reference implementation of the proposal described in #10420. In addition to what described in #10420, in this PR I've introduced labels.StableHash(). The idea is to offer an hashing function which doesn't change over time, and that's used by query sharding in order to get a stable behaviour over time. The implementation of labels.StableHash() is the hashing function used by Prometheus before stringlabels, and what's used by Grafana Mimir for query sharding (because built before stringlabels was a thing). Follow up work As mentioned in #10420, if this PR is accepted I'm also open to upload another foundamental piece used by Grafana Mimir query sharding to accelerate the query execution: an optional, configurable and fast in-memory cache for the series hashes. Signed-off-by: Marco Pracucci <marco@pracucci.com>	1 year ago
Arve Knudsen	ba7012ec6a	TestHeadLabelValuesWithMatchers: Add test case (#13414 ) Add test case to TestHeadLabelValuesWithMatchers, while fixing a couple of typos in other test cases. Also enclosing some implicit sub-tests in a `t.Run` call to make them explicitly sub-tests. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	1 year ago
Filip Petkovski	583f3e587c	Optimize histogram iterators (#13340 ) Optimize histogram iterators Histogram iterators allocate new objects in the AtHistogram and AtFloatHistogram methods, which makes calculating rates over long ranges expensive. In #13215 we allowed an existing object to be reused when converting an integer histogram to a float histogram. This commit follows the same idea and allows injecting an existing object in the AtHistogram and AtFloatHistogram methods. When the injected value is nil, iterators allocate new histograms, otherwise they populate and return the injected object. The commit also adds a CopyTo method to Histogram and FloatHistogram which is used in the BufferedIterator to overwrite items in the ring instead of making new copies. Note that a specialized HPoint pool is needed for all of this to work (`matrixSelectorHPool`). --------- Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	1 year ago
Matthieu MOREL	8f6cf3aabb	tsdb: use Go standard errors Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Arthur Silva Sens	5082655392	Append Created Timestamps (#12733 ) * Append created timestamps. Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Log when created timestamps are ignored Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Proposed changes to Append CT PR. Changes: * Changed textparse Parser interface for consistency and robustness. * Changed CT interface to be more explicit and handle validation. * Simplified test, change scrapeManager to allow testability. * Added TODOs. Signed-off-by: bwplotka <bwplotka@gmail.com> * Updates. Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * Refactor head_appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Fix linter issues Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Use model.Sample in head appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> --------- Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: bwplotka <bwplotka@gmail.com>	2 years ago
Filip Petkovski	10a82f87fd	Enable reusing memory when converting between histogram types The 'ToFloat' method on integer histograms currently allocates new memory each time it is called. This commit adds an optional *FloatHistogram parameter that can be used to reuse span and bucket slices. It is up to the caller to make sure the input float histogram is not used anymore after the call. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2 years ago
Matthieu MOREL	9c4782f1cc	golangci-lint: enable testifylint linter (#13254 ) Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Xiaochao Dong	28d8f1650c	tsdb: Make sure the cache for postings cardinality properly honors the label name (#12653 ) Add a string remembering which label and limit the cache corresponds to. Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>	2 years ago
Arve Knudsen	1200c89d0c	Fix tsdb.stripeSeries.gc so it handles conflicts properly (#13195 ) * Fix tsdb.stripeSeries.gc so it handles conflicts properly tsdb.stripeSeries.gc needs to prune seriesHashmap.conflicts first, otherwise seriesHashmap replaces the unique field with the first among the conflicts. Also add regression test. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * TestStripeSeries_gc: Support stringlabels, don't use internals Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
Fiona Liao	5bee0cfce2	Change `ChunkReader.Chunk()` to `ChunkOrIterable()` The ChunkReader interface's Chunk() has been changed to ChunkOrIterable(). This is a precursor to OOO native histogram support - with OOO native histograms, the chunks.Meta passed to Chunk() can result in multiple chunks being returned rather than just a single chunk (e.g. if oooMergedChunk has a counter reset in the middle). To support this, ChunkOrIterable() requires either a single chunk or an iterable to be returned. If an iterable is returned, the caller has the responsibility of converting the samples from the iterable into possibly multiple chunks. The OOOHeadChunkReader now returns an iterable rather than a chunk to prepare for the native histograms case. Also as a beneficial side effect, oooMergedChunk and boundedChunk has been simplified as they only need to implement the Iterable interface now, not the full Chunk interface. --------- Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2 years ago
Arve Knudsen	ecc37588b0	tsdb: seriesHashmap.set by making receiver a pointer (#13193 ) * Fix tsdb.seriesHashmap.set by making receiver a pointer The method tsdb.seriesHashmap.set currently doesn't set the conflicts field properly, due to the receiver being a non-pointer. Fix by turning the receiver into a pointer, and add a corresponding regression test. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
George Krajcsovits	acc114fe55	Fix panic during tsdb Commit (#13092 ) * Fix panic during tsdb Commit Fixes the following panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x19deb45] goroutine 651118930 [running]: github.com/prometheus/prometheus/tsdb.(*headAppender).Commit(0xc19100f7c0) /drone/src/vendor/github.com/prometheus/prometheus/tsdb/head_append.go:855 +0x245 github.com/prometheus/prometheus/tsdb.dbAppender.Commit({{0x35bd6f0?, 0xc19100f7c0?}, 0xc000fa4c00?}) /drone/src/vendor/github.com/prometheus/prometheus/tsdb/db.go:1159 +0x2f We theorize that the panic happened due the the series referenced by the exemplar being removed between AppendExemplar and Commit due to being idle. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
George Krajcsovits	39a35d92bc	tsdb/head: wlog exemplars after samples (#13113 ) When samples are committed in the head, they are also written to the WAL. The order of WAL records should be sample then exemplar, but this was not the case for native histogram samples. This PR fixes that. The problem with the wrong order is that remote write reads the WAL and sends the recorded timeseries in the WAL order, which means exemplars arrived before histogram samples. If the receiving side is Prometheus TSDB and the series has not existed before then the exemplar does not currently create the series. Which means the exemplar is rejected and lost. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
Linas Medziunas	1f8aea11d6	Move histogram validation code to model/histogram Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>	2 years ago
Linas Medziunas	1cd6c1cde5	ValidateHistogram: strict Count check in absence of NaNs Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>	2 years ago
Oleksandr Redko	8e5f0387a2	ci(lint): enable nolintlint and remove redundant comments (#12926 ) Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>	2 years ago
Bryan Boreham	6fe8217ce4	tsdb: shrink txRing with smaller integers 4 billion active transactions ought to be enough for anyone. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Ganesh Vernekar	4df2f2432b	Additionally wrap WBL replay error (#12406 ) * Additionally wrap WBL replay error Although WBL replay is already wrapped with errLoadWbl, there are other errors that can happen during a WBL replay. We should not try to repair WAL in those cases. This commit additionally wraps the final error in Head.Init again with errLoadWbl so that WBL replay errors can be identified properly. Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com> Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com> Signed-off-by: Levi Harrison <git@leviharrison.dev>	2 years ago
Ganesh Vernekar	f5913266a1	Additionally wrap WBL replay error (#12406 ) * Additionally wrap WBL replay error Although WBL replay is already wrapped with errLoadWbl, there are other errors that can happen during a WBL replay. We should not try to repair WAL in those cases. This commit additionally wraps the final error in Head.Init again with errLoadWbl so that WBL replay errors can be identified properly. Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com> Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>	2 years ago
Björn Rabenstein	ec9170b8bf	Merge pull request #12874 from krajorama/outof-order-chunks Fix duplicate sample detection at chunk size limit Signed-off-by: SuperQ <superq@gmail.com>	2 years ago
Linas Medziunas	1aad4004c3	Additional test case for ValidateHistogram Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>	2 years ago
György Krajcsovits	56b3a015b6	Add regression test for duplicate detection at chunk size limit TestHeadDetectsDuplcateSampleAtSizeLimit tests a regression where a duplicate sample,is appended to the head, right when the head chunk is at the size limit. The test adds all samples as duplicate, thus expecting that the result has exactly half of the samples. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
Arve Knudsen	156222cc50	Add context argument to LabelQuerier.LabelValues (#12665 ) Add context argument to LabelQuerier.LabelValues and LabelQuerier.SortedLabelValues. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
Arve Knudsen	a964349e97	Add context argument to LabelQuerier.LabelNames (#12666 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago

1 2 3 4 5

224 Commits (50cd453c8fc5d7fd9270fc72dfa50367fc7a9c23)