prometheus

Commit Graph

Author	SHA1	Message	Date
Fiona Liao	c599d37668	Always return unknown hint for first sample in non-gauge histogram chunk (#15343 ) Always return unknown hint for first sample in non-gauge histogram chunk --------- Signed-off-by: Fiona Liao <fiona.liao@grafana.com> Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
György Krajcsovits	e6a682f046	Reproduce populateWithDelChunkSeriesIterator corrupting chunk meta When handling recoded histogram chunks the min time of the chunk is updated by mistake. It should only update when the chunk is completely new. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
György Krajcsovits	631fadc4ca	Unit test for data race in head.Appender.AppendHistogram Two Appenders race when creating a series with a native histogram as the memSeries will be common and the lastHistogram field is written without lock. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
Matthieu MOREL	ab64966e9d	fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" (#15094 ) * fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" --------- Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	1 year ago
Arthur Silva Sens	95a53ef982	Join tests for appending float and histogram CTs Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	1 year ago
Arthur Silva Sens	6bd9b1a7cc	Histogram CT Zero ingestion Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	1 year ago
George Krajcsovits	0d22a91267	Merge pull request #14874 from krajorama/fix-panic-in-ooo-query2 BUGFIX: TSDB: panic in chunk querier	1 year ago
Carrie Edwards	14e3c05ce8	tsdb: Add support for ingestion of out-of-order native histogram samples (#14546 ) Add support for ingesting OOO native histograms * Add flag for enabling and disabling OOO native histogram ingestion * Update OOO querying tests to include native histogram samples * Add OOO head tests * Add test for OOO native histogram counter reset headers Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored by: Carrie Edwards <edwrdscarrie@gmail.com> Co-authored by: Jeanette Tan <jeanette.tan@grafana.com> Co-authored by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored by: Fiona Liao <fiona.liao@grafana.com>	1 year ago
Harry John	919dc0cbc6	storage: Update LabelQuerier interface to return sorted label values (#14849 ) * Change LabelQuerier.LabelValues() to return sorted values --------- Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>	1 year ago
Nathan Baulch	50cd453c8f	chore: Fix typos (#14868 ) * Fix typos --------- Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>	1 year ago
György Krajcsovits	d3f4e7c223	Remove unnecessary conversion Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
György Krajcsovits	60ab1cc5a5	BUGFIX: TSDB: panic in chunk querier Followup to #14831 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
George Krajcsovits	536d9f9ce9	BUGFIX: TSDB: panic in query during truncation with OOO head (#14831 ) Check if headQuerier is nil before trying to use it. * TestQueryOOOHeadDuringTruncate: unit test to check query during truncate Regression test for #14822 * Simulate race between query and Compact() Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
Arthur Silva Sens	442f24e099	chore: Simplify TestHeadAppender_AppendCTZeroSample (#14812 ) Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	1 year ago
Oleg Zaytsev	ce7d830f1f	Bring back BenchmarkLoadRealWLs (#14757 ) This was part of #14525 which was reverted. I still think that having this benchmark committed in to the repo is useful. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	1 year ago
Marco Pracucci	ef649d5968	Revert " Store `mmMaxTime` in same field as `seriesShard`" Signed-off-by: Marco Pracucci <marco@pracucci.com>	1 year ago
Arve Knudsen	3a78e76282	Upgrade golangci-lint to v1.60.1 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	1 year ago
cuiweiyuan	1800af54f0	chore: fix some function names Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>	1 year ago
Oleg Zaytsev	0833d2a230	Fix appendable: check whether last val was a histogram (#14613 ) * Fix appendable: check whether last val was a histogram When appending a float, we were checking whether lastValue was equal to current value, but we didn't check whether last value was a float value. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	1 year ago
Oleg Zaytsev	b7f2f3c3ac	Add BenchmarkLoadRealWLs This benchmark runs on real WLs rather than fake generated ones. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	1 year ago
Carrie Edwards	79b53bd3de	Refactor TestWBLReplay to use scenarios Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Co-authored by: Fiona Liao <fiona.liao@grafana.com>:	2 years ago
Carrie Edwards	2e0e4e9ce9	Add support for handling multiple chunks in OOO head Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Co-authored by: Jeanette Tan <jeanette.tan@grafana.com>: Co-authored by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>: Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Co-authored by: Fiona Liao <fiona.liao@grafana.com>:	2 years ago
Carrie Edwards	06550883c1	Clean up of tests and test utils Co-authored by: Fiona Liao <fiona.liao@grafana.com>: Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>	2 years ago
Carrie Edwards	45a32a29ef	Update tsdb tests to use test utils. Co-authored-by: Fiona Liao <fiona.liao@grafana.com> Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>	2 years ago
Oleg Zaytsev	fd1a89b7c8	Pass affected labels to `MemPostings.Delete()` (#14307 ) * Pass affected labels to MemPostings.Delete As suggested by @bboreham, we can track the labels of the deleted series and avoid iterating through all the label/value combinations. This looks much faster on the MemPostings.Delete call. We don't have a benchmark on stripeSeries.gc() where we'll pay the price of iterating the labels of each one of the deleted series. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	2 years ago
Arve Knudsen	b2396c0c8f	Upgrade to golangci-lint v1.59.0 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
Alan Protasio	8894d65cd6	Fix head stats and hooks when replaying a corrupted snapshot (#14079 ) * Fixing head stats and hooks when replaying a corrupted snapshot Signed-off-by: alanprot <alanprot@gmail.com> * Fixing create/removed series metrics Signed-off-by: alanprot <alanprot@gmail.com> * Refactoring to have common code between gc and flush method Signed-off-by: alanprot <alanprot@gmail.com> * Update tsdb/head.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> * refactor Signed-off-by: alanprot <alanprot@gmail.com> * Update tsdb/head_test.go Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> * Update tsdb/head_test.go Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> --------- Signed-off-by: alanprot <alanprot@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Oleksandr Redko	f10c3454e9	Enable perfsprint linter and fix up code Signed-off-by: Oleksandr Redko <oleksandr.red+github@gmail.com>	2 years ago
Matthieu MOREL	6f595c6762	golangci-lint: enable whitespace linter (#13905 ) Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Nick Pillitteri	481f14e1c0	TSDB: Don't rely on integer overflow in head compaction check (#13755 ) * TSDB: Don't compact the head block when empty Don't compact the Head block if there have not yet been any samples appended. Previously, the logic for determining if the head should be compacted relied on the default values for min and max time and integer overflow when they were checked in `Head.compactable()`. The check in `Head.compactable()` effectively did `math.MinInt64 - math.MaxInt64` which overflowed and wrapped to `1`. Since `1` is less than `1.5` times the chunk range, compaction did not happen. This was the correct behavior but relying on overflow wrapping is surprising. This change add a method for checking if the min and max time for the head is unset and uses it to short-circuit compaction in that case. It also replaces several explicit checks for the default value to determine if the head has not yet had any samples added. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	2 years ago
Bryan Boreham	925134e6de	tsdb tests: make work with labels SymbolTable Need to initialize decoders with SymbolTable. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	c0e36e6bb3	Standardise exemplar label as "trace_id" This is consistent with the OpenTelemetry standard, and an example in OpenMetrics. https://github.com/open-telemetry/opentelemetry-specification/blob/89aa01348139/specification/metrics/data-model.md#exemplars https://github.com/OpenObservability/OpenMetrics/blob/138654493130/specification/OpenMetrics.md#exemplars-1 Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	12cac5bd5c	tsdb tests: use go-cmp instead of DeepEquals Also one simpler call checking nil. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	39af788dbd	Tests: use replacement DeepEquals using go-cmp Use DeepEqual replacement using go-cmp, which is more flexible. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Paweł Szulik	5961f78186	Refactor tsdb tests to use testify. Signed-off-by: Paweł Szulik <paul.szulik@gmail.com>	2 years ago
Marco Pracucci	501bc6419e	Add ShardedPostings() support to TSDB (#10421 ) This PR is a reference implementation of the proposal described in #10420. In addition to what described in #10420, in this PR I've introduced labels.StableHash(). The idea is to offer an hashing function which doesn't change over time, and that's used by query sharding in order to get a stable behaviour over time. The implementation of labels.StableHash() is the hashing function used by Prometheus before stringlabels, and what's used by Grafana Mimir for query sharding (because built before stringlabels was a thing). Follow up work As mentioned in #10420, if this PR is accepted I'm also open to upload another foundamental piece used by Grafana Mimir query sharding to accelerate the query execution: an optional, configurable and fast in-memory cache for the series hashes. Signed-off-by: Marco Pracucci <marco@pracucci.com>	2 years ago
Arve Knudsen	ba7012ec6a	TestHeadLabelValuesWithMatchers: Add test case (#13414 ) Add test case to TestHeadLabelValuesWithMatchers, while fixing a couple of typos in other test cases. Also enclosing some implicit sub-tests in a `t.Run` call to make them explicitly sub-tests. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
Filip Petkovski	583f3e587c	Optimize histogram iterators (#13340 ) Optimize histogram iterators Histogram iterators allocate new objects in the AtHistogram and AtFloatHistogram methods, which makes calculating rates over long ranges expensive. In #13215 we allowed an existing object to be reused when converting an integer histogram to a float histogram. This commit follows the same idea and allows injecting an existing object in the AtHistogram and AtFloatHistogram methods. When the injected value is nil, iterators allocate new histograms, otherwise they populate and return the injected object. The commit also adds a CopyTo method to Histogram and FloatHistogram which is used in the BufferedIterator to overwrite items in the ring instead of making new copies. Note that a specialized HPoint pool is needed for all of this to work (`matrixSelectorHPool`). --------- Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2 years ago
Matthieu MOREL	8f6cf3aabb	tsdb: use Go standard errors Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Arthur Silva Sens	5082655392	Append Created Timestamps (#12733 ) * Append created timestamps. Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Log when created timestamps are ignored Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Proposed changes to Append CT PR. Changes: * Changed textparse Parser interface for consistency and robustness. * Changed CT interface to be more explicit and handle validation. * Simplified test, change scrapeManager to allow testability. * Added TODOs. Signed-off-by: bwplotka <bwplotka@gmail.com> * Updates. Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * Refactor head_appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Fix linter issues Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Use model.Sample in head appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> --------- Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: bwplotka <bwplotka@gmail.com>	2 years ago
Filip Petkovski	10a82f87fd	Enable reusing memory when converting between histogram types The 'ToFloat' method on integer histograms currently allocates new memory each time it is called. This commit adds an optional *FloatHistogram parameter that can be used to reuse span and bucket slices. It is up to the caller to make sure the input float histogram is not used anymore after the call. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2 years ago
Matthieu MOREL	9c4782f1cc	golangci-lint: enable testifylint linter (#13254 ) Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Xiaochao Dong	28d8f1650c	tsdb: Make sure the cache for postings cardinality properly honors the label name (#12653 ) Add a string remembering which label and limit the cache corresponds to. Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>	2 years ago
Arve Knudsen	1200c89d0c	Fix tsdb.stripeSeries.gc so it handles conflicts properly (#13195 ) * Fix tsdb.stripeSeries.gc so it handles conflicts properly tsdb.stripeSeries.gc needs to prune seriesHashmap.conflicts first, otherwise seriesHashmap replaces the unique field with the first among the conflicts. Also add regression test. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * TestStripeSeries_gc: Support stringlabels, don't use internals Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
Fiona Liao	5bee0cfce2	Change `ChunkReader.Chunk()` to `ChunkOrIterable()` The ChunkReader interface's Chunk() has been changed to ChunkOrIterable(). This is a precursor to OOO native histogram support - with OOO native histograms, the chunks.Meta passed to Chunk() can result in multiple chunks being returned rather than just a single chunk (e.g. if oooMergedChunk has a counter reset in the middle). To support this, ChunkOrIterable() requires either a single chunk or an iterable to be returned. If an iterable is returned, the caller has the responsibility of converting the samples from the iterable into possibly multiple chunks. The OOOHeadChunkReader now returns an iterable rather than a chunk to prepare for the native histograms case. Also as a beneficial side effect, oooMergedChunk and boundedChunk has been simplified as they only need to implement the Iterable interface now, not the full Chunk interface. --------- Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2 years ago
Arve Knudsen	ecc37588b0	tsdb: seriesHashmap.set by making receiver a pointer (#13193 ) * Fix tsdb.seriesHashmap.set by making receiver a pointer The method tsdb.seriesHashmap.set currently doesn't set the conflicts field properly, due to the receiver being a non-pointer. Fix by turning the receiver into a pointer, and add a corresponding regression test. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
George Krajcsovits	acc114fe55	Fix panic during tsdb Commit (#13092 ) * Fix panic during tsdb Commit Fixes the following panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x19deb45] goroutine 651118930 [running]: github.com/prometheus/prometheus/tsdb.(*headAppender).Commit(0xc19100f7c0) /drone/src/vendor/github.com/prometheus/prometheus/tsdb/head_append.go:855 +0x245 github.com/prometheus/prometheus/tsdb.dbAppender.Commit({{0x35bd6f0?, 0xc19100f7c0?}, 0xc000fa4c00?}) /drone/src/vendor/github.com/prometheus/prometheus/tsdb/db.go:1159 +0x2f We theorize that the panic happened due the the series referenced by the exemplar being removed between AppendExemplar and Commit due to being idle. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
George Krajcsovits	39a35d92bc	tsdb/head: wlog exemplars after samples (#13113 ) When samples are committed in the head, they are also written to the WAL. The order of WAL records should be sample then exemplar, but this was not the case for native histogram samples. This PR fixes that. The problem with the wrong order is that remote write reads the WAL and sends the recorded timeseries in the WAL order, which means exemplars arrived before histogram samples. If the receiving side is Prometheus TSDB and the series has not existed before then the exemplar does not currently create the series. Which means the exemplar is rejected and lost. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
Linas Medziunas	1f8aea11d6	Move histogram validation code to model/histogram Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>	2 years ago
Linas Medziunas	1cd6c1cde5	ValidateHistogram: strict Count check in absence of NaNs Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>	2 years ago

1 2 3 4 5

232 Commits (bwplotka-patch-2)