prometheus

Commit Graph

Author	SHA1	Message	Date
Charles Korn	59844498f7	Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115 ) * Add failing test. Signed-off-by: Charles Korn <charles.korn@grafana.com> * Don't run OOO head garbage collection while reads are running. Signed-off-by: Charles Korn <charles.korn@grafana.com> * Add further test cases for different order of operations. Signed-off-by: Charles Korn <charles.korn@grafana.com> * Ensure all queriers are closed if `DB.blockChunkQuerierForRange()` fails. Signed-off-by: Charles Korn <charles.korn@grafana.com> * Ensure all queriers are closed if `DB.Querier()` fails. Signed-off-by: Charles Korn <charles.korn@grafana.com> * Invert error handling in `DB.Querier()` and `DB.blockChunkQuerierForRange()` to make it clearer Signed-off-by: Charles Korn <charles.korn@grafana.com> * Ensure that queries that touch OOO data can't block OOO head garbage collection forever. Signed-off-by: Charles Korn <charles.korn@grafana.com> * Address PR feedback: fix parameter name in comment Co-authored-by: Jesus Vazquez <jesusvazquez@users.noreply.github.com> Signed-off-by: Charles Korn <charleskorn@users.noreply.github.com> * Address PR feedback: use `lastGarbageCollectedMmapRef` Signed-off-by: Charles Korn <charles.korn@grafana.com> * Address PR feedback: ensure pending reads are cleaned up if creating an OOO querier fails Signed-off-by: Charles Korn <charles.korn@grafana.com> --------- Signed-off-by: Charles Korn <charles.korn@grafana.com> Signed-off-by: Charles Korn <charleskorn@users.noreply.github.com> Co-authored-by: Jesus Vazquez <jesusvazquez@users.noreply.github.com>	2 years ago
Oleg Zaytsev	f997c72f29	Make head block ULIDs descriptive (#13100 ) * Make head block ULIDs descriptive As far as I understand, these ULIDs aren't persisted anywhere, so it should be safe to change them. When debugging an issue, seeing an ULID like `2ZBXFNYVVFDXFPGSB1CHFNYQTZ` or `33DXR7JA39CHDKMQ9C40H6YVVF` isn't very helpful, so I propose to make them readable in their ULID string version. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com> * Set a different ULID for RangeHead Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com> --------- Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	2 years ago
zenador	32ee1b15de	Fix error on ingesting out-of-order exemplars (#13021 ) Fix and improve ingesting exemplars for native histograms. See code comment for a detailed explanation of the algorithm. Note that this changes the current behavior for all kind of samples slightly: We now allow exemplars with the same timestamp as during the last scrape if the value or the labels have changed. Also note that we now do not ingest exemplars without timestamps for native histograms anymore. Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Björn Rabenstein <github@rabenste.in> --------- Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Signed-off-by: zenador <zenador@users.noreply.github.com> Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2 years ago
Matthieu MOREL	d7c3bc4cb0	tsdb/tsdbutil: use Go standard errors Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Matthieu MOREL	e60a508dd8	tsdb/errors: fix errorlint linter Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Matthieu MOREL	e3041740e4	tsdb/fileutil: use Go standard errors Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Matthieu MOREL	dd8871379a	remplace errors.Errorf by fmt.Errorf Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Bryan Boreham	1bfb3ed062	Labels: reduce allocations when creating from TSDB WAL (#13044 ) * Labels: reduce allocations when creating from TSDB When reading the WAL, by passing references into the buffer we can avoid copying strings under `-tags stringlabels`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	65a443e6e3	TSDB: initialize conflicts map only when we need it. Suggested by @songjiayang. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
George Krajcsovits	acc114fe55	Fix panic during tsdb Commit (#13092 ) * Fix panic during tsdb Commit Fixes the following panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x19deb45] goroutine 651118930 [running]: github.com/prometheus/prometheus/tsdb.(*headAppender).Commit(0xc19100f7c0) /drone/src/vendor/github.com/prometheus/prometheus/tsdb/head_append.go:855 +0x245 github.com/prometheus/prometheus/tsdb.dbAppender.Commit({{0x35bd6f0?, 0xc19100f7c0?}, 0xc000fa4c00?}) /drone/src/vendor/github.com/prometheus/prometheus/tsdb/db.go:1159 +0x2f We theorize that the panic happened due the the series referenced by the exemplar being removed between AppendExemplar and Commit due to being idle. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
Matthieu MOREL	469e415d09	Update record.go Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Matthieu MOREL	69c07ec6ae	Update record_test.go Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Matthieu MOREL	63691d82a5	tsdb/record: use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Matthieu MOREL	c74b7ad4fb	Update tombstones.go Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Matthieu MOREL	118460a64f	tsdb/tombstones: use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Matthieu MOREL	4d6d3c1715	tsdb/encoding: use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
George Krajcsovits	39a35d92bc	tsdb/head: wlog exemplars after samples (#13113 ) When samples are committed in the head, they are also written to the WAL. The order of WAL records should be sample then exemplar, but this was not the case for native histogram samples. This PR fixes that. The problem with the wrong order is that remote write reads the WAL and sends the recorded timeseries in the WAL order, which means exemplars arrived before histogram samples. If the receiving side is Prometheus TSDB and the series has not existed before then the exemplar does not currently create the series. Which means the exemplar is rejected and lost. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
Matthieu MOREL	2972cc5e8f	tsdb/index: use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Bryan Boreham	e6c0f69f98	TSDB: Only pay for hash collisions when they happen Instead of a map of slices of `memSeries`, ready for any of them to hold series where hash values collide, split into a map of `memSeries` and a map of slices which is usually empty, since hash collisions are a one-in-a-billion thing. The `del` method gets more complicated, to maintain the invariant that a series is only in one of the two maps. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	ce4e757704	TSDB: refine variable naming in chunk gc Slight further refactor. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	071d5732af	TSDB: refactor cleanup of chunks and series Extract the middle of the loop into a function, so it will be easier to modify the `seriesHashmap` data structure. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
machine424	a32fbc3658	head.go: Remove an unneeded snapshot trigger that was moved in https://github.com/prometheus/prometheus/pull/9328 and brougt back by mistake in `095f572d4a` as part of https://github.com/prometheus/prometheus/pull/11447 Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2 years ago
Matthieu MOREL	fb48a351f0	tsdb/wlog: use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
songjiayang	443867f1aa	symbolCacheEntry field type alignment, thus saving 8 bytes. Signed-off-by: songjiayang <songjiayang1@gmail.com>	2 years ago
Arve Knudsen	ae9221e152	tsdb/index.Symbols: Drop context argument from Lookup method (#13058 ) Drop context argument from tsdb/index.Symbols.Lookup since lookup should be fast and the context checking is a performance hit. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
Matthieu MOREL	ece8286305	tsdb/chunk: use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Matthieu MOREL	b60f9f801e	tsdb/chunkenc: use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Matthieu MOREL	724737006d	tsdb/agent: use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Linas Medziunas	ebed7d0612	Change Validate to be a method on histogram structs Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>	2 years ago
Linas Medziunas	1f8aea11d6	Move histogram validation code to model/histogram Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>	2 years ago
Linas Medziunas	1cd6c1cde5	ValidateHistogram: strict Count check in absence of NaNs Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>	2 years ago
Jeanette Tan	52eb303031	Refactor assigning MinTime in histogram chunks Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2 years ago
Jeanette Tan	3ccaaa40ba	Fix according to code review Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2 years ago
György Krajcsovits	4296ecbd14	tsdb/compact_test.go: test mixed typed series with PopulateBlock Add testcase and update test so that it can test native histograms as well. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
Jeanette Tan	27abf09e7f	Fix missing MinTime in histogram chunks Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2 years ago
Jeanette Tan	2f7060bd5a	Expand TestPopulateWithTombSeriesIterators to test earlier deletion intervals for histogram chunks as well as time-overlapping chunks Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2 years ago
Jeanette Tan	7a4a1127b7	Expand TestPopulateWithTombSeriesIterators to test min max times of chunks, including mixed chunks Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2 years ago
Jeanette Tan	04aabdd7cc	Refactor TestPopulateWithDelSeriesIterator unit tests to reuse more code Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2 years ago
Jeanette Tan	46be85f2dc	Make TestPopulateWithDelSeriesIterator tests cover histogram types and check MinTime Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2 years ago
Oleksandr Redko	fa90ca46e5	ci(lint): enable godot; append dot at the end of comments Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>	2 years ago
Oleksandr Redko	8e5f0387a2	ci(lint): enable nolintlint and remove redundant comments (#12926 ) Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>	2 years ago
zenador	80e977aae6	Remove `NewPossibleNonCounterInfo` and minimise creating empty annotations (#13012 ) * Remove NewPossibleNonCounterInfo until it can be made more efficient, and avoid creating empty annotations as much as possible Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2 years ago
Márcio Carôso	dff1c395f6	Expose --storage.tsdb.retention.time in metric prometheus_tsdb_retention_limit_seconds (#12986 ) * Expose --storage.tsdb.retention.time in a metric Signed-off-by: Marcio Caroso <msscaroso@gmail.com> --------- Signed-off-by: Marcio Caroso <msscaroso@gmail.com>	2 years ago
Jeanette Tan	71a36d2396	Very minor refactor of the integer overflow fix Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2 years ago
Bryan Boreham	26fa2e8356	TSDB: Pre-size buffer to read samples from WAL When reading the WAL this method is called with buffers from a pool, on multiple goroutines. Pre-allocating sufficient size avoids slow growth and many reallocations in `append`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
George Krajcsovits	7d7b9eacff	Fix int32 overflow issues (#12978 ) On a 32 bit architecture the size of int is 32 bits. Thus converting from int64, uint64 can overflow it and flip the sign. Try for yourself in playground: package main import "fmt" func main() { x := int64(0x1F0000001) y := int64(1) z := int32(x - y) // numerically this is 0x1F0000000 fmt.Printf("%v\n", z) } Prints -268435456 as if x was smaller. Followup to #12650 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
Paschalis Tsilias	42b8f2f5fc	tsdb/agent: allow ingestion of OOO samples (#12897 ) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> Signed-off-by: Levi Harrison <git@leviharrison.dev>	2 years ago
Ganesh Vernekar	4df2f2432b	Additionally wrap WBL replay error (#12406 ) * Additionally wrap WBL replay error Although WBL replay is already wrapped with errLoadWbl, there are other errors that can happen during a WBL replay. We should not try to repair WAL in those cases. This commit additionally wraps the final error in Head.Init again with errLoadWbl so that WBL replay errors can be identified properly. Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com> Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com> Signed-off-by: Levi Harrison <git@leviharrison.dev>	2 years ago
Paschalis Tsilias	afab845e65	tsdb/agent: allow ingestion of OOO samples (#12897 ) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>	2 years ago
Ganesh Vernekar	f5913266a1	Additionally wrap WBL replay error (#12406 ) * Additionally wrap WBL replay error Although WBL replay is already wrapped with errLoadWbl, there are other errors that can happen during a WBL replay. We should not try to repair WAL in those cases. This commit additionally wraps the final error in Head.Init again with errLoadWbl so that WBL replay errors can be identified properly. Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com> Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>	2 years ago

1 2 3 4 5 ...

898 Commits (784a2d2c7418f6dfd1864781d65acc4e7804a708)