prometheus

Commit Graph

Author	SHA1	Message	Date
bwplotka	efdfb8fed6	refactor(appenderV2): 1:1 copy of head append test files for v2 (starting point) Signed-off-by: bwplotka <bwplotka@gmail.com>	2 months ago
bwplotka	129650df9d	refactor(appenderV2): 1:1 copy of head_append.go -> head_append_v2.go (starting point) Signed-off-by: bwplotka <bwplotka@gmail.com>	2 months ago
bwplotka	04696703fe	refactor(appenderV2): add AppenderV2 interface Signed-off-by: bwplotka <bwplotka@gmail.com>	2 months ago
Bartlomiej Plotka	f6ca7145ca	refactor(tsdb): use one test newTestDB constructor (#17638 ) For tests only, we had various ways of opening DB. Reduced to one instead of: * Open * newTestDB * newTestDBOpts * openTestDB This so https://github.com/prometheus/prometheus/pull/17629 is smaller and bit easier. Also for test maintainability and consistency. Signed-off-by: bwplotka <bwplotka@gmail.com>	2 months ago
Julien	93edf912a7	Merge pull request #17630 from zjumathcode/main chore: Fix function name typo in createBatchSpan comment	2 months ago
Ben Edmunds	0e682a70a6	RW2: Allow custom scope in azuread (#17483 ) Signed-off-by: Ben Edmunds <sammybenblue2@gmail.com>	2 months ago
Łukasz Mierzwa	8a1086a128	feat: Add flag that blocks lvl 1 compactions until upload is confirmed in an external JSON file (#17435 ) * Delay compactions until Thanos uploads all blocks Using Thanos sidecar with Prometheus requires us to disable TSDB compactions on Prometheus side by setting --storage.tsdb.min-block-duration and --storage.tsdb.max-block-duration to the same value. See https://thanos.io/tip/components/sidecar.md. The main problem this avoids is that Prometheus might compact given block before Thanos uploads it, creating a gap in Thanos metrics. Thanos does not upload compacted blocks because that would upload the same sample multiple times. You can tell Thanos to upload compacted blocks but that is aimed at one time migrations. This patch creates a bridge between Thanos and Prometheus by allowing Prometheus to read the shipper file Thanos creates, where it tracks which blocks were already uploaded, and using that data delays compaction of blocks until they are marked as uploaded by Thanos. Thanks to this both services can coordinate with each other (in a way) and we can stop disabling compaction on Prometheus side when Thanos uploads are enabled. The reason to have this is that disabling compactions have very dramatic performance cost. Since most time series exist for longer than a single block duration (2h by default) large chunks of block index will reference the same series, so 10 * 2h blocks will each have an index that is usually fairly big and is almost the same for all 10 blocks. Compaction de-duplicates the index so merging 10 blocks together would leave us with a single index that is around the same size as each of these 10 2h blocks would have (plus some extra for series that only exists in some blocks, but not all). Every range query that iterates over all 10 blocks would then have to read each index and so we're doing 10x more work then if we had a single compacted block. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * Rename structs and functions to make this more generic Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * Address review comments Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * Cache UploadMeta for 1 minute Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> --------- Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>	2 months ago
zjumathcode	0ac2221a20	chore: Fix function name typo in createBatchSpan comment Signed-off-by: zjumathcode <pai314159@2980.com>	2 months ago
Nikos Angelopoulos	b0649e08c4	rules: replace error strings with sentinel errors for duplicate labelsets (#17620 ) Signed-off-by: Nikos Angelopoulos <nikolaos.angelopoulos@grafana.com>	2 months ago
Julius Volz	96d3d641e3	Merge pull request #17605 from ADITYATIWARI342005/fix/CompSugg [BUGFIX] UI :Suppress duration unit autocomplete when unit already present in query selector	2 months ago
Solomon Jacobs	abffb92847	drop GO111MODULE=on (#17520 ) This is ignored as of go 1.17, see e.g., for reference https://go.dev/blog/go116-module-changes Signed-off-by: Solomon Jacobs <solomonjacobs@protonmail.com>	2 months ago
harsh kumar	30be1483d1	instrumentation: add native histograms to complement high-traffic summaries (#17374 ) This adds the following native histograms (with a few classic buckets for backwards compatibility), while keeping the corresponding summaries (same name, just without `_histogram`): - `prometheus_sd_refresh_duration_histogram_seconds` - `prometheus_rule_evaluation_duration_histogram_seconds` - `prometheus_rule_group_duration_histogram_seconds` - `prometheus_target_sync_length_histogram_seconds` - `prometheus_target_interval_length_histogram_seconds` - `prometheus_engine_query_duration_histogram_seconds` Signed-off-by: Harsh <harshmastic@gmail.com> Signed-off-by: harsh kumar <135993950+hxrshxz@users.noreply.github.com> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2 months ago
ADITYA TIWARI	49427cfcd2	Refactor duration regex and remove RegExp.escape polyfill Removed polyfill for RegExp.escape and updated duration regex. Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com>	2 months ago
Andrew Hall	7bb95d548c	promql: Ensure that `rate`/`increase`/`delta` of histograms results in a gauge histogram. (#17608 ) Signed-off-by: Andrew Hall <andrew.hall@grafana.com>	2 months ago
Ayoub Mrini	e7999528fa	fix(test): make TestRemoteWrite_ReshardingWithoutDeadlock more reliable and re-enable it (#17490 ) Improve test stability by waiting for the relevant metrics to appear on /metrics before the first check on the desired shard count. Increase the scrape interval to avoid timeouts, as 100 ms may be insufficient for Prometheus to scrape itself in some environments (e.g., CI). Have Prometheus scrape itself multiple times to increase the volume of data sent and help fill the queue more quickly. Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2 months ago
ADITYA TIWARI	42418660d3	fix: lint errors in the files; move regex to one-line only Signed-off-by: ADITYA TIWARI <adityatiwari342005@gmail.com>	2 months ago
ADITYA TIWARI	4fa435fad2	feat: use RegExp.escape polyfill for robust PromQL duration regex; add compound duration test cases Signed-off-by: ADITYA TIWARI <adityatiwari342005@gmail.com>	2 months ago
George Krajcsovits	a66c696530	chore(storage): update docstring (#17609 ) The original implementation in #9705 for native histograms included a technical dept #15177 where samples were committed ordered by type not by their append order. This was fixed in #17071, but this docstring was not updated. I've also took the liberty to mention that we do not order by timestamp either, thus it is possible to append out of order samples. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 months ago
ADITYA TIWARI	3b098799d4	Export durationWithUnitRegexp for external use Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com>	2 months ago
ADITYA TIWARI	137f846527	Add tests for durationWithUnitRegexp functionality Added tests for durationWithUnitRegexp to validate matching of complete durations with units and ensure non-matching cases are correctly identified. Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com>	2 months ago
ADITYA TIWARI	bf76fde0c8	Update duration regex for complete duration matching Refactor duration regex to match complete durations with units. Signed-off-by: ADITYA TIWARI <142050150+ADITYATIWARI342005@users.noreply.github.com>	2 months ago
Faustas Butkus	e43f1bafca	chore: fix rangeEval comment (#17607 ) Signed-off-by: Faustas Butkus <faustas.butkus@chronosphere.io>	2 months ago
ADITYA TIWARI	04a5a488b8	fix: Suppress autocomplete for duration units when unit already present - No duration unit suggestions shown if a valid unit follows the digit (e.g. , ) - Adds related test cases Signed-off-by: ADITYA TIWARI <adityatiwari342005@gmail.com>	2 months ago
Julius Volz	fb47037435	Merge pull request #17602 from ADITYATIWARI342005/fix/closingAutocomplete [BUGFIX] UI: Fixed codemirror-promql incorrectly showing label completion suggestions	2 months ago
ADITYA TIWARI	02f405692e	fix: autocomplete suggestions for using cursor position Signed-off-by: ADITYA TIWARI <adityatiwari342005@gmail.com>	2 months ago
Bartlomiej Plotka	b2eb2bc989	chore(labels): add more context to labels.MetricName deprecation. (#17590 ) Signed-off-by: bwplotka <bwplotka@gmail.com>	2 months ago
Ben Kochie	d0d2699dc5	Update Prometheus Agent doc (#17591 ) * Add a nav title to fix docs website generator. * Make it more clear that "Prometheus Agent" is a mode, not a seaparate service. * Add to index. * Cleanup some wording. * Add a downsides section. Signed-off-by: SuperQ <superq@gmail.com>	2 months ago
Björn Rabenstein	b8d19543b8	Add histogram validation in remote-read and during reducing resolution (#17561 ) ReduceResolution is currently called before validation during ingestion. This will cause a panic if there are not enough buckets in the histogram. If there are too many buckets, the spurious buckets are ignored, and therefore the error in the input histogram is masked. Furthermore, invalid negative offsets might cause problems, too. Therefore, we need to do some minimal validation in reduceResolution. Fortunately, it is easy and shouldn't slow things down. Sadly, it requires to return errors, which triggers a bunch of code changes. Even here is a bright side, we can get rud of a few panics. (Remember: Don't panic!) In different news, we haven't done a full validation of histograms read via remote-read. This is not so much a security concern (as you can throw off Prometheus easily by feeding it bogus data via remote-read) but more that remote-read sources might be makeshift and could accidentally create invalid histograms. We really don't want to panic in that case. So this commit does not only add a check of the spans and buckets as needed for resolution reduction but also a full validation during remote-read. Signed-off-by: beorn7 <beorn@grafana.com>	2 months ago
Ben Kochie	fc27eef43f	Group more dependabot updates (#17563 ) Reduce the number of dependabot PRs for related udpdates. Signed-off-by: SuperQ <superq@gmail.com>	2 months ago
Julius Volz	36d054cb2e	UI: Add graph option to start the chart's Y axis at zero (#17565 ) To reduce main UI clutter, I added a new settings submenu above the chart itself for the new setting. So far it only has the one new axis setting, but it could accommodate further settings in the future. For now I'm only adding a boolean on/off setting to the UI to set the Y axis to 0 or not. However, the underlying stored URL field is already named y_axis_min={number} and would support other Y axis minima, in case we want to support custom values in the UI in the future - but then we'd probably also want to add an axis maximum and possibly other settings. Fixes https://github.com/prometheus/prometheus/issues/520 Signed-off-by: Julius Volz <julius.volz@gmail.com>	2 months ago
Björn Rabenstein	5947cc1459	Merge pull request #17559 from prometheus/beorn7/histogram3 model/histogram: Make histogram bucket iterators more robust	2 months ago
Julien	61f64a4cb1	Makefile.common: Use git ls-files instead of find for license check and style check (#17557 ) Also improve find fallback to use -prune for better performance. Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>	2 months ago
beorn7	2dfc324821	model/histogram: Make histogram bucket iterators more robust Currently, iterating over histogram buckets can panic if the spans are not consistent with the buckets. We aim for validating histograms upon ingestion, but there might still be data corruptions on disk that could trigger the panic. While data corruption on disk is really bad and will lead to all kind of weirdness, we should still avoid panic'ing. Note, though, that chunks are secured by checksums, so the corruptions won't realistically happen because of disk faults, but more likely because a chunk was generated in a faulty way in the first place, by a software bug or even maliciously. This commit prevents panics in the situation where there are fewer buckets than described by the spans. Note that the missing buckets will simply not be iterated over. There is no signalling of this problem. We might still consider this separately, but for now, I would say that this kind of corruption is exceedingly rare and doesn't deserve special treatment (which will add a whole lot of complexity to the code). Signed-off-by: beorn7 <beorn@grafana.com>	2 months ago
Grégoire	1174b0ce4f	model/textparse: Remove unit validation in protobuf parsing (#16834 ) Signed-off-by: Gregoire Verdier <gregoire.verdier@gmail.com>	2 months ago
Björn Rabenstein	d943f445f0	Merge pull request #17528 from prometheus/beorn7/histogram cmd: Make feature flag `native-histograms` a no-op.	2 months ago
Julien	e47c7e2f96	Merge pull request #17549 from prometheus/superq/improve_repo_sync_logging Improve repo sync script logging	2 months ago
Andrew Hall	1193e63896	PromQL: Modify RatioSampler to expose more methods for the benefit of downstream projects (#17516 ) Methods added: - `SampleOffset(metric labels.Labels) float64` to calculate the sample offset for a given label set. - `AddRatioSampleWithOffset(ratioLimit, sampleOffset float64) bool` to find out whether a given sample offset falls within a given ratio limit. The already existing method `AddRatioSample(ratioLimit float64, sample Sample) bool` is now implemented as a simple combination of the two other methods. Exposing these methods helps downstream projects to re-use the implementations including easier testing. Signed-off-by: Andrew Hall <andrew.hall@grafana.com>	2 months ago
SuperQ	4bee2c754e	Improve repo sync script logging Improve the repo sync logging output and add some additional logging. This should help debugging some failed updates. Signed-off-by: SuperQ <superq@gmail.com>	2 months ago
Minh Nguyen	5087a25848	Remote Write Receive Fix: Remove duplicate labels when type-and-unit-label feature is on (#17546 ) * drop extra label from receiver Signed-off-by: pipiland2612 <nguyen.t.dang.minh@gmail.com> * used constant Signed-off-by: pipiland2612 <nguyen.t.dang.minh@gmail.com> --------- Signed-off-by: pipiland2612 <nguyen.t.dang.minh@gmail.com>	2 months ago
Bartlomiej Plotka	cefefc6897	prw2: Move Remote Write 2.0 CT to be per Sample; Rename to ST (start timestamp) (#17411 ) Relates to https://github.com/prometheus/prometheus/issues/16944#issuecomment-3164760343 Signed-off-by: bwplotka <bwplotka@gmail.com>	2 months ago
Laurent Dufresne	d99f8dacc4	chore: remove dead code (#17542 ) Signed-off-by: Laurent Dufresne <laurent.dufresne@grafana.com>	2 months ago
beorn7	be4efd740c	cmd: Make feature flag `native-histograms` a no-op. Signed-off-by: beorn7 <beorn@grafana.com>	2 months ago
Will Bollock	4aa8941eb1	fix(discovery): aws discovery test fix (#17527 ) * fix: aws discovery test fix Fixes a problem introduced after the merge of this https://github.com/prometheus/prometheus/pull/17138 PR didn't take into account another merged PR! ``` discovery/aws/aws.go:218:54: too many arguments in call to NewEC2Discovery have (EC2SDConfig, slog.Logger, ec2Metrics) want (EC2SDConfig, discovery.DiscovererOptions) discovery/aws/aws.go:222:66: too many arguments in call to NewLightsailDiscovery have (LightsailSDConfig, slog.Logger, lightsailMetrics) want (LightsailSDConfig, discovery.DiscovererOptions) ``` Signed-off-by: Will Bollock <wbollock@linode.com> * fix: align ecs style ECS was a new service discovery tool added after this PR was merged: https://github.com/prometheus/prometheus/pull/17138 Aligns the style of passing a single "opts" to it like almost all the other service discovery engines now use Signed-off-by: Will Bollock <wbollock@linode.com> --------- Signed-off-by: Will Bollock <wbollock@linode.com>	2 months ago
0xkato	ae00fd45ab	tsdb: guard chunk length overflow in head chunk reader (#17533 ) Signed-off-by: 0xkato <0xkkato@gmail.com>	2 months ago
zenador	c64dd612ef	PromQL: Fix bug with inconsistent results for queries with OR expression and EnableDelayedNameRemoval (#17161 ) Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: zenador <zenador@users.noreply.github.com> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2 months ago
Julius Volz	3c53abc9e7	Merge pull request #17537 from rauldsl/prometheus-docs [Doc] Storage.md - Paragraph Rewrite	2 months ago
Raul Leite	407b697ee2	structure adjusted as reccomended Corrected the structure of the explanation regarding how samples are organized in the chunks directory and the handling of deletion records. Signed-off-by: Raul Leite <sp4wn.root@gmail.com>	2 months ago
Raul Leite	c9827ef983	Fix formatting in storage.md extra space removed Signed-off-by: Raul Leite <sp4wn.root@gmail.com>	2 months ago
Raul Leite	e022a727a8	I’ve proposed a slight rewording of this section to improve clarity and readability. (On-Disk Layout Paragraph) Signed-off-by: Raul Leite <sp4wn.root@gmail.com>	2 months ago
Bryan Boreham	1240402620	Merge pull request #17439 from bboreham/faster-postings tsdb: couple of postings optimizations	2 months ago

1 2 3 4 5 ...

16508 Commits (efdfb8fed626ebbcddeeb60ea9ba8b29c13dbdf2) All Branches Search

16508 Commits (efdfb8fed626ebbcddeeb60ea9ba8b29c13dbdf2)

All Branches