3613 Commits (f80e487dcf106a6c4ea0a82d8afe7b0104addbc3)

 

All Branches   

Search

Author	SHA1	Message	Date
Michel Hollands	f80e487dcf	Remove whole stream deletion mode (#6435) ... * Remove whole-stream-deletion mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove whole-stream-deletion from docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update the changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Sort changelog entries Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove link to wrong configuration Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix integration test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Set default deletion mode to disabled Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove extra white line in documentation Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix default value in docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add DeletionEnabled method on mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename test Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Christian Haudum	798677ae7f	Fix panic in instant query splitting when using unwrapped rate (#6348) ... * Fix panic in instant query splitting when using unwrapped rate The range aggregation `rate()` supports both log ranges and unwrapped ranges, e.g. `rate({app="foo"} [$__interval])` and `rate({app="foo"} | unwrap bar [$__interval])` Since `rate()` was split into multiple `count_over_time()` over total duration, but `count_over_time()` does not support `unwrap`, unwrapped rate queries caused panics. This fix changes the splitting of `rate({app="foo"} | unwrap bar [$__interval]` into multiple `sum_over_time()` over total duration. Fixes #6344 Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add tests Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Integrate review feedback Co-authored-by: Susana Ferreira <susana.ferreira@grafana.com> Co-authored-by: Susana Ferreira <susana.ferreira@grafana.com>	3 years ago
Dylan Guedes	8dcc2d6e55	Use shipper as indexClient for read mode. (#6461)	3 years ago
Dylan Guedes	66324a65a2	Set RingManager mode as Server if in read target. (#6427)	3 years ago
Michel Hollands	3eecd7fe62	operator: Add shellcheck disables (#6460) ... Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Periklis Tsirakidis	381f9d2142	operator: Add e2e local development with kind (#6339)	3 years ago
Karen Miller	167c95c320	Docs: revise multi-tenancy prose (#6355) ... * Docs: revise multi-tenancy prose * Update multi-tenancy docs using review comments	3 years ago
Conor Evans	b65e11ec9b	fix(documentation): correct spelling and scrape_config (#6454) ... Signed-off-by: Conor Evans <coevans@tcd.ie>	3 years ago
Periklis Tsirakidis	7b1ca8df6c	operator: Move cli flags to component config (#6433)	3 years ago
Mohamed-Amine Bouqsimi	4f4b42ba73	Add TLS config to query frontend (#6444) ... * added tls config to query frontend * changelog * remove commented code * lint	3 years ago
Robert Jacob	56363801b6	operator: Update Grafana and forwarding documentation (#6442)	3 years ago
Gerard Vanloo	8d6e2f0275	operator: Extend schema validation in LokiStack webhook (#6411)	3 years ago
Shweta Padubidri	334c39461f	operator: Enhancement Proposal to configure log retention (#6232) ... Signed-off-by: Shweta Padubidri <spadubid@redhat.com>	3 years ago
Salva Corts	1cfc30fb23	Evenly spread queriers across available nodes (#6415) ... * Evenly spread queriers across available nodes * Fix lint issue * Add entry to the CHANGELOG and the Upgrade Guide * Make topology spread configurable * Apply CR feedback	3 years ago
Michel Hollands	b4e6c59936	Compactor: add per tenant compaction delete enabled flag (#6410) ... * Add per tenant compaction delete enabled flag Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove changes in wrong place Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add compactor deletion enabled field Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use limit in compactor Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use http middleware and add test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix lint issue Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Revert to default setting if no override Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add default value command line option Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update the docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Enable access to deletion API for integration test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename flag to allow_deletes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update per review comments Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Cyril Tovena	8ee0d62d9e	Fixes bad initialization for uniqueString (#6432)	3 years ago
Owen Diehl	a1c72ad562	Tsdb/wal rotate fix (#6423) ... * rotate tsdb wal under mutex Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * ensure we rotate and doublecheck conditions under lock in tsdb headmanager Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Callum Styan	aa31b46d26	[modules] pin version of memberlist (#6413) ... * Use a replace directive to ensure we're using the same version of memberlist as mimir. Signed-off-by: Callum Styan <callumstyan@gmail.com> * run go mod tidy Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Christian Haudum	a56a88bd35	Add unwrapped aggregation `rate_counter()` (#6412) ... `rate_counter()` takes an unwrapped range as input, but unlike `rate()` it treats the values extracted from the log lines as "counter metric" like in Prometheus' `rate()` function. This is a replacement for the reverted change of #5013 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Tomáš Linhart	35cb40af96	Add support for ARM64 to lambda-promtail drone build job (#5354) ... * Add support for ARM64 to lambda-promtail drone build job * update drone.yml based on tomas' changes Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add entry to CHANGELOG.md * Change Docker image for building Promtail Lambda * reverse previous commit Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add manifest for ECR Public images * Minor change to drone.yml for volume mounts + generate drone.jsonnet again. Signed-off-by: Callum Styan <callumstyan@gmail.com> * temp, test drone changes without having to push to main branch Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix linting issues Signed-off-by: Callum Styan <callumstyan@gmail.com> * temp, push the lambda-promtail images always so we can test the manifest task Signed-off-by: Callum Styan <callumstyan@gmail.com> * We don't build a base arm image, only arm64 Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add volumes definition to ECR manifest pipeline * Regenerate drone.yml with Tomas' most recent commit Signed-off-by: Callum Styan <callumstyan@gmail.com> * Test some some drone changes. Signed-off-by: Callum Styan <callumstyan@gmail.com> * fix lint error in drone.jsonnet Signed-off-by: Callum Styan <callumstyan@gmail.com> * Drop out one of my more recent test changes. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Regenerate drone config. Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Owen Diehl	d6f50ca5e1	[TSDB] Use index sampling to determine shard factor (#6396) ... * use index sampling to determine shard factor * logging/tracing + better parallelism planning * queryrange downstreamer now checks max_query_parallelism * lint * handles interval, offset in tsdb planning and adds logging * fixes ns->ms confusion in index stats proto * handle zero shard value without panics * shardmapper will downstream a single unsharded query * uses concat expr with no shards to avoid query parsing errors * better logging * fixes wrong Size() method call and rounds to nearest KB when calculating chunk size * humanize bytes in log line * only adds defaultLookback to index sampling when interval is zero * removes comment * more logging * adjust through correctly * adds query length for index queries	3 years ago
Trevor Whitney	01fe534751	Add index gateway to the read path (#6385)	3 years ago
Christian Simon	9ff621825a	Parallize label queries to both stores (#5997) ... * Parallize label queries to both stores This executes label queries against ingesters and store in parallel, rather than sequentially * Return err without any checks as suggested	3 years ago
Christian Haudum	b315ed0ddf	Sum values in unwrapped rate aggregation instead of treating them as counter (#6361) ... * Revert unwrapped rate aggregation to previous implementation This PR reverts the implementation done in #5013 to the original implementation that sums the extracted values from the log lines instead of treating them like a Prometheus counter metric. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Move changelog entry Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Remove unused/dead code Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Clean changelog Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Tristan Colgate-McFarlane	21f69643b3	promtail: fix infinite loop in file target (fixes #5350) (#6135)	3 years ago
Periklis Tsirakidis	f91f6a354b	operator: Add support for gRPC over TLS for Loki components (#6224)	3 years ago
Jesus Vazquez	f657122ccd	Git ignore pkg/loki/wal (#6405) ... * Git ignore pkg/loki/wal I noticed that after running make test the directory pkg/loki/wal is created which a bunch of files and data. I think it would be a good idea to ignore the entire directory with git. * Trigger CLA sign	3 years ago
Jesus Vazquez	db9dbe8e1d	Bump prometheus dependency (#6403) ... * Bump prometheus dependency We had to add a replace of prometheus in go mod because thanos is importing an older version and then replacing it. * Undo textparser.New variable mismatch * modtimevfs path update * Fix labels withlabels and withoutlabels usages WithoutLabels and WithLabels methods have dissappeared. Now labels.NewBuilder has to be used. * Fix RuleManager interface for Update() Update now accepts a new param called ruleGroupPostProcessFunc which is an optional function to be executed after rule group processing. In this case I've set it to nil in the usages i've found. * NewQueryRange now accepts QueryOpts The NewQueryRange function now accepts QueryOptions so I had to update all the calls. I've set the opts to nil since we're not using them. * gofmt	3 years ago
Gerard Vanloo	5c70901c02	operator: Storage schema update (#5952)	3 years ago
Karen Miller	75e1d97d85	Docs: revise changelog entry for PR 5626 (#6406)	3 years ago
Karen Miller	ba7e36afd1	Docs: Edits for several PRs that did not have a docs review (#6398)	3 years ago
Michel Hollands	20db4df542	Add more logging for delete requests (#6394) ... * Add more logging for delete requests Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix typo Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rephrase logging message and add chunk ID Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename userID field to user Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * rename more userID Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix existing log messages Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Michel Hollands	303856ef7e	Add deleted lines panel to deletion dashboard (#6390) ... * Add deleted lines panel to deletion dashboard Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix missing comma Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use cluster variable Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Run make loki-mixin Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Alan Conway	962d081cf0	Add lokistack prefix to role and rolebinding names. (#6397)	3 years ago
Karen Miller	10283d9bc1	Docs: edit the CHANGELOG (#6386) ... * Docs: edit the CHANGELOG * Resolve conflicts caused by 2 new changelog entries. * Add missing characters of new changelog entry.	3 years ago
Karen Miller	1ab642f970	Docs: add unpopulated v2.6 release notes section (#6354)	3 years ago
Michel Hollands	671afe62ba	Use correct counter when deleting lines (#6376) ... Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Christian Haudum	c12a1f4f43	feature: Add ingester handler for shutdown and forget tokens (#6179) ... * Update grafana/dskit to 07166f9 Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add /ingester/shutdown handler This handler replaces the deprecated /ingester/flush_shutdown handler and can be used to gracefully shut down a Loki instance and delete the file that persists the tokens of the ingester ring. In production environments you usually want to persist ring tokens so that during a restart of an ingester instance, or during rollout, the tokens from that instance are not re-distributed to other instances, but instead kept so that the same streams end up on the same instance once it is up and running again. For that, the tokens are written to a file that can be specified via the `-ingester.tokens-file-path` argument. In certain cases, however, you want to forget the tokens and re-distribute them when shutting down an ingester instance. This was already possible by calling `/ingester/flush_shutdown`, deleting the tokens file and terminating the process. The new handler `/ingester/shutdown` combines these manual steps into a single handler. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add changelog entry Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Hervé Nicol	50533cbd51	Mixins templatize clusterlabel fixes (#6383) ... * fix formating * re-apply https://github.com/grafana/loki/pull/6353 * promtail-mixin: gitignore vendor/ * promtail-mixin: support for config * mixins: improve config files * re-trigger build Co-authored-by: Herve Nicol <12008875+hervenicol@users.noreply.github.com>	3 years ago
Travis Patterson	efda5b2af7	Don't interupt the query path if deletes aren't available (#6368)	3 years ago
Kaviraj Kanagaraj	01481cca0c	Promtail: Support all `cri-o` tags (multiline tags). (#6177) ... * Promtail: Support `cri-o` tags. Hacked something together that works during flight to Grafanafest. * Fix newCri test cases Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add test for real "panic" stacktrace Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add bounded size for partial lines slice Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Leave the promtail config alone Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks: Avoid discarding lines when partial lines reaches upper bound. Instead merge into single full line and print warning. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Danny Kopping	a649e0d7c4	Naïve fix for optimize bug with `line_format` and `json` expression parser (#6375) ... * Naïve fix Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * CHANGELOG Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Update CHANGELOG.md Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Mathew Heard	771e218b58	JSON label extraction field validation expression overly strict (#6372) ... * Extend json label extraction test * Add json label extraction to aggregation * fix lexeer issue, it's value to have numbers inside the field (just not at the start) Also simplified logic * integrate feedback from reviews	3 years ago
Kaviraj Kanagaraj	116d61e198	chore(jsonnet-lint): Make `promtail-mixin` depends on `loki-mixin` `main` version (#6379) ... We had to change it to some older version to revert this PR https://github.com/grafana/loki/pull/6377 Issue, context and long-term fix are explained in this https://github.com/grafana/loki/issues/6378 Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Kaviraj Kanagaraj	666bac28d9	Revert "Loki Prometheus mixin: templatize cluster label" (#6377) ... * Revert "Loki Prometheus mixin: templatize cluster label (#6353)" This reverts commit 459036f9f9. * Replace `loki-mixin` dependency to working commit. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Hervé Nicol	459036f9f9	Loki Prometheus mixin: templatize cluster label (#6353)	3 years ago
Trevor Wood	aed11c25e6	fix(loki): allow global and per tenant sigv4 config (#6358) ... Signed-off-by: Trevor Wood <Trevor.G.Wood@gmail.com>	3 years ago
Travis Patterson	87d04d58a0	Add compactor route for the cache gen number api (#6365)	3 years ago
Wu Zhiyuan	f61cbe4288	GCS: Add service_account option (#6366) ... * gcs client: add service_account support * Update docs for GCS's service_account * Fixed lint check failed	3 years ago
Rutger	1794a76613	Promtail: Add metrics for journal target (#6105) ... * Add metrics for journal target There are no metrics for the journal target yet, this adds two basic ones: - Total number of lines processed - Total number of errors processing lines (with a reason) Because of the way the journal works, tailing one file with many possible process writing to it, I chose note to expose counts per unit. * Add changelog entry * Fix compilation of (unsupported) journalmanager target on windows * Extract possible errors to constants & add tests	3 years ago