3570 Commits (e3aedbcd9ed637f3fa33444d78602eb8148f8349)

 

All Branches   

Search

Author	SHA1	Message	Date
Grot (@grafanabot)	e3aedbcd9e	[k102] Backport Remove whole stream deletion mode (#6435) (#6477) (#6497) ... * Remove whole stream deletion mode (#6435) * Remove whole-stream-deletion mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove whole-stream-deletion from docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update the changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Sort changelog entries Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove link to wrong configuration Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix integration test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Set default deletion mode to disabled Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove extra white line in documentation Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix default value in docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add DeletionEnabled method on mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> (cherry picked from commit f80e487dcf) * Fix changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix changelog link Signed-off-by: Michel Hollands <michel.hollands@grafana.com> (cherry picked from commit da3fbd965b) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Grot (@grafanabot)	ca53d41af9	Compactor: add per tenant compaction delete enabled flag (#6410) (#6476) (#6496) ... * Compactor: add per tenant compaction delete enabled flag (#6410) * Add per tenant compaction delete enabled flag Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove changes in wrong place Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add compactor deletion enabled field Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use limit in compactor Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use http middleware and add test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix lint issue Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Revert to default setting if no override Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add default value command line option Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update the docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Enable access to deletion API for integration test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename flag to allow_deletes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update per review comments Signed-off-by: Michel Hollands <michel.hollands@grafana.com> (cherry picked from commit b4e6c59936) * Fix changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> (cherry picked from commit df48f9374b) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Grot (@grafanabot)	236f90d24d	Add more logging for delete requests (#6394) (#6473) (#6495) ... * Add more logging for delete requests Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix typo Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rephrase logging message and add chunk ID Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename userID field to user Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * rename more userID Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix existing log messages Signed-off-by: Michel Hollands <michel.hollands@grafana.com> (cherry picked from commit 20db4df542) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> (cherry picked from commit c707faf285)	3 years ago
Grot (@grafanabot)	0a2b760168	Use correct counter when deleting lines (#6376) (#6472) (#6494) ... Signed-off-by: Michel Hollands <michel.hollands@grafana.com> (cherry picked from commit 671afe62ba) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> (cherry picked from commit a7f8be246d)	3 years ago
Grot (@grafanabot)	565f982be5	Don't interupt the query path if deletes aren't available (#6368) (#6471) (#6493) ... (cherry picked from commit efda5b2af7) Co-authored-by: Travis Patterson <travis.patterson@grafana.com> (cherry picked from commit f79ff953a6)	3 years ago
Grot (@grafanabot)	9e26b87e50	Backport #6413 and #6377 to k102 (#6422) (#6492) ... * Backport memberlist dependency version pin to k102 Signed-off-by: JordanRushing <rushing.jordan@gmail.com> * Revert "Loki Prometheus mixin: templatize cluster label" (#6377) * Revert "Loki Prometheus mixin: templatize cluster label (#6353)" This reverts commit 459036f9f9. * Replace `loki-mixin` dependency to working commit. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> (cherry picked from commit 666bac28d9) Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> (cherry picked from commit 372961d8bd) Co-authored-by: JordanRushing <rushing.jordan@gmail.com>	3 years ago
Rutger	1794a76613	Promtail: Add metrics for journal target (#6105) ... * Add metrics for journal target There are no metrics for the journal target yet, this adds two basic ones: - Total number of lines processed - Total number of errors processing lines (with a reason) Because of the way the journal works, tailing one file with many possible process writing to it, I chose note to expose counts per unit. * Add changelog entry * Fix compilation of (unsupported) journalmanager target on windows * Extract possible errors to constants & add tests	3 years ago
Owen Diehl	9218e4654b	TSDB/query-planning-groundwork (#6367) ... * begins building queryrange compatible types for index sampling * adds MatcherGroups() method to SampleExpr * MatcherGroups test + ShardResolver interface * index stats codec wiring * millisecond precision on index stats codec * lint * cleanMatchers in tsdb.IndexClient for consistency + adds catchall matcher if nil	3 years ago
Michael Grigoryan	67ebb0430b	cmd/loki: minor consistency patch (#6364) ... * cmd/loki: minor consistency patch * cmd/loki: fixed a typo	3 years ago
Dylan Guedes	9686e4ba30	Add total number of unique users involved in query readiness. (#6035)	3 years ago
Periklis Tsirakidis	09712dfa67	operator: Allow optional installation of webhooks (#6363)	3 years ago
Periklis Tsirakidis	6205b962d0	operator: Allow reduced tenant OIDC authentication requirements (#6362)	3 years ago
Mohamed-Amine Bouqsimi	73a144e064	operator: Expose only an HTTPS gateway when in openshift-logging mode (#6288)	3 years ago
Trevor Wood	23cc93862d	chore(fluentd): resolve rubocop failures (#6359) ... Signed-off-by: Trevor Wood <Trevor.G.Wood@gmail.com>	3 years ago
Owen Diehl	3f4a663dbe	Tsdb/index sampling endpoint (#6347) ... * index sampling endpoint * fix bug in withTenantLabelMatcher * show zero values in stats * handles new errcase * fixes comments	3 years ago
Chris Hodges	6cbcd6aa76	loki-canary: Add support for client-side TLS certs for Loki connection (#6310) ... * Add support for client-side TLS certs to loki-canary * fix lint errors, make the case of using TLS but not client certs more clear, check that -tls is set if client certs are used * move changelog to canary section instead of main * wrap httpClient errors and simplify dialer creation for websocket	3 years ago
Periklis Tsirakidis	caf7dde32b	operator: Add missing rbac to get/list/watch namespaces (#6303)	3 years ago
Christian Simon	3161d995b2	Update gopkg.in/yaml.v3 (#6284) ... This updates to a version that contains the fix to CVE-2022-28948.	3 years ago
Kaviraj Kanagaraj	ad65cc29bc	Support `series|labels` query_type in `logql_query_duration` (#6341) ... So that you can do quantile calculation on metadata queries. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Christopher Norris	874aa6495e	Fix max series test loop that returns early (#6345) ... Signed-off-by: Christopher Norris <christopher.norris@grafana.com>	3 years ago
Callum Styan	97ee1b93f3	[logql] don't try to execute a modified function that is not allowed by the AST (#6346) ... * Panic with an actual error if clone (which calls range aggregation expr's validate) errors, instead of swallowing the error and then attempting to execute a nil function, which panics anyways. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Call the validation function on a modified expression before attempting to execute it. If the validation fails we should return the original expression. Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Karen Miller	e725a21b1a	Docs: Add changelog entry for configuration parameter name change (#6340) ... * Docs: Add changelog entry for configuration parameter name change * Update changelog message, per review request	3 years ago
Owen Diehl	0011c2d7b7	TSDB/index-sampling-wiring (#6338) ... * wires up index sampling * regens protos * mockstore stats impl * addtl linting * renames to avoid duplicate via linter	3 years ago
Danny Kopping	3a1fdd03f4	Define a reason why the fifocache has evicted an item (#6335) ... * Define a reason why the fifocache has evicted an item Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Fixing tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Making reason more clear Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Do not increment eviction counter on Stop() Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
JordanRushing	c9b374dd08	Set `MetricsNamespace` configurable for MemberlistKV module in Loki (#6330) ... Signed-off-by: JordanRushing <rushing.jordan@gmail.com>	3 years ago
Michał Jaroń	57d543eff7	Update _index.md - an unofficial Loki client added (#6312) ... I would like to add mjaron-tinyloki-java to unofficial Loki clients. It is standalone small library without any dependencies. It may be an alternative to other libraries which use many dependencies and require more configuration to send logs.	3 years ago
JordanRushing	405cdab96a	Expose Memberlist status page on MemberlistKV module init (#6322) ... * Expose Memberlist status page on MemberlistKV module init Signed-off-by: JordanRushing <rushing.jordan@gmail.com> * Add Server dependency to MemberlistKV module to properly serve status page Signed-off-by: JordanRushing <rushing.jordan@gmail.com>	3 years ago
JordanRushing	e0528b6f00	Add multi_kv_config block to overrides configmap when multi kv is enabled (#6323) ... Signed-off-by: JordanRushing <rushing.jordan@gmail.com>	3 years ago
Owen Diehl	5a6157ecf3	Tsdb/index sampling (#6329) ... * groundwork for index sampling * go mod * bloom filter utils * removes unnecessary atomics * Stats() impl for tsdb index * use passed var * lint	3 years ago
Sandeep Sukhani	086dbf7270	always retry syncing index files from indexshipper when the index list cache is stale (#6327)	3 years ago
Danny Kopping	36e0979cf5	Add cache usage statistics (#6317) ... * Adding cache statistics Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding metrics to metrics.go Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Creating new stats context for use in metric queries middleware Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Clean up unnecessary log fields Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Fixing tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding stats tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * CHANGELOG entry Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Documenting function Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Moving CHANGELOG entry to appropriate section Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Implementing a stats collector cache wrapper to simplify stats collection If we keep the stats collection in pkg/storage/chunk/cache/instrumented.go, then any implementation that wraps it will cause the stats collected to be incomplete. For example: NewBackground(cacheName, cfg.Background, Instrument(cacheName, cache, reg), reg)) - the background cache requests are not collected Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Fixing tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Michel Hollands	65e3148bc9	Update delete integration test (#6192) ... * Add wait until processed Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add query lines Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove Cortex reference Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add deleted lines metric and check in test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Set negative cancel period so default 1 minute delay is compensated for Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Reduce timeout Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add helper function Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Comment out assertions that only work with flush Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove unused metric type param Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Check counter in unit test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename ClientOption to Option as per the linter Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove redundant check Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * fill in MsgAndArgs field of test assertion Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Evangelos Foutras	a877febf75	Use fake credentials in storage/chunk/storage tests (#6187) ... Enable insecure mode for the GCS client so fake credentials are used when running the storage tests (see commit c0cc004333 ("gcs client: use fake credentials in unit test"). Fixes: https://github.com/grafana/loki/issues/6185	3 years ago
Richard Hartmann	c1be35eb31	Add licensing boiler plate in case we ever need it (#5251) ... * Add licensing boiler plate in case we ever need it Signed-off-by: Richard Hartmann <richih@richih.org> * Update LICENSING.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Sandeep Sukhani	82da01ad43	make index shipper read path handle range of tables by type of index (#6304)	3 years ago
Sandeep Sukhani	9a2df5a06e	copy boltdb-shipper cache changes from PR #6054 to generic index shipper (#6316)	3 years ago
Mathew Heard	de418bf6ac	Reduce to a single code path through sendBatches regardless of limit (#6216) ... * Reduce to a single code path through sendBatches regardless of limit specified * simplify sendBatches by using -1 to specify no limit * test fix	3 years ago
Shane St Savage	135f672c91	fix: allow promtail docker_sd_config to handle long log lines (#6256) ... In promtail's docker_sd_config, parsing of a container's logs would fail if any log line was over 64k bytes in length. This is because bufio.Scanner uses a 64k buffer by default. Since there doesn't appear to be a way to elegantly catch `bufio.ErrTooLong` and carry on, the parsing approach was changed to bufio.Reader which can handle lines of arbitrary length.	3 years ago
Travis Patterson	69eabfcefd	Fix delete cancellation timeout (#6285)	3 years ago
Vladyslav Diachenko	b757753e3b	fixed type of the property in memberlist config (#6307)	3 years ago
Jan-Otto Kröpke	a1e0298a57	Support tenant id from value (#6290) ... Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>	3 years ago
Vladyslav Diachenko	7df7169ccd	We shouldn't modify the loki config at all if the memberlist ring is not (#6299) (#6308) ... enabled. Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Owen Diehl	4903df6ae5	Tsdb/inverted index wiring (#6252) ... * multi inverted index * delete in inverted index signature * multi inverted index * inverted index testing + skip nil indices * wires up period config aware multi-inverted-idx through ingesters * better inverted index validation * apply shipper defaults to last relevant period config, not current one. * more verbose error	3 years ago
Travis Patterson	ea0a5246cc	Only consider an interval to have possible expired chunks if it overlaps a delete. (#6297)	3 years ago
Owen Diehl	cd7ebf2ae0	exposes line length hist on distributors (#6309) ... * exposes line length hist on distributors + fixes some metric instantiation * adjusts buckets	3 years ago
Sandeep Sukhani	1ad75d0b63	do not initialize tsdb index store when using index gateway client for queries (#6305) ... * do not initialize tsdb index store when using index gateway client for queries * fix test * lint	3 years ago
Sandeep Sukhani	7158785a89	update loki-mixin json dashboards with latest jsonnet changes (#6306)	3 years ago
Christian Haudum	9a82f5c04a	Look up config file in default locations (#6160) ... * Look for config.yaml file in ./ and ./config directories This change makes Loki look for a config file in the current directory or in the config/ sub-directory. This is a pre-requisite for making an easy-to-use tarball distribution that also ships a config.yaml Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Rename YAMLFlag to ConfigFileLoader The `YAMLFlag` function is solely used for loading config files. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Look for config.yaml file in ./ and ./config directories * fixup! fixup! Look for config.yaml file in ./ and ./config directories Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! fixup! fixup! Look for config.yaml file in ./ and ./config directories Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Cyril Tovena	7e53381eee	Avoid line copy during LogQL line_format (#6104)	3 years ago
Cyril Tovena	d9cc6a87c2	Improve write dashboard by removing unrelated route (#5853) ... * Improve write dashboard by removing unrelated route Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update production/loki-mixin/dashboards/loki-writes.libsonnet Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	3 years ago