3098 Commits (7378b697f8dc05762e45d4a243005dad27d93d35)

 

All Branches   

Search

Author	SHA1	Message	Date
Dylan Guedes	ad3c8d07b7	Cleanup `util/validation` code. (#5285) ... * Cleanup `util/validation` code. * Remove `errors.go` file.	3 years ago
Ed Welch	efffe0ce50	add some timing to the logs for downloading and opening index tables (#5042) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Christian Haudum	57b384df8f	Auto-expire old items from FIFO cache (#5148) ... * Remove validity from FIFO cache We can remove the validity setting of 1h in 2.4 for the chunks cache because it doesn't reduce memory usage and instead leads to valid chunks being ignore when querying the cache. Chunks are immutable so it doesn't make sense to not return them if they are present in the cache. Closes #4922 Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add periodic task to prune old FIFO cache elements As described in #4921 this PR adds a periodic task that prunes expired items from the FIFO cache to free up memory. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Jennifer Villa	7e372cbc56	Adding a docs page that helps users understand memberlist a bit more (#5186) ... * Added some new docs on memberlist. * Added a page with more memberlist info.	3 years ago
Trevor Whitney	926a0b2a4e	Update local configs to fix issues with incorrect addresses (#5207) ... * Update local configs to fix issues with incorrect addresses * Add back instance_addr for local config	3 years ago
Dan Poltawski	379ca6b409	Promtail docs: Fix link to `unpack` LogQL documentation (#5274)	3 years ago
Christian Haudum	8d2b3f7aa5	Return early if push payload does not contain data (#5229) ... Optimize for: * payload does not contain any streams * a stream does not contain any entries * none of the streams contain any entries Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Kaviraj Kanagaraj	531905c066	Remove `cortex` references from `go.mod` and makefile (#5269) ... * Remove cortex references from `go.mod` and makefile Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove from `modules.txt` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Dylan Guedes	7856db62bb	Copy `cortex/pkg/storage/bucket` package dependency into Loki (#5258) ... * Fork cortex `storage/bucket` into Loki. * Replace storage/bucket usage from Cortex to Loki one. * Remove redundant error check. * Revert SwiftConfig struct and remove cortex pkgs alias. * Add missing test files. * Replace `cortex/pkg/util` with our own implementation. * Remove unused cortex packages.	3 years ago
Kaviraj Kanagaraj	858f8ea54a	Fix transitive dependency failure while building logstash plugin (#5268) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Lydia Lee	a4cfab4b84	feat(promtail): fix TargetManager.run() not exit (#5238)	3 years ago
Kaviraj Kanagaraj	f083aab09d	Remove `cortexpb` dependency from Loki packages. Use only `logproto` now. (#5264) ... * Getting rid of `cortexpb` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Get rid of `cortex/pkg/tenants` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix linter errors Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * go mod vendor Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Rename some import alias Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Ed Welch	09a5a2cee3	Logstash: Add config option which allows seting up an allowlist for labels to be mapped to Loki (#5244) ... * add a config called `include_fields` which lets you allowlist a specific set of fields which will be mapped to labels and sent to Loki. Signed-off-by: Edward Welch <edward.welch@grafana.com> * update docs Signed-off-by: Edward Welch <edward.welch@grafana.com> * restore path for Dockerfile so it works in CI Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Kaviraj Kanagaraj	f5d570177a	Fork `cortex/pkg/util` into Loki (#5257) ... * Fork `cortex/pkg/util/validation` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fork `pkg/cortex/util/limiter` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * fork util/extract Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Spanlogger add more helpers Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * getting rid of `cortex/pkg/util` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * With lesser linter errors Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linter happy by removing unused method Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove missed util_log Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix gofmt and goimports Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Do not touch operator Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Do not touch generated go files Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Format Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linters happy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix lambda promtail tools Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Upgrade Dockerfile of lambda promtail Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add tests for `pkg/util` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Michel Hollands	87edab0867	Ingester: remove the labelFilter (#5262) ... * Remove the labelFilter Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Sandeep Sukhani	37fcbf9428	use default retention period to check user index may have expired chunks when user does not have custom retention (#5261)	3 years ago
Cyril Tovena	cde9a711f3	fluent-bit: Attempt to unmarshal nested json. (#5223) ... * fluent-bit: Attempt to unmarshal nested json. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Also support nested json array. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Owen Diehl	91d837e79c	Refactor/remove global splitby (#5243) ... * cleans up internal limits struct, removes unnecessary logic * moves querier.split-queries-by-interval to limits, simplifies resulting code. * docs, changelog * flag supports model.duration instead of time.duration * lint and adds new error for deprecated field * reregisters queryrange flags * comment lint * ensures we validate queryrange config and simplifies embedded struct validate() * 0 still disables * simplify cache key generation and protect against divide by zero	3 years ago
Dylan Guedes	3d135e5338	Copy `cortex/pkg/configs` package dependency into Loki (#5139) ... * Fork cortex `configs` into Loki. * Use new forked `configs` package. * Remove duplicated import. * Fix lint issues. * Use new forked util/logger. * Drop support for promql rules v1. - This allow us to remove the `legacy_promql` implementation - If a rulesconfig is received in v1 format, we output a specific error, so we can differentiate between a unknown version and a v1 one. * Modify tests to expect error for legacy rules format. * Add changelog and upgrading guide. - Added them as we dropped support for legacy promql rules configuration which might be a breaking change for some environments. * Update vendor/modules after rebasing.	3 years ago
Dylan Guedes	4a70f8ec84	Copy `cortex/pkg/ingester/client` package dependency into Loki (#5164) ... * Fork ingester client. * Document magic numbers.	3 years ago
Dylan Guedes	ba6a2fd41e	Get rid of Cortex chunk. (#5239)	3 years ago
3JIou_from_home	863408684c	Push: add deflate compression in post requests (#5249) ... * deflate-compression-5247 - add deflate compression in post requests * deflate-compression-5247 - add deflate tests by example gzip	3 years ago
Eng Zer Jun	392fde1e27	refactor: use `T.TempDir` to create temporary test directory (#5253) ... * refactor: use `T.TempDir` to create temporary test directory The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete. Reference: https://pkg.go.dev/testing#T.TempDir Signed-off-by: Eng Zer Jun <engzerjun@gmail.com> * docs: update CHANGELOG.md Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	3 years ago
Sandeep Sukhani	9fdcacff15	v12 chunk format handling in retention (#5254) ... Also add some tests and safety mechanism to avoid dropping whole table when we cant find any chunks	3 years ago
Travis Patterson	8646436b18	Fix docker plugin build (#5240) ... - Rebuild the rootfs before each plugin build	3 years ago
Michel Hollands	a3fcd3cc75	Update Go version to latest and bump build image version (#5250) ... Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Florian Klink	ca2132d7cb	Update _index.md (#5241) ... We also fixes typo.	3 years ago
Roger Steneteg	bc88f298cb	skip glob search if path is an existing file and not a directory (#5246) ... Signed-off-by: Roger Steneteg <rsteneteg@ea.com>	3 years ago
RangerCD	87cbe50b59	Lock consistencyMtx outside locking chunkMtx (#5210) ... * Lock consistencyMtx outside locking chunkMtx * Remove unnecessary RLock	3 years ago
Nabeel Saad	d0c6e3d016	Update _index.md (#5159) ... Adding default value for line_format so customers know which is picked up if the option is not provided.	3 years ago
RangerCD	4ee686c22c	Remove ErrStreamMissing (#5234)	3 years ago
JordanRushing	849f45ea8c	Update loki-mixin gcs queries in the Loki operational dashboard (#5232) ... Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com>	3 years ago
Callum Styan	bfef7bad44	Start refactor of storage (chunks) clients metrics into structs (#5057) ... * Refactor azure metrics to be a struct (+ add an overall client metrics struct), add an azure client metric for total egress bytes. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add changelog entry for new azure metric. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fixes based on Kavi's review. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix one more merge conflict that I missed in the merge commit. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix failing tests. Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Kaviraj Kanagaraj	7713ff78e2	Fix `cortexpb` -> `logproto` rename in some tests (#5231) ... * Fix `cortexpg` -> `logproto` rename in some tests Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix TimestampMs rename Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * queryrange test Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix tests on queryrangebase pacakage\ Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix some typos Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Callum Styan	f24b84c15a	[querier] Abort a query in limitedRoundTripper if parallelism is 0 (#5190) ... * Abort a query in limitedRoundTripper if parallelism is 0. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix failing tests related to fakeLimits change. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Change limitedRoundTripper parallelism error message. Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Cyril Tovena	fa25e54e26	Bump logstash plugin gem (#5230) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Karsten Jeschkies	4716eea0db	Provide Docker target and discovery in Promtail. (#4911) ... **What this PR does / why we need it**: This patch adds support to fetch Docker container logs through the Docker daemon API. This should be more robust than the Loki Docker driver or scraping the logs files. The new Docker target will also collect meta information of the scraped containers. **Which issue(s) this PR fixes**: Addresses #2361 Closes #4703 **Special notes for your reviewer**: **Checklist** - [x] Documentation added - [x] Tests updated - [x] Add an entry in the `CHANGELOG.md` about the changes.	3 years ago
Christian Haudum	2e6e050453	Improve docs on how to install logcli from source (#5126) ... * Improve docs on how to install logcli from source Closes #5108 Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Improve docs on how to install logcli from source Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Kaviraj Kanagaraj	db283da088	Add `MaxSurge` and `MaxUnavailable` strategy to all Loki k8 workloads. (#5227) ... * Add `MaxSurge` and `MaxUnavailable` strategy to all loki k8 workloads. This fixes couple of issues. 1. By default these configs are 25% in k8, meaning during rollout 25% of pods are allowed to shutdown immediately. 2. Due to (1), during graceful shutdown process, 25% of all the pods access consul to `unregister()` from shared key value. (2) makes CAS rate of underlying KV store high (leads to lots of retry and failing) sometimes failing to unregister leaving the ring "unhealthy" Also this PR make these configs consistent across all k8 workloads. More details: https://github.com/grafana/dskit/issues/117 * Remove it from statefulset workloads Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Dylan Guedes	3f496e7336	Copy `cortex/pkg/cortexpb` package dependency into Loki (#5151) ... * Fork cortexpb into Cortex `pkg/logproto`. * Use `pkg/logproto` instead of Cortex `pkg/cortexpb`. * Rename `cortexpb` usage to `logproto`. * Fix encoding/decoding test. * Replace cortexpb usage in the ruler/base package. * Remove exemplars from logproto.	3 years ago
Dylan Guedes	3f78d876ff	Copy `cortex/pkg/prom1` package dependency into Loki (#5196) ... * Fork cortex `prom1`. * Use forked `prom1` instead of Cortex one.	3 years ago
aengusrooneygrafana	c9d49cea6f	Update to Unofficial clients for JS (#5180) ... Came across these packages for JS with help from Danny - I thought they would be useful to include in our docs for posterity. CC @dannykopping for info!	3 years ago
Susana Ferreira	50ca4d5186	Addition of cortex' queryrange tests (#5183)	3 years ago
Trevor Whitney	99e046c42f	Default max_outstanding_per_tenant to 2048 (#5204) ... * Default max_outstanding_per_tenant to 2048 * Update changelog	3 years ago
李国忠	d88d20e44f	[instrument] cache: add tracing instrumentation when error !=nil (#5202)	3 years ago
Kaviraj Kanagaraj	0913ea5bf3	Update Loki Team members in the Governance doc. (#5219) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Sandeep Sukhani	0426870f3f	fix broken boltdb-shipper test (#5222)	3 years ago
Sandeep Sukhani	cd1537d06e	step align start and end time of the original query while splitting it (#5217) ... * step align start and end time of the original query while splitting it * also step align queries with step > split interval	3 years ago
Sandeep Sukhani	d87f2a1c8a	per user index changes on the write path (#5193) ... * per user index changes on the write path - add a flag to enable building per user index from the ingesters - when enabled, ingesters would build boltdb files with buckets per tenant	3 years ago
Dylan Guedes	61ff29a574	Copy `cortex/pkg/chunk/purger` package dependency into Loki (#5169) ... * Fork cortex `pkg/chunk/purger`. * Replace `chunk/purger` usage. - Move away from Cortex implementation to instead use the new forked Loki implementation. * Remove unused code from `storage/chunk/purger`. * Remove unused vendored dependencies. * Remove `chunk/purger` implementation. - Loki already has the `compactor/deletion`, so the `chunk/purger` isn't really used.	3 years ago