3098 Commits (7378b697f8dc05762e45d4a243005dad27d93d35)

 

All Branches   

Search

Author	SHA1	Message	Date
Travis Patterson	7378b697f8	Fix Azure issue where 404 not recognized (#5399)	3 years ago
Sas Swart	64589d0339	Logql: Correct grouping type in pkg/logql/expr.y (#5306) ... * Correct grouping type in pkg/logql/expr.y * Use loki-build-image to build loki instead of golang image * Include goyacc and other required tools to allow Makefile to run * Remove dependencies not relevant to yacc in cmd/loki/Dockerfile * Use go install instead of go get to install goyacc in cmd/loki/Dockerfile * Add goyacc to logcli and loki-canary Dockerfiles * Rerun CI * Rerun CI * Revert "Add goyacc to logcli and loki-canary Dockerfiles" This reverts commit 0ee887c57f. * Remove goyacc from cmd/loki/Dockerfile * Formatting of cmd/loki/Dockerfile	3 years ago
Kaviraj Kanagaraj	d970e737ab	Add more context to tailer-> ingester connect error. (#5394) ... NOTE: When client is tailing, Querier **has** to create grpc connections to **all** the ingesters. Sometimes when ingester is rolloing out and Querier unable to connect to ingester when client is tailing, then error message reported by Querier is confusing. This PR add some context to it. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Ed Welch	b3af2dbeef	Loki: include the component in metrics.go log line (#5379) ... * add a component key to our metrics.go line Signed-off-by: Edward Welch <edward.welch@grafana.com> * new computer... who dis Signed-off-by: Edward Welch <edward.welch@grafana.com> * one more lint Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Karsten Jeschkies	ec1ae42083	Verify that drone.yml was updated. (#5396)	3 years ago
Sandeep Sukhani	e65d1b8c69	add more context to logs, improve comments for clarity in boltdb-shipper code (#5341)	3 years ago
Cyril Tovena	4fafc9c234	Fixes a flaky hedging test. (#5389) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Sandeep Sukhani	da9b4c9369	cleanup common index set after cleaning up all the user index sets (#5372)	3 years ago
Vladyslav Diachenko	0e518967a7	updated documentation for xk6-loki extension (#5321) ... * documented custom k6 metrics that xk6-loki extension reports for instant and range queries * Update docs/sources/clients/k6/query-scenario.md Co-authored-by: Christian Haudum <christian.haudum@gmail.com> * changed description * documented new metrics Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Cyril Tovena	d44b6892eb	Fixes memberlist usage report (#5369) ... * Fixes memberlist usage report Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes the linter and improve comment Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Karsten Jeschkies	b19ce6230a	Build and publish build image. (#5322)	3 years ago
Cyril Tovena	21fa187fa8	Improve report usage to never fail (#5364) ... * Improve report usage to never fail Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * wait until cancelled for the service to not fail Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Cyril Tovena	bbaef790db	Add usage report into Loki. (#5361) ... * Adds leader election process Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fluke Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixes the kv typecheck * wire up the http client * Hooking into loki services, hit a bug * Add stats variable. * re-vendor dskit and improve to never fail service * Intrument Loki with the package * Add changelog entry Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes compactor test Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add configuration documentation Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/usagestats/reporter.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * Add boundary check Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add log for success report. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/usagestats/reporter.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	3 years ago
gotjosh	02416736d0	Ruler: Rule group not found API message (#5362) ... * Ruler: Rule group not found API message We copied over the ruler code as part of #5089, and it seems that at the same time we stopped depending on the object storage implementation of Cortex. Turns out, there are (slight) differences between the clients which meant that response returned by the API when we try to get a rule group that is not found had changed. This ensures that is consistent with the error assertion that we have in the code for the ruler. * Use the method from the interfaace to determine object storage * also include DeleteObject * appease the linter	3 years ago
Sandeep Sukhani	7fafe94e1a	add a ms instead of ns to end time of the rebound chunk interval (#5351) ... * add a ms instead of ns to end time of the rebound chunk interval conversion between ms and ns loses ns precision which causes us to not include all the logs * improve error when failing to rebound chunk	3 years ago
Sandeep Sukhani	0afd113c96	fix rounding of time to milliseconds (#5352) ... Current implementation of RoundToMilliseconds is broken due to improper handling of floating point operation in go. Fixing it by using simple division and modulo operators	3 years ago
Karsten Jeschkies	8e9009a7e3	Speed up `EntrySortIterator` by 20%. (#5318) ... **What this PR does / why we need it**: This patch replace the underlying heap of the `EntrySortIterator`. Instead of using a min or max heap the entry iterators are sorted at the beginning. Once the iteration starts only the first entry iterator is re-positioned depending on its current value. This will yield the same runtime complexity as the current heap based solution. However, since the new algorithm accesses the first element it can be cached. Thus the number of access calls to `Entry().Timestamp` are halved. ``` › benchstat before.txt after.txt name old time/op new time/op delta SortIterator/merge_sort-16 4.40ms ± 4% 4.38ms ± 3% ~ (p=0.400 n=9+10) SortIterator/sort-16 3.65ms ± 2% 2.80ms ± 3% -23.25% (p=0.000 n=9+10) ``` **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Add an entry in the `CHANGELOG.md` about the changes.	3 years ago
Dylan Guedes	a3bacd5739	Test `LegacySample` and `LegacyLabelPair` compatibility. (#5313) ... The expected values were built from Cortex' `cortexpb.Sample` and `cortexpb.LabelPair`. They prove that `logproto.LegacyLabelPair` and `logproto.LegacySample` are not compatible with `logproto.LabelPair` and `logproto.Sample`, requiring us to keep both implementations for now.	3 years ago
Josh Arrington	d934139612	fix changelog issue reference to lambda-promtail (#5347)	3 years ago
Sandeep Sukhani	e4d6767412	Fix apply retention issue (#5342) ... * call mark phase started/finished only when applying retention * correctly pass userID to IntervalMayHaveExpiredChunks call on retention and dletion expiry check	3 years ago
Nate Walters	ffc87055b8	Delete docs/sources/copyright directory (#5325) ... Removing copyright notice per https://raintank-corp.slack.com/archives/CH34H23HD/p1643999482151029	3 years ago
Danny Kopping	20041b74d6	Revert "Simplify code for write path in distributor (#5248)" (#5337) ... This reverts commit 10b93cebe5. We discovered that these changes can lead to log deduplication issues. Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Kaviraj Kanagaraj	2db309271c	Makes `tailer.droppedStreams` slice bounded. (#5334) ... * Makes `tailer.droppedStreams` slice bounded. For slow receiver, this dropped streams may cause ingester memory to keep increasing. Sometimes OOMing the ingester. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make droppedStreams slice size configurable Update tests Adding logging when slice is dropped Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add test for dropped stream Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Bryan Boreham	a50cac7674	filters: use faster regexp package (#5315) ... * regexp-filter: improve benchmark Add a few more cases based on real-world usage. Also simplify the loop to just run the number of times requested. Go benchmarks run 'N' times to reach a duration, default 1 second. Previously the benchmark was running N*1000000 times, which took 7 seconds minimum for some cases. * regexp filter: use modified package with optimisations See https://github.com/grafana/regexp/tree/speedup#readme Includes the following changes proposed upstream: * [regexp: allow patterns with no alternates to be one-pass](https://go-review.googlesource.com/c/go/+/353711) * [regexp: speed up onepass prefix check](https://go-review.googlesource.com/c/go/+/354909) * [regexp: handle prefix string with fold-case](https://go-review.googlesource.com/c/go/+/358756) * [regexp: avoid copying each instruction executed](https://go-review.googlesource.com/c/go/+/355789) * [regexp: allow prefix string anchored at beginning](https://go-review.googlesource.com/c/go/+/377294) * Add grafana/regexp to vendor directory	3 years ago
Owen Diehl	f598484a94	Release entryBufferPool once (#5324) ... * release entryBufferPool once * reverseEntryIterator.Next is safe to call past expiry	3 years ago
Andre Ziviani	699fffe9e5	lambda-promtail: Add ability to ingest logs from S3 (#5065) ... * Add ability to ingest logs from S3 on lambda-promtail * fix ci * fix typo * bump golang and alpine version * update changelog * add s3 permissions on terraform * use for_each instead of count * fix typo * improve function naming * add documentation and an example of a s3 file path * refact logic to identify event type * add missing iam permission to allow lambda to run inside a vpc * fix typo * allow lambda to access only specified s3 buckets * configure a default log retention policy on log group * add missing depends_on to make sure iam role is created before lambda function * update docs * fix label naming convention * fix merge conflict * use new backoff lib and update dependencies * add option to limit batch size * cache s3 client * update docs and terraform * address some feedback on PR * fix typo	3 years ago
Karen Miller	d787a0fe28	Docs: remove "regular" from string definition (#5309)	3 years ago
Vladyslav Diachenko	f0ef4662fe	Documentation for load testing Loki with xk6-loki (#5182) ... * Add initial page for xk6-loki docs Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * added documentation for xk6-loki query scenario * Add simple installation and usage instructions Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add page about log generation with xk6-loki * Add documentation for k6 write scenario Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup: incorporate feedback from Karen Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! fixup: incorporate feedback from Karen Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup: @ppcano's suggestions * fixup: @cyriltovena's suggestions * fixup: make read scenario more generic * Apply suggestions from code review Co-authored-by: Pepe Cano <pepe@loadimpact.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Use correct URL for git clone command Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Pepe Cano <pepe@loadimpact.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	3 years ago
Christian Haudum	10b93cebe5	Simplify code for write path in distributor (#5248) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Cyril Tovena	e3b74158a5	Fixes log deduplication when mutating Labels using LogQL (#5289) ... * Changes the iterator interface to include labelshash. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve unused args and fixes build Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Implement the new hash in the memchunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add the hash in logproto Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Insert the hash from ingester-querier Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Rename LabelsHash to StreamHash Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working through fixing tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor HeapIterator into merge and sort Iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Order alphabetically when ordering samples. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Do not JSON encode the hash for legacy model. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more regression tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update CHANGELOG.md Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback	3 years ago
Owen Diehl	2af3ca077e	Logql/parallel binop (#5317) ... * adds justification for keeping Downstreamer parallelism * loads binop legs in parallel * increases downstreamer default concurrency * astmapper spanlogger * always clone expr during mapping to prevent mutability bugs * Revert "astmapper spanlogger" This reverts commit 23f6b55c14. * cleanup + use errgroup	3 years ago
Trevor Whitney	c477e29c3f	Add tooling for local k3d clusters under tools/dev (#5272) ... * add k3d folder with loki-distributed env * add readme * shellcheck create_cluster.sh * lint jsonnet * Update grafana image to latest	3 years ago
Vladyslav Diachenko	ea37f4595d	added upgrade notes (#5311)	3 years ago
Susana Ferreira	82fac990a1	Update promtail to support duration string formats (#5290) ... * Update promtail to support duration string formats * Address comments	3 years ago
Vladyslav Diachenko	d4a4728933	changed logic to respond with '500 Internal Server Error' if rpc error with code 'Cancelled' happened. it's made to allow loki to retry the request to the querier instead of returning '499 The request was cancelled by the client.' back to the client. (#5297)	3 years ago
Travis Patterson	89b04a97b3	storage: update azure blobstore client to use new sdk (#5302) ... * Update azure blobstore client to use new sdk * changelog * fix hardcoded maxretries * linter * more linting * Fix flaky test	3 years ago
Travis Patterson	de1faa0671	Fix transitive dependency error (#5308) ... * Fix transitive dependency error - Don't update the gems that come with the logstash image * ci bump * ci bump	3 years ago
Owen Diehl	af3457b92e	less opaque chunk keys on fs with v12 (#5291) ... * less opaque chunk keys on fs with v12 * lint	3 years ago
Travis Patterson	1925d77f65	cleanup: consolidate util strings contain functions (#5304) ... * Consolidate util strings contain functions * fix test	3 years ago
Dylan Guedes	cf419d1cb8	Implement `LegacySample`, `LegacyLabelPair` proto messages (#5301) ... * Add LegacySample and modify protos to use it. * [Skip this commit] Regenerate protos. * Modify codebase to use LegacySample instead of Sample. For reference, take a look at PR https://github.com/grafana/loki/pull/5151. * Implement LegacyLabelPair. * Use new Legacy implementations accordingly. * Use LegacySample in compat tests. - Since `compat.go` was forked from Cortex, we can reliably change all mentions from `Sample` to `LegacySample` there.	3 years ago
Sashank Agarwal	e19c5863ea	Fix `go.mod` path of operator (#5287) ... * fix go mod path of operator * change import path in all operator files	3 years ago
Owen Diehl	a88ee16534	better logic for when to shard wrt disabled lookback (#5303)	3 years ago
Sandeep Sukhani	40e05198f9	fix a panic in index-gateway caused by double closing of a channel (#5298)	3 years ago
Christian Haudum	3fb572abda	Fix Promtail watching deadlock (#5283) ... * Reuse channel from fileEventWatcher map if already exists Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Remove sync() call in FileTarget constructor The constructor `NewFileTarget` is called from within a locked code block by the FileTargetManager. However, the constructor also calls `sync()` which starts and stops watching new directories. These methods `startWatching` and `stopWatching` both send to the `targetEventHandler` channel, causing the `FileTargetManager` to receive the event in a goroutine (`watch()` function) running in the background. Handling the event calls `sendFileCreateEvent` on the syncers, which subsequently tries to write to the `fileEventWatcher` channel which is never read because of the locked mutex. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Sashank Agarwal	c88734b72c	Link `replace` pipeline stage to doc (#5200)	3 years ago
Karsten Jeschkies	b92b544ca2	Write Promtail position file atomically. (#5266) ... Fixes #5218	3 years ago
Karsten Jeschkies	54bc2dc40e	Restart Docker target if stopped. (#5280) ... Summary: The Docker target would not reconnect to a container if it restarted and kept the same ID. This should introduces a `startIfNotRunning` method that will be called to restart scraping the container's logs. Fixes #5259	3 years ago
Cyril Tovena	3af7b7900e	Refactor HeapIterator into Merge and Sort Iterator. (#5281) ... * Refactor HeapIterator into merge and sort Iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Ed Welch	44ba39055d	logstash: re-add bundle update and update bundler (#5284) ... * update to latest logstash and bundler Signed-off-by: Edward Welch <edward.welch@grafana.com> * re-add bundle update Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Owen Diehl	96ff7f900b	Revert "Adding a docs page that helps users understand memberlist a bit more (#5186)" (#5286) ... This reverts commit 7e372cbc56.	3 years ago