apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Sas Swart	c80244a3e4	Parse duration expressions in accordance with promql (#5275 ) * Add support for Prometheus like duration units in logql * Remove redundant type conversion * deduplicate call to sb.String Add a comment to explain why both promql and time's ParseDuration methods are used * extract duration parsing to separate function. * Add tests to ensure promql duration units are supported * Sort duration rune comment and list of runes in ascending order of length.	3 years ago
Ed Welch	4ca1fd0adb	set match_max_concurrent true by default (#5435 ) * set match_max_concurrent true by default Signed-off-by: Edward Welch <edward.welch@grafana.com> * add note to upgrade guide, fix up some of the existing notes Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Sandeep Sukhani	ff21fb063f	jsonnet: move boltdb-shipper configs set as compactor args to yaml config (#5393 ) * move boltdb-shipper configs set as compactor args to yaml config * update changelog * update upgrade guide	3 years ago
Cyril Tovena	07e5eb33d3	Do not use WaitGroup context for StepEvaluator (#5425 )	3 years ago
Josh Risley	39e9c14611	Use output plugin grafana-loki (#5386 ) Reflects name change in v2.1.0. - If an ECS user configures `"Name": "loki"` then the native output plugin will be used and the task configuration crashes in ECS.	3 years ago
Cyril Tovena	5d8bc6129e	Correctly sets hash value for headblock iterator (#5423 )	3 years ago
Owen Diehl	137a28d82b	Adds bytes and entries to chunk metadata in tsdb (#5414 ) * adds bytes and entries to chunk metadata in tsdb * removes comment	3 years ago
Ed Welch	91b0bcf36a	remove promtail_log_entries_bytes_bucket (#5377 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Jonathan Bédard Schami	776d5c41c7	Lambda-promtail: Enhance lambda-promtail to support adding extra labels from an environment variable value (#5359 ) * updated with ability to add extra tags * updated CHANGELOG.md * simplified extra label env variable to a simple comma seperated list in the form of "name1,value1,name2,value2,..." * refactored parsing of extra labels into its own method. Added unit tests. * removed invalid test, exit early on parseExtraLabels, refactored apply extra labels code to utility method, replaced hardcoded string with const global var in module. * updated docs, terraform template and cloud formation template * updated fmt.Println to fmt.Printf with line termination * updated to validate extra labels and added unit test for extra label validation behaviour * removed unused ExtraLabel struct * updated to use the new applyExtraLabels * pull latest * updated lambda promtail docs with instructions and example on how to use extra labels.	3 years ago
Travis Patterson	d8780da454	Revert to working version of Azure client (#5413 ) * Revert "storage: update azure blobstore client to use new sdk (#5302)" This reverts commit `89b04a97b3`. * Fix Azure SDK Issue - Revert SDK to working version - Introduce patches to prevent deadlocks * make metrics global	3 years ago
Danny Kopping	00bd6cc1a2	Run `make drone` to use the new build image (#5368 ) Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Michel Hollands	d8ed87d04c	Update dskit version (#5392 ) * Update dskit version Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * update changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add debug printing to see interfaces in CI. This will be reverted Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add comment to remember to remove Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use netutil.PrivateNetworkInterfacesWithFallback for the other rings Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Move import to place indicated by linter Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove debug printing Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Determine local interfaces in tests Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rearrange import to satify linter Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update build image in CircleCI Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Change changelegog message to show customer visible changes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add doc tag and use private network interfaces in common config Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add comment about filtered out non private IP addresses Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update the documentation of the interface names settings Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Cyril Tovena	b4bb8a1a46	Fixes double metrics registration in reporting (#5407 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Owen Diehl	1837c9e0b2	Experimental TSDB index (#5376 ) * moves encoding to own pkg * decwrap impl * forks tsdb index * renames tsdb_enc * wraps all tsdb decbufs * use loki encbuf * custom chunk meta replacing ref-id for uint32 checksum * handle overlapping chunks in tsdb encoding * tsdb builder impl * tsdb querier beginnings to use modified tsdb * fixes nonexistent reference * lint/remove unused * adds tsdb testware * lint	3 years ago
Cyril Tovena	9b6f17f168	Add the total number of subqueries to query statistics (#5397 ) * Add subqueries count Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Sashank Agarwal	b8e997343d	[operator] Add degraded condition when gateway is enabled and tenants spec is nil (#5383 ) * deploy gateway when both flag, specs provided * run in degraded when invalid config provided * return nil after applying degraded condition	3 years ago
Gerard Vanloo	82897832a2	Cleaning up deploy-example-secret.sh (#5374 )	3 years ago
Travis Patterson	7378b697f8	Fix Azure issue where 404 not recognized (#5399 )	3 years ago
Sas Swart	64589d0339	Logql: Correct grouping type in pkg/logql/expr.y (#5306 ) * Correct grouping type in pkg/logql/expr.y * Use loki-build-image to build loki instead of golang image * Include goyacc and other required tools to allow Makefile to run * Remove dependencies not relevant to yacc in cmd/loki/Dockerfile * Use go install instead of go get to install goyacc in cmd/loki/Dockerfile * Add goyacc to logcli and loki-canary Dockerfiles * Rerun CI * Rerun CI * Revert "Add goyacc to logcli and loki-canary Dockerfiles" This reverts commit `0ee887c57f`. * Remove goyacc from cmd/loki/Dockerfile * Formatting of cmd/loki/Dockerfile	3 years ago
Kaviraj Kanagaraj	d970e737ab	Add more context to tailer-> ingester connect error. (#5394 ) NOTE: When client is tailing, Querier has to create grpc connections to all the ingesters. Sometimes when ingester is rolloing out and Querier unable to connect to ingester when client is tailing, then error message reported by Querier is confusing. This PR add some context to it. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Ed Welch	b3af2dbeef	Loki: include the component in metrics.go log line (#5379 ) * add a component key to our metrics.go line Signed-off-by: Edward Welch <edward.welch@grafana.com> * new computer... who dis Signed-off-by: Edward Welch <edward.welch@grafana.com> * one more lint Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Karsten Jeschkies	ec1ae42083	Verify that drone.yml was updated. (#5396 )	3 years ago
Sandeep Sukhani	e65d1b8c69	add more context to logs, improve comments for clarity in boltdb-shipper code (#5341 )	3 years ago
Cyril Tovena	4fafc9c234	Fixes a flaky hedging test. (#5389 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Sandeep Sukhani	da9b4c9369	cleanup common index set after cleaning up all the user index sets (#5372 )	3 years ago
Vladyslav Diachenko	0e518967a7	updated documentation for xk6-loki extension (#5321 ) * documented custom k6 metrics that xk6-loki extension reports for instant and range queries * Update docs/sources/clients/k6/query-scenario.md Co-authored-by: Christian Haudum <christian.haudum@gmail.com> * changed description * documented new metrics Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Cyril Tovena	d44b6892eb	Fixes memberlist usage report (#5369 ) * Fixes memberlist usage report Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes the linter and improve comment Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Karsten Jeschkies	b19ce6230a	Build and publish build image. (#5322 )	3 years ago
Cyril Tovena	21fa187fa8	Improve report usage to never fail (#5364 ) * Improve report usage to never fail Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * wait until cancelled for the service to not fail Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Cyril Tovena	bbaef790db	Add usage report into Loki. (#5361 ) * Adds leader election process Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fluke Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixes the kv typecheck * wire up the http client * Hooking into loki services, hit a bug * Add stats variable. * re-vendor dskit and improve to never fail service * Intrument Loki with the package * Add changelog entry Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes compactor test Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add configuration documentation Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/usagestats/reporter.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * Add boundary check Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add log for success report. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/usagestats/reporter.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	3 years ago
gotjosh	02416736d0	Ruler: Rule group not found API message (#5362 ) * Ruler: Rule group not found API message We copied over the ruler code as part of #5089, and it seems that at the same time we stopped depending on the object storage implementation of Cortex. Turns out, there are (slight) differences between the clients which meant that response returned by the API when we try to get a rule group that is not found had changed. This ensures that is consistent with the error assertion that we have in the code for the ruler. * Use the method from the interfaace to determine object storage * also include DeleteObject * appease the linter	3 years ago
Sandeep Sukhani	7fafe94e1a	add a ms instead of ns to end time of the rebound chunk interval (#5351 ) * add a ms instead of ns to end time of the rebound chunk interval conversion between ms and ns loses ns precision which causes us to not include all the logs * improve error when failing to rebound chunk	3 years ago
Sandeep Sukhani	0afd113c96	fix rounding of time to milliseconds (#5352 ) Current implementation of RoundToMilliseconds is broken due to improper handling of floating point operation in go. Fixing it by using simple division and modulo operators	3 years ago
Karsten Jeschkies	8e9009a7e3	Speed up `EntrySortIterator` by 20%. (#5318 ) What this PR does / why we need it: This patch replace the underlying heap of the `EntrySortIterator`. Instead of using a min or max heap the entry iterators are sorted at the beginning. Once the iteration starts only the first entry iterator is re-positioned depending on its current value. This will yield the same runtime complexity as the current heap based solution. However, since the new algorithm accesses the first element it can be cached. Thus the number of access calls to `Entry().Timestamp` are halved. ``` › benchstat before.txt after.txt name old time/op new time/op delta SortIterator/merge_sort-16 4.40ms ± 4% 4.38ms ± 3% ~ (p=0.400 n=9+10) SortIterator/sort-16 3.65ms ± 2% 2.80ms ± 3% -23.25% (p=0.000 n=9+10) ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Add an entry in the `CHANGELOG.md` about the changes.	3 years ago
Dylan Guedes	a3bacd5739	Test `LegacySample` and `LegacyLabelPair` compatibility. (#5313 ) The expected values were built from Cortex' `cortexpb.Sample` and `cortexpb.LabelPair`. They prove that `logproto.LegacyLabelPair` and `logproto.LegacySample` are not compatible with `logproto.LabelPair` and `logproto.Sample`, requiring us to keep both implementations for now.	3 years ago
Josh Arrington	d934139612	fix changelog issue reference to lambda-promtail (#5347 )	3 years ago
Sandeep Sukhani	e4d6767412	Fix apply retention issue (#5342 ) * call mark phase started/finished only when applying retention * correctly pass userID to IntervalMayHaveExpiredChunks call on retention and dletion expiry check	3 years ago
Nate Walters	ffc87055b8	Delete docs/sources/copyright directory (#5325 ) Removing copyright notice per https://raintank-corp.slack.com/archives/CH34H23HD/p1643999482151029	3 years ago
Danny Kopping	20041b74d6	Revert "Simplify code for write path in distributor (#5248 )" (#5337 ) This reverts commit `10b93cebe5`. We discovered that these changes can lead to log deduplication issues. Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Kaviraj Kanagaraj	2db309271c	Makes `tailer.droppedStreams` slice bounded. (#5334 ) * Makes `tailer.droppedStreams` slice bounded. For slow receiver, this dropped streams may cause ingester memory to keep increasing. Sometimes OOMing the ingester. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make droppedStreams slice size configurable Update tests Adding logging when slice is dropped Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add test for dropped stream Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Bryan Boreham	a50cac7674	filters: use faster regexp package (#5315 ) * regexp-filter: improve benchmark Add a few more cases based on real-world usage. Also simplify the loop to just run the number of times requested. Go benchmarks run 'N' times to reach a duration, default 1 second. Previously the benchmark was running N1000000 times, which took 7 seconds minimum for some cases. regexp filter: use modified package with optimisations See https://github.com/grafana/regexp/tree/speedup#readme Includes the following changes proposed upstream: * [regexp: allow patterns with no alternates to be one-pass](https://go-review.googlesource.com/c/go/+/353711) * [regexp: speed up onepass prefix check](https://go-review.googlesource.com/c/go/+/354909) * [regexp: handle prefix string with fold-case](https://go-review.googlesource.com/c/go/+/358756) * [regexp: avoid copying each instruction executed](https://go-review.googlesource.com/c/go/+/355789) * [regexp: allow prefix string anchored at beginning](https://go-review.googlesource.com/c/go/+/377294) * Add grafana/regexp to vendor directory	3 years ago
Owen Diehl	f598484a94	Release entryBufferPool once (#5324 ) * release entryBufferPool once * reverseEntryIterator.Next is safe to call past expiry	3 years ago
Andre Ziviani	699fffe9e5	lambda-promtail: Add ability to ingest logs from S3 (#5065 ) * Add ability to ingest logs from S3 on lambda-promtail * fix ci * fix typo * bump golang and alpine version * update changelog * add s3 permissions on terraform * use for_each instead of count * fix typo * improve function naming * add documentation and an example of a s3 file path * refact logic to identify event type * add missing iam permission to allow lambda to run inside a vpc * fix typo * allow lambda to access only specified s3 buckets * configure a default log retention policy on log group * add missing depends_on to make sure iam role is created before lambda function * update docs * fix label naming convention * fix merge conflict * use new backoff lib and update dependencies * add option to limit batch size * cache s3 client * update docs and terraform * address some feedback on PR * fix typo	3 years ago
Karen Miller	d787a0fe28	Docs: remove "regular" from string definition (#5309 )	3 years ago
Vladyslav Diachenko	f0ef4662fe	Documentation for load testing Loki with xk6-loki (#5182 ) * Add initial page for xk6-loki docs Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * added documentation for xk6-loki query scenario * Add simple installation and usage instructions Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add page about log generation with xk6-loki * Add documentation for k6 write scenario Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup: incorporate feedback from Karen Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! fixup: incorporate feedback from Karen Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup: @ppcano's suggestions * fixup: @cyriltovena's suggestions * fixup: make read scenario more generic * Apply suggestions from code review Co-authored-by: Pepe Cano <pepe@loadimpact.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Use correct URL for git clone command Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Pepe Cano <pepe@loadimpact.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	3 years ago
Christian Haudum	10b93cebe5	Simplify code for write path in distributor (#5248 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Cyril Tovena	e3b74158a5	Fixes log deduplication when mutating Labels using LogQL (#5289 ) * Changes the iterator interface to include labelshash. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve unused args and fixes build Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Implement the new hash in the memchunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add the hash in logproto Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Insert the hash from ingester-querier Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Rename LabelsHash to StreamHash Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working through fixing tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor HeapIterator into merge and sort Iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Order alphabetically when ordering samples. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Do not JSON encode the hash for legacy model. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more regression tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update CHANGELOG.md Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback	3 years ago
Owen Diehl	2af3ca077e	Logql/parallel binop (#5317 ) * adds justification for keeping Downstreamer parallelism * loads binop legs in parallel * increases downstreamer default concurrency * astmapper spanlogger * always clone expr during mapping to prevent mutability bugs * Revert "astmapper spanlogger" This reverts commit `23f6b55c14`. * cleanup + use errgroup	3 years ago
Trevor Whitney	c477e29c3f	Add tooling for local k3d clusters under tools/dev (#5272 ) * add k3d folder with loki-distributed env * add readme * shellcheck create_cluster.sh * lint jsonnet * Update grafana image to latest	3 years ago
Vladyslav Diachenko	ea37f4595d	added upgrade notes (#5311 )	3 years ago

1 2 3 4 5 ...

3115 Commits (c80244a3e42ff99566d4f41e7c7526fee6c6f270) All Branches Search

3115 Commits (c80244a3e42ff99566d4f41e7c7526fee6c6f270)

All Branches