apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Vladyslav Diachenko	b494713106	fixed typo in API host (#6892 )	3 years ago
Vladyslav Diachenko	bec014dd86	Logql analyzer page cherry picks (#6889 ) * added Logql Analyzer page into loki docs * changed template to support changes in API and supported the case when query is run against empty line * added more json examples, changed some labels and changes css for line numbers * fixed last comments from Karen and pointed to production API	3 years ago
Sandeep Sukhani	3c5fdb749a	add metric for measuring overall index query latency (#6880 )	3 years ago
Owen Diehl	3cba86faf2	[bugfix] fixes issues sharding series by storageref vs fingerprint in stripeSeries (#6882 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Travis Patterson	3170edf3e3	Add timeouts to deletions (#6879 ) * Add context to chunk iteration * Configure timeouts * add expiration timeout * update test * update test * Clarify timeout implications, log more specifically	3 years ago
Travis Patterson	afedabf0e9	Remove unused code (#6872 )	3 years ago
Karen Miller	8ac08e65d9	Docs: improve wording and grammar (#6861 )	3 years ago
Travis Patterson	d602c1331a	Clarify delete upgrade path (#6871 ) * Clarify delete upgrade path * more specificity	3 years ago
Travis Patterson	983ab80e7c	Add delete api validations (#6860 ) * break out the middleware * Add validation to the API - Time must be in RFC3339 or a 10-digit-unix-seconds timestamp - Start time must always exist * lint * docs * improve tests * cleanup * cleanup * access runtime config via function on validation	3 years ago
Owen Diehl	b114dc93d6	update deletion docs (#6869 ) * update deletion docs Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * Update docs/sources/operations/storage/logs-deletion.md	3 years ago
Michel Hollands	7c680e4ce0	update dskit and fix (#6868 ) * update dskit and fix Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use a separate registry to avoid duplicate registration Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix runtime config initialization Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * run go mod tidy Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update vendor/modules.txt Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Travis Patterson	bd3d80ad80	Add ability to limit the impact of pathological delete requests. (#6854 ) * make per-tenant override the mode rather than binary on/off * Rely on overrides to configure deletion - Remove 'deletion_mode' flag * add ability to delete requests at any time * don't load requests from disabled tenants * review comments * fix tests * loadDeleteRequestsToProcess cleanup * lint * docs * review feedback * lint	3 years ago
Trevor Whitney	f4478490db	Existing limits were low for large envs (#6846 ) * Existing limits were low for large envs * fix distributor libsonnet syntax	3 years ago
Sandeep Sukhani	11a34fb48c	avoid creating multiple tenant heads in tsdb during a race (#6866 ) It causes us to have same ref ids for different streams since the refid in each head starts with 1	3 years ago
Kaviraj Kanagaraj	e8b44172d8	test(promtail): Fix flaky test with promtail bind port. (#6859 ) * test(promtail): Fix flaky test with promtail bind port. You cannot run promtail tests if there is already running promtail in your local. e.g: ``` $ go test -run TestPromtail ./clients/pkg/promtail/ ``` It gives you error ``` --- FAIL: TestPromtail (0.02s) promtail_test.go:112: error creating promtail listen tcp 127.0.0.1:9080: bind: address already in use FAIL FAIL github.com/grafana/loki/clients/pkg/promtail 0.043s ``` Cause for the error is, test uses static default port (the same we use for promtail normally) in tests. This is not too annonying if seen in isolation. But mainly annoying in two cases particularly 1. Hard to run tests parallely in general with these kind of static ports 2. These are causing some real problems in some integeration tests on GEL which starts docker-compose to run some test suites and even stopping containers won't release ports soon enough making CI fail randomly. This PR takes advantage of Go idiom of passing `port=0` to http (and also grpc) servers enforce Go's `net` package to bind some unused random ports. Making each test run choose random ports enabling easy running theses tests in parallel. NOTE: This PR also contains upgrade in `common/server` package that made this fix possible. https://github.com/weaveworks/common/pull/249 Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * go mod tidy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Callum Styan	e372383f0d	don't deploy consul if we're using memberlist and not actively migrating (#6836 ) from consul to memberlist Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Kaviraj Kanagaraj	a6bbe8a298	chore(makefile): Remove `NETGO` check after building loki binaries. (#6856 ) Encountered after upgrade to go1.19 in local breaks this assumption. Meaning, even if your loki binary built with `-tags netgo` that file `cgo_stub.go` doesn't exist in the binary. And any make <binary> keep failing. ``` \nYour go standard library was built without the 'netgo' build tag. To fix that, run sudo go clean -i net sudo go install -tags netgo std make: *** [Makefile:149: cmd/loki/loki] Error 1 ``` But even those commands not helping. Looks like this check was added by Tom at 2018. tbh, checking for filename inside the binary is too much internal details that is subjected to change (which is what happened here I think). NOTE: Had some internal discussions and confirmed that we don't need this check in recent Go versions anymore. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Kaviraj Kanagaraj	7e3d2433ed	embedded-cache: Bring `fifocache` and `groupcache` into single tent. (#6821 ) * chore(groupcache): Rename groupcache into memcache{distributed: true} 1. Introduced new config called memorycache config. 2. It has flag `distributed`. By enabling it, we use groupcache Going forward. We will bring fifocache into memcache config(with distributed:false) This PR is just a POC. To show how we can simplify in-built memory cache config and bring it into single config. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Memorycache -> EmbeddedCache * Bring fifocache into embeddedcache tent Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix edge case with enabling fifocache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix missing flags for embedded cache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add changelog Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Warning message for fifocache config Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linter happy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix listenport issue Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * idk. review remarks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks 1. update changelog and upgrade guide 2. s/Embeddedcache/EmbeddedCache/g 3. /groupcache/ handler -> /embedded-cache/ 4. some typos 5. Fix some cache prefixes Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fixing corresponding tests Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Support extra timeouts on embedded-cache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove unwanted single struct field Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * fix(stats-collector): Let cache.New() wrap stats collector for distributed cache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Deprecate max_size_items from fifocache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks about the flag description Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Update docs/sources/upgrading/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Alexandre de Sá	6b47aab2de	promtail/client: add bot fields (#6828 ) * chore: add bot fields * Update CHANGELOG.md * chore: update cloudflare configuration section URI * docs: include BotScore and BotScoreSrc to all set of fields * Update CHANGELOG.md	3 years ago
Owen Diehl	001a627f67	fix context propagation for child spans in shard resolver (#6853 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Owen Diehl	57cec8e496	[TSDB] Ingester Index Sampling (#6852 ) * queryrangebase definitions protos & aliases Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * indexgatewaypb to logproto Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * updates queryrange.proto for new definitions file Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * s/indexgatewaypb/logproto/g Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * renames to indexseries for clobbering reasons Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * more indexgatewaypb removal compatibility work Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * proto fmt Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * rebuild imports Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * goimports -s Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * query ingesters for index stats Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * compatibility during rollouts for idx-gw+ingester-querier post proto refactor Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * proto compatibility with old indexgateway pkg path Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * test compat Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * gofmt Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Travis Patterson	bab049ca6f	Only process deletes in batches (#6851 ) * Only process deletes in batches * better logging	3 years ago
Sandeep Sukhani	7c66a363f7	Delete requests processing improvements (#6837 ) * return early while checking whether a chunk is deleted if user does not have any delete requests * track oldest start and latest end time of delete requests for each user * add some tests * Update pkg/storage/stores/indexshipper/compactor/deletion/delete_requests_manager.go Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Owen Diehl	f9d78e1748	[TSDB] Indexgateway Protobuf refactor (#6841 ) * queryrangebase definitions protos & aliases Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * indexgatewaypb to logproto Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * updates queryrange.proto for new definitions file Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * s/indexgatewaypb/logproto/g Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * renames to indexseries for clobbering reasons Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * more indexgatewaypb removal compatibility work Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * proto fmt Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * rebuild imports Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * goimports -s Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Periklis Tsirakidis	d1365c17ab	operator: Use mod readonly when building container (#6843 )	3 years ago
Christian Simon	19c7315bcc	Improve error message when max error message is hit (#6834 ) before: `entry with timestamp 2022-08-02 12:32:37 +0000 UTC ignored, reason: 'entry too far behind' for stream: {foo="bar2"}` after: `entry with timestamp 2022-08-02 12:32:37 +0000 UTC ignored, reason: 'entry too far behind, oldest acceptable timestamp is: 2022-08-03T15:25:57Z' for stream: {foo="bar2"}`	3 years ago
Danny Kopping	c63928e20f	Support building against Go 1.19 (#6833 ) * Support building against Go 1.19 Fixes https://github.com/grafana/loki/issues/6831 Hat-tip to @rfratto in https://github.com/grafana/agent/pull/1985 Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * go mod tidy Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Travis Patterson	3054a0ead5	Add groupcache timeouts (#6808 ) * Add groupcache timeouts * Made groupcache timeouts configurable	3 years ago
Gerard Vanloo	9cab665055	Adding properties file (#6799 )	3 years ago
Kaviraj Kanagaraj	7b89f20378	fix(logql): Make `LabelSampleExtractor` work as expected when label doesn't exist in logline (#6766 ) * fix(logql): Make `StringLabelfilter` work as expected Consider Log lines ``` msg=hello subqueries=5 msg=hello subqueries=0 msg=hello ``` If you run following logql ``` {foo="bar"}\|logfmt\|subqueries!=0 ``` it returns ``` msg=hello subqueries=5 ``` But if you run following logql ``` {foo="bar"}\|logfmt\|subqueries!="0" # NOTE: "0" as string. ``` it returns ``` msg=hello subqueries=5 msg=hello ``` NOTE: it also returns log lines that doesn't contain `subqueries` label in the first place. This cause subtle error if used in metric queries using that `label` in unwrap. e.g: following logql on above log lines returns ``` sum_over_time({foo="bar"}\|logfmt\|subqueries!="0"\|unwrap subqueries[1m]) ``` Returns `pipeline error: 'SampleExtractionErr'`. Because, the lines without `subqueries` labels are not skipped during `ProcessLine` and during metric extraction, our extractor sets this `SampleExtractionerror`. Fixes: https://github.com/grafana/loki/issues/6713 * Add it on changelog Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix linter Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add more tests and fix the edge case with equality to empty strings Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * typo on the comment Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * handle this edge case during metric extraction. Not in label filter Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix linter Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove commented code Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Update changelog Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Karen Miller	0313f8648f	Change existing labeler action to use type/docs label (#6754 )	3 years ago
Pablo	c9ef0621d1	Promtail: Add new target for receiving GCP PubSub Push messages (#6777 )	3 years ago
Christian Simon	49a6c03c09	Treat version flag at any position (#6827 ) The behaviour was changed as part of #6802. This should restore the previous behaviour and allow the version to be displayed without config file.	3 years ago
Danny Kopping	66fdb3c572	Log stream flushes to see how many chunks we flush at a time (#6819 ) * Log stream flushes to see how many chunks we flush at a time Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Smuggling in an error wrapping which aides debugging Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Update docs/sources/upgrading/_index.md Co-authored-by: Ed Welch <ed@oqqer.com> * Removing unnecessary label Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Ed Welch <ed@oqqer.com>	3 years ago
Kaviraj Kanagaraj	1c6a2410d6	doc(retention): Add clarity on default behavior. (#6815 ) * doc(retention): Add clarity on default behavior. Currently if you read through the entire doc, it's not super clear what happens if I don't enable retention. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * update Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Owen Diehl	46ae0efc0f	Simplify shard membership calculations (#6810 )	3 years ago
Kaviraj Kanagaraj	9f06bc0083	chore(groupcache): Groupcache doesn't need global server handler anymore (#6820 ) Remove it from the constructor Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Julien Duchesne	7b70d3d939	deb and rpm packaging for all binaries (#6456 ) * deb and rpm packaging for all binaries There is currently deb and rpm packages being published for logcli amd64 but they aren't signed In this PR: - Add RPM and deb packages for `logcli`, `loki-canary` `loki` and `promtail` for arm, arm64 and amd64 - Sign them with the Grafana GPG key (packages.grafana.com/gpg.key). This will be the same key for all Grafana products - Add CI steps to test the RPM and deb packages. This launches a docker image with systemd installed, then it installs both loki and promtail and does a `logcli` query on the Loki instance * Remove specific version in test scripts * Bump build image to 0.22.0 * Add restart policy for promtail's service	3 years ago
Danny Kopping	c5daa31b65	Remove reference to non-existent config field (#6781 ) Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Karsten Jeschkies	51b09d771f	Open access to Minio console in dev setup. (#6807 )	3 years ago
Michel Hollands	03e4db46eb	Fix log messages for deletion requests (#6727 ) * Fix log messages for deletion requests Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Store deleted lines in log message instead of as a metric Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix linting issue Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Test for equal values Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Config for running with deletion mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add back per user counter of deleted lines Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Revert example config changes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Change test name Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Gyanesh Malhotra	3a0b982fe2	Run `loki -version` without needing a config defined (#6802 ) * fix version. * address pr comment.	3 years ago
Callum Styan	a53ba0bb04	populate the memberlist label by default (#6800 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Callum Styan	4030c12e17	only set the consul kvstore configs if memberlist is not being used (#6809 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Travis Patterson	3bbc5a92f1	Allow fifocache to be configured with groupcache (#6806 )	3 years ago
Kaviraj Kanagaraj	acc3a2fe23	chore(groupcache): Fix panic with stats collector wrapper (#6798 ) * chore(groupcache): Fix panic with stats collector wrapper Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * update Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
yngwiewang	de288ccbb8	docs: fix a typo (#6791 )	3 years ago
Dylan Guedes	27b9fe50f0	Fix Promtail dashboard. (#6796 ) Fix Promtail production dashboard/panels. Error message was `bad_data: 1:40: parse error: unexpected character inside braces: '~'`.	3 years ago
Gerard Vanloo	ebc13669c0	operator: Changing channel to stable (#6795 )	3 years ago
Sandeep Sukhani	2e95a64cb4	query-tee: support for multi backend response comparison and disabling query proxy for select backends (#6736 ) * add support for comparing responses with multiple secondary backends * allow disabling read proxy for select secondary backends * Update tools/querytee/proxy.go Co-authored-by: Salva Corts <salva.corts@grafana.com> Co-authored-by: Salva Corts <salva.corts@grafana.com>	3 years ago

1 2 3 4 5 ...

3793 Commits (b494713106f8ac5a7af50a7aeadc01fc24d5f058) All Branches Search

3793 Commits (b494713106f8ac5a7af50a7aeadc01fc24d5f058)

All Branches