apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Trevor Whitney	4a56445686	Upgrade `golangci-lint` and fix linting errors (#9601 ) What this PR does / why we need it: Upgrade `golangci-lint` and fixes all the errors. The upgrade includes some stricter linting.	2 years ago
Sandeep Sukhani	0a5e149ea5	query-scheduler: fix query distribution in SSD mode (#9471 ) What this PR does / why we need it: When we run the `query-scheduler` in `ring` mode, `queriers` and `query-frontend` discover the available `query-scheduler` instances using the ring. However, we have a problem when `query-schedulers` are not running in the same process as queriers and query-frontend since [we try to get the ring client interface from the scheduler instance](`abd6131bba/pkg/loki/modules.go (L358)`). This causes queries not to be spread across all the available queriers when running in SSD mode because [we point querier workers to query frontend when there is no ring client and scheduler address configured](`b05f4fced3/pkg/querier/worker_service.go (L115)`). I have fixed this issue by adding a new hidden target to initialize the ring client in `reader`/`member` mode based on which service is initializing it. `reader` mode will be used by `queriers` and `query-frontend` for discovering `query-scheduler` instances from the ring. `member` mode will be used by `query-schedulers` for registering themselves in the ring. I have also made a couple of changes not directly related to the issue but it fixes some problems: * [reset metric registry for each integration test](`18c4fe5907`) - Previously we were reusing the same registry for all the tests and just [ignored the attempts to register same metrics](`01f0ded7fc/integration/cluster/cluster.go (L113)`). This causes the registry to have metrics registered only from the first test so any updates from subsequent tests won't reflect in the metrics. metrics was the only reliable way for me to verify that `query-schedulers` were connected to `queriers` and `query-frontend` when running in ring mode in the integration test that I added to test my changes. This should also help with other tests where earlier it was hard to reliably check the metrics. * [load config from cli as well before applying dynamic config](`f9e2448fc7`) - Previously we were applying dynamic config considering just the config from config file. This results in unexpected config changes, for example, [this config change](`4148dd2c51/integration/loki_micro_services_test.go (L66)`) was getting ignored and [dynamic config tuning was unexpectedly turning on ring mode](`52cd0a39b8/pkg/loki/config_wrapper.go (L94)`) in the config. It is better to do any config tuning based on both file and cli args configs. Which issue(s) this PR fixes: Fixes #9195	2 years ago
Kaviraj Kanagaraj	3a0be48975	config: Improve error message loading config with ENV variables. (#7759 ) What this PR does / why we need it: Before: ``` Unable to parse config: ./clients/cmd/promtail/promtail-local-config.yaml: parse "http://${LOKI_URL}localhost:3100/loki/api/v1/push": invalid character "{" in host name ``` After: ``` Unable to parse config: ./clients/cmd/promtail/promtail-local-config.yaml: parse "http://${LOKI_URL}localhost:3100/loki/api/v1/push": invalid character "{" in host name. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file ``` Which issue(s) this PR fixes: Fixes #NA Special notes for your reviewer: Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [x] `CHANGELOG.md` updated	3 years ago
Eng Zer Jun	e2842c69c5	Replace deprecated `io/ioutil` with `io` and `os` packages (#7212 ) The `io/ioutil` package has been deprecated in Go 1.16 (See https://pkg.go.dev/io/ioutil). This PR replaces the existing `io/ioutil` functions with their new definitions in `io` and `os` packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	3 years ago
Mohamed-Amine Bouqsimi	3d4788fb51	Add ability for Ruler to remote write to multiple clients (#6823 )	3 years ago
Adam	eaf7f34f49	bugfix:add cases.NoLower option (#7052 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: see #7048 if we don't apply cases.NoLower, it will upper first alphabet lower the others ```go package main import ( "fmt" "golang.org/x/text/cases" "golang.org/x/text/language" "strings" ) func main() { source := "testAPI" fmt.Println(cases.Title(language.Und).String(source)) // out: Testapi fmt.Println(cases.Title(language.Und, cases.NoLower).String(source)) // out: TestAPI fmt.Println(strings.Title(source)) // out: TestAPI } ``` Which issue(s) this PR fixes: Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: wujw39640 <wujw39640@hundsun.com> Signed-off-by: wujunwei <wjw3323@live.com> Co-authored-by: wujw39640 <wujw39640@hundsun.com>	3 years ago
Adam	d69d3b664c	Deprecation Fix: Replace strings.Title with cases.Title().String() (#7048 ) Signed-off-by: wujunwei <wjw3323@live.com> <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: [Golang 1.18 deprecates the `strings.title function`](https://github.com/golang/go/issues/48367) use `cases.Title(language.Und, cases.NoLower).String()` instead. Which issue(s) this PR fixes: Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: wujw39640 <wujw39640@hundsun.com>	3 years ago
Christian Haudum	9a82f5c04a	Look up config file in default locations (#6160 ) * Look for config.yaml file in ./ and ./config directories This change makes Loki look for a config file in the current directory or in the config/ sub-directory. This is a pre-requisite for making an easy-to-use tarball distribution that also ships a config.yaml Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Rename YAMLFlag to ConfigFileLoader The `YAMLFlag` function is solely used for loading config files. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Look for config.yaml file in ./ and ./config directories * fixup! fixup! Look for config.yaml file in ./ and ./config directories Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! fixup! fixup! Look for config.yaml file in ./ and ./config directories Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Trevor Whitney	81d0f1d533	Add more tests around config parsing changes from common config PR (#4433 ) * add test to demonstrate bug Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * a totally empty yaml file causes weird behavior Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * add default unmarshal test for test config wrapper Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * add additional tests around default unmarshal to confirm bug is not there Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * clean up tests Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * check errors in test Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com>	4 years ago
Ed Welch	11e215d79f	Loki: Common Config (#4347 ) * refactor the pkg/util/cfg package to simplify and make it accessable outside of this package. create a pkg/cfg which handles building the Loki config, this is created separately from pkg/util (which is Apache 2 licensed) to maintain proper separations of AGPL and Apache 2 code Update the main.go for several of the apps with the refactoring * refactor how dynamic/common config is loaded Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Add common config logic for path prefix Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Missed a file, update common config with prefix property Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * fix test, remove loki from args Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Clean up test names Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * fix typo in filename, make linter happy Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Update pkg/loki/config_wrapper.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/loki/config_wrapper.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/loki/config_wrapper_test.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * remove temp files in test Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Arve Knudsen	cfb4fc1f55	Use flagext from dskit (#4225 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Ed Welch	6e67d1ea5d	Loki/Promtail: Client Refactor (#3623 ) * move docker driver * move fluent-bit output plugin * move fluentd plugin * move logstash plugin * move promtail * match stage uses simple logql label matcher and filter expressions only * remove remaining logql imports * remove helpers package and move contents to util package * move validation and runtime out of util package * fix imports * move build and cfg packages into util * move ParseRequest function to util package * create a new package for logql models and move marshal/unmarshal code to util folder * move ErrorLabel to logqlmodel * move stats package to logqlmodel * decouple loghttp from logql by moving validation to logql package * move gcplog to tools * move PackedEntryKey constant to logqlmodel * fix stats proto import and rebuild protos * fix windows promtail tests * update drone to new dockerfile locations	4 years ago

12 Commits (bc0069756e2e6942cfd34924d2e3ab0b9474b967)