apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
JordanRushing	d8a0c6f87f	Revert "[loki-canary] Support passing loki address as environment variable (#8024 )" (#8183 ) What this PR does / why we need it: This PR reverts commit `f9ed9cb7d9` which introduced breaking config changes for the Loki canary. The relevant change introduced support for configuring a new `LOKI_ADDRESS` canary environment variable by adopting a new library for flags. This is the easiest path forward to avoid forcing breaking config changes on canary operators; such a change is likely better suited to join other configuration changes in a future major Loki release. Which issue(s) this PR fixes: N/A Special notes for your reviewer: Related PR: - https://github.com/grafana/loki/pull/8024 Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Travis Patterson	4a3e6f9d95	Bump go modules GitHub.com prometheus alertmanager 0.25.0 (#8100 ) This PR required some code changes: https://github.com/grafana/loki/pull/8095 We have all the information, it's just plumbed in a little different now. Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
Jijo Thomas John	f9ed9cb7d9	[loki-canary] Support passing loki address as environment variable (#8024 ) What this PR does / why we need it: - Replace flag with kingpin which has more capabilities to override defaults - Override default for `addr`, `user` and `pass` from environment the variables `LOKI_ADDRESS`, `LOKI_USERNAME` and `LOKI_PASSWORD` respectively. Which issue(s) this PR fixes: Fixes #8021 Special notes for your reviewer: Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Kaviraj Kanagaraj	30ddd77150	loki-canary: respect `useTLS` flag when `push` mode is enabled. (#7701 ) What this PR does / why we need it: Currently `useTLS` flag is ignored on the push writer. This PR consider that flag and adjust the schema (http -> https) when that flag is enabled. Which issue(s) this PR fixes: Fixes #7692 Special notes for your reviewer: Checklist	3 years ago
Joel Verezhak	46d3dec010	[loki-canary] Allow insecure TLS connections (#7398 ) What this PR does / why we need it: This change allows client certificates signed by a self-signed certificate authority to be used by the Loki canary. Which issue(s) this PR fixes: Fixes #4366 Special notes for your reviewer: This has been tested on linux amd64 with self-signed certificates. Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Kaviraj Kanagaraj	2c9fa05c29	feat(loki-canary): Add support to push logs directly to Loki. (#7063 ) Add additional `push` mode to loki-canary, which pushes the logs directly to given Loki URL as it generates logs. The real function of Loki Canary is to act like a tenant and help us know the whether Loki installation is working as perceived by a real tenant. Main rationale for this additional push mode is to make canary more standalone without needing for `promtail` (or `grafana-agent`) to scrape it's logs and send to loki, with this change, Loki canary happily tests Loki behavior without needing any other dependencies. NOTES: 1. If you run Loki behind any proxy that has different authorization policies to READ or WRITE to Loki, then important change that canary operator need to be aware of it, now the user credentials they pass via `-user` and `-pass` to access loki endpoints need to have both `read' and `write` permissions (previously canary just query the logs where as promtail is the one pushes the logs, so just READ permissions was sufficient). 2. There will be follow up PR(s) to cleanup, particularly `reader` and `comparitor` component in terms of logging with proper logger. Rationale is to keep the changes small per PR to make it easy to review. 3. This changes were tested it in one of the internal Loki dev cell. 4. This PR is a no-op if this new `push` flag is disabled (it's disabled by default) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Chris Hodges	c582c5a727	loki-canary: Fix resource leak when using client-side certs (#6579 ) * loki-canary: Fix resource leak when using client-side certs * swap panic for os.Exit when failing to create httpClient	4 years ago
Chris Hodges	6cbcd6aa76	loki-canary: Add support for client-side TLS certs for Loki connection (#6310 ) * Add support for client-side TLS certs to loki-canary * fix lint errors, make the case of using TLS but not client certs more clear, check that -tls is set if client certs are used * move changelog to canary section instead of main * wrap httpClient errors and simplify dialer creation for websocket	4 years ago
Daniel Kovacs	e9d2a30733	Fixed reader to support both authentication and tenant-id at the same time. (#5719 )	4 years ago
Callum Styan	3ee8ed3b1d	Allow for loki-canary to generate a percentage of out of order log lines (#4126 ) * Allow for loki-canary to generate a percentage of log lines with out of order timestamps, percentage and out of order time range is all configurable. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Switch OoO min/max flags to time duration and properly calculate OoO percentage check. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Change some variable names + add comments to make it more obvious what various parts of the canary code are actually doing. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update pkg/canary/writer/writer.go * Add notes about usage of the out-of-order-percentage flag and the requirement to use pipeline stages. Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Sandeep Sukhani	6b775ac31c	allow setting tenant id for querying logs from loki (#4185 )	4 years ago
Ed Welch	6e67d1ea5d	Loki/Promtail: Client Refactor (#3623 ) * move docker driver * move fluent-bit output plugin * move fluentd plugin * move logstash plugin * move promtail * match stage uses simple logql label matcher and filter expressions only * remove remaining logql imports * remove helpers package and move contents to util package * move validation and runtime out of util package * fix imports * move build and cfg packages into util * move ParseRequest function to util package * create a new package for logql models and move marshal/unmarshal code to util folder * move ErrorLabel to logqlmodel * move stats package to logqlmodel * decouple loghttp from logql by moving validation to logql package * move gcplog to tools * move PackedEntryKey constant to logqlmodel * fix stats proto import and rebuild protos * fix windows promtail tests * update drone to new dockerfile locations	5 years ago
Ed Welch	204e61afe3	Loki-Canary: Backoff retries on query failures, add histograms for query performance. (#2413 ) * Canary needs to wait before spot-checking entries some reasonable amount of time else the nature of the async thread doing the testing can ask for them from Loki only a few ms after they are written and they would have no chance of being present. Adding two histograms for tracking the performance of the spot check and metric test queries * Adding backoff to queries if they fail specifically waiting 5mins when a 429 is received from Loki * fix order of parameters	6 years ago
Ed Welch	76ceec45d9	If an entry is missing on the websocket, start checking after `wait` but continue checking until `max-wait` (#2369 ) Refactored out a useful pruneList function	6 years ago
Ed Welch	2b145da7d7	Canary tweaks (#2350 ) * tweak the canary a bit, add a metric for websocket pings, fix some logging, improve the mutex locking a bit, add a read timeout on the websocket, add a lastmessage time checker to restart websocket * fix constructor * fix counter name change metric count reporting to two separate metrics, expected vs actual * add note to upgrade guide for metric name changes * fixed tests.... * Update docs/operations/upgrade.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	6 years ago
Ed Welch	81b9decfb5	Loki-Canary: Add query spot checking and metric count checking (#2344 ) * Add spot check capability to canary to query certain values over time to make sure they are still in Loki. Add a context timeout to Loki queries * Updating canary to use /loki/api/v1 endpoints adding a metric test query * fixing metric names to have _total for counters * fix parsing of timestamp from log line and don't use entry timestamp * Tweak the metric test query to adjust the range to the time the canary has been running to get accurate results until the requested range has been reached. Add a total spot check entry counter to get a ratio of success * Add some debugging logs to troubleshoot missing entries * add more tests, fix some log lines * update docs, make the spot-check timer separate from prune entries * just how many paramaters can I add to the constructor.... * run prune entries on a separate thread, only run one at a time.	6 years ago
Lukas Grimm	17390fd45d	Canary: make stream configurable (#2259 ) * tried to fix canary in non kubernetes env * update doc	6 years ago
Joe Elliott	383eb6e70b	Addition of a `/suspend` endpoint to Loki Canary (#1891 ) * First pass loki canary pause Signed-off-by: Joe Elliott <number101010@gmail.com> * Write suspending msg to stderr Signed-off-by: Joe Elliott <number101010@gmail.com> * Added resume endpoint Signed-off-by: Joe Elliott <number101010@gmail.com> * Added documentation on new endpoints Signed-off-by: Joe Elliott <number101010@gmail.com> * Pass both signals to the same channel Signed-off-by: Joe Elliott <number101010@gmail.com> * lint Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed return on startCanary Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Peter Štibraný	49d0d0f0da	Makefile changes to allow easy builds with or without vendoring. Also fixes version bug for both cases. (#1095 ) * Extracted -mod=vendor into a variable * Use ?= to set variable only if not set yet. This enables "MOD_VENDOR= make all" use from command line * Enable/disable vendoring with VENDOR=true or false. Enabled by default * Make go generate use -mod=vendor depending on VENDOR flag * Use GOMOD to specify module version to use. GOMOD can be set to empty string, "vendor", "readonly" and in Go 1.14 to "mod". If not defined, defaults to "vendor". Non empty values are passed to -mod parameter to Go tools, and --modules-download-mode parameter for golangci-cli linting tool. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix versioning info when not using vendoring. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Updated build image to use Golintci-cli v1.21.0 and Go 1.13.4. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Updated build image to version 0.8.0. Changs: Golintci-cli v1.21.0, Go 1.13.4. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Renamed MOD_VENDOR to MOD_FLAG to better describe its function. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix version info in all command line tools. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use loki-build-image:0.8.0 Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
sh0rez	794d7da845	feat: -version flag (#1036 ) While we were already setting the version on the built binaries, this now also exposes them as a user-facing flag (`-version`). The already used Prometheus package `github.com/prometheus/common/version` is used for generated the printed string.	6 years ago
Ed	173e8ea2a4	fix(canary): Fix Flaky Tests (#889 ) * make the missing entry lookup synchronous during tests to hopefully avoid races around the entry order and test verifications * playing around with timings some more to get all tests to pass even on a really slow machine	6 years ago
Ed	fc12a2355d	style(loki-canary): spelling (#832 )	7 years ago
Edward Welch	e6f431ad46	finishing up loki-canary move into loki repo	7 years ago
Edward Welch	cd5368a44d	preparing for move into loki repo	7 years ago
Edward Welch	eedd3d9fc2	prune interval is configurable canary will suspend all operations on SIGINT but not exit, allowing you to shutdown the canary without it being restarted by docker/kubernetes SIGTERM will shutdown everything and end the process	7 years ago
Edward Welch	bcccbbfaaf	changing the default prune interval to 60 seconds, at the previous 1 second it would cause some way too aggressive querying when scaled out over all our clusters and loki had a hiccup	7 years ago
Edward Welch	02aed71a28	refactoring things so that the comparator can use the reader to make a direct query to loki to look for logs not received over the websocket before reporting them as missing	7 years ago
Edward Welch	5aac00f4d9	changing response_latency buckets some more, making exponential and configurable	7 years ago
Edward Welch	9dde2c4e5f	fixing how auth is applied	7 years ago
Edward Welch	34bb9db758	using an environment variable passed into an arg instead of reading the pod name from a file and the downardApi added a makefile	7 years ago
Edward Welch	45a7556da2	improving metrics improving tests config via flags added ksonnet config	7 years ago
Edward Welch	8340232b04	initial checkin mostly working, need to extract out the config needs a proper ksonnet or helm deployment or both needs a makefile	7 years ago

32 Commits (db3a9c961e65f186f910cc07e7f46b32779ca9a0)