apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Karsten Jeschkies	125e168a91	Replace flaky server for log level handler test. (#9822 ) What this PR does / why we need it: I've noticed that the `TestLevelHandler` were flaky because the code wasn't waiting to the server to be running. Since the test is for a handler it's much simpler to call it directly with a response recorder. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
indX	a55404766c	Logging: Add HTTP API to change the log level at runtime (#9357 ) What this PR does / why we need it: To be able to change the log level at runtime, particularly from info to debug to enable debugging information during an incident without restarting Loki. Which issue(s) this PR fixes: Fixes #6805 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`: No such changes --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Ed Welch	67c0b3ed2c	Loki: log all trace ID's but include `sampled=true` when the trace is logged (#9039 ) What this PR does / why we need it: Currently when we log "with context" we will only add the traceID if the particular trace is being sampled. This is nice because it makes it obvious that if you go search for a trace you'll find it. However, traceID is also useful for selecting all the log lines from a single query within Loki itself and if the trace wasn't sampled we remove this useful piece of context for querying the log lines. In this PR, instead of dropping the traceID from the logs, we instead prefix it with `NS-` Where NS is short for Not Sampled Example: ``` level=debug ts=2023-04-05T15:06:24.377067817Z caller=fetcher.go:292 org_id=145265 traceID=NS-13ed6c30228e221d chunks=1 decodeRequests=1 missing=0 ``` Questions: * should we bikeshed a little on this prefix? (I wanted something concise but if it's concise it's probably confusing to folks who've never seen it before) * Instead of a prefix, we could change the logfmt key? `ID=13ed6c30228e221d` (this is less confusing in some ways but maybe more difficult/confusing in others?) Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Trevor Whitney	b46e392adb	gofmt is needed for the build (#8678 ) What this PR does / why we need it: `gofmt` is needed for the build, `make format` fails without it.	2 years ago
Danny Kopping	a183d6308e	Only print logger error when required (#8256 )	2 years ago
Danny Kopping	7c78d7ea44	Removing "replace" directive for line-buffered logger (#8162 )	2 years ago
Karsten Jeschkies	c71620ae94	Flush buffered logger on exit. (#7924 ) What this PR does / why we need it: I ran into this issue several times. No error logs were printed or no generated token for GEL. This is because crashing runs can be much faster than the flush period. Special notes for your reviewer: Checklist - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Danny Kopping	33bf36aa21	Promtail: flush logs on exit (#7400 ) Promtail is using the line-buffered logger (https://github.com/grafana/loki/pull/6954), and log messages were not being printed on exit	3 years ago
Dennis Szczepanski	4e61c60296	Loki: Renames metric 'loki_log_messages_total' to 'loki_internal_log_messages_total' (#7361 ) What this PR does / why we need it: Renames the metric 'loki_log_messages_total' to a more descriptive name. Keeps and deprecates the previously existing metric to not break functionality. Which issue(s) this PR fixes: Fixes #7088 Special notes for your reviewer: Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Danny Kopping	c3e92b876c	Restoring lint checks that were previously disabled (#7340 ) I removed these checks in https://github.com/grafana/loki/pull/7337 to not make https://github.com/grafana/loki/pull/7243 overly complex to review. Refactored code to fix valid lint issues, ignored the irrelevant/dangerous ones. I also removed `unused` and `unparam` as these produce low-quality results that are largely useless.	3 years ago
Danny Kopping	6bf5b5d8c1	Use a line-buffered logger to deamplify write syscalls (#6954 ) We initialise a global logger in `pkg/util/log/log.go` and use it extensively throughout the Loki codebase. Every time we write a log message, a `write` syscall is invoked. Syscalls are problematic because they transition the process from userspace to kernelspace, which means: - a context-switch is incurred, which is inherently expensive ([1-2 microseconds](https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/)) - the goroutine executing the code is blocked - the underlying OS thread (_M_ in the go scheduler model) is also blocked - the goroutine has to be rescheduled once the syscall exits - the go scheduler may need to spawn additional OS threads if all are blocked in syscalls - which can also be expensive This change introduces the use of a line-buffered logger. It has a buffer of [256](https://gist.github.com/dannykopping/0704db32c0b08751d1d2494efaa734c2) entries, and once that buffer is filled it will flush to disk. However, a situation will arise in which that buffer remains somewhat empty for a period of time, so there is a periodic flush mechanism, configured to flush every 100ms. There is also a preallocated bytes slice of 10MB which is reused, to avoid excessive slice resizing & garbage collection. This does mean that we could lose up to 256 log messages in case of an ungraceful termination of the process, but this would need to be precisely timed within the 100ms flushes - in other words, the likelihood is low, and generally we shouldn't `kill -9` any Loki process.	3 years ago
Karsten Jeschkies	46956d0bf6	Use dskit/tenant intead of loki/pkg/tenant. (#5724 )	3 years ago
Danny Kopping	75328872dc	Print stacktrace to stderr for easier readability (#5616 ) Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Arve Knudsen	a2dd731586	Chore: Add pkg/util/log package (#5187 ) * Chore: Add pkg/util/log package Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix documentation of loki_log_messages_total Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 years ago

14 Commits (7cbfdb5d6d5f6bb7fd6cc97d3f63cfb2997bfed6)