Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Danny Kopping 6bf5b5d8c1 Use a line-buffered logger to deamplify write syscalls (#6954) ... We initialise a global logger in `pkg/util/log/log.go` and use it extensively throughout the Loki codebase. Every time we write a log message, a `write` syscall is invoked. Syscalls are problematic because they transition the process from userspace to kernelspace, which means: - a context-switch is incurred, which is inherently expensive ([1-2 microseconds](https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/)) - the goroutine executing the code is **blocked** - the underlying OS thread (_M_ in the go scheduler model) is **also blocked** - the goroutine has to be rescheduled once the syscall exits - the go scheduler may need to spawn additional OS threads if all are blocked in syscalls - which can also be expensive This change introduces the use of a line-buffered logger. It has a buffer of [256](https://gist.github.com/dannykopping/0704db32c0b08751d1d2494efaa734c2) entries, and once that buffer is filled it will flush to disk. However, a situation will arise in which that buffer remains somewhat empty for a period of time, so there is a periodic flush mechanism, configured to flush every 100ms. There is also a preallocated bytes slice of 10MB which is reused, to avoid excessive slice resizing & garbage collection. This does mean that we could lose up to 256 log messages in case of an ungraceful termination of the process, but this would need to be precisely timed within the 100ms flushes - in other words, the likelihood is low, and generally we shouldn't `kill -9` any Loki process.		4 years ago
..
api	[promtail] Refactor promtail client metrics so that we can't have duplicate metrics collected for the lag metric. (#5521)	4 years ago
client	Promtail: Add support for max stream limit (#7101)	4 years ago
config	Added mapstructure tag to promtail configs structs (#7320)	4 years ago
discovery/consulagent	Replacing go-kit/kit/log with go-kit/log (#4484)	5 years ago
limit	Added mapstructure tag to promtail configs structs (#7320)	4 years ago
positions	Added mapstructure tag to promtail configs structs (#7320)	4 years ago
scrapeconfig	Added mapstructure tag to promtail configs structs (#7320)	4 years ago
server	Replace deprecated `io/ioutil` with `io` and `os` packages (#7212)	4 years ago
targets	Use a line-buffered logger to deamplify write syscalls (#6954)	4 years ago
promtail.go	Promtail: Add support for max stream limit (#7101)	4 years ago
promtail_test.go	Replace deprecated `io/ioutil` with `io` and `os` packages (#7212)	4 years ago