Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Danny Kopping 6bf5b5d8c1 Use a line-buffered logger to deamplify write syscalls (#6954) ... We initialise a global logger in `pkg/util/log/log.go` and use it extensively throughout the Loki codebase. Every time we write a log message, a `write` syscall is invoked. Syscalls are problematic because they transition the process from userspace to kernelspace, which means: - a context-switch is incurred, which is inherently expensive ([1-2 microseconds](https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/)) - the goroutine executing the code is **blocked** - the underlying OS thread (_M_ in the go scheduler model) is **also blocked** - the goroutine has to be rescheduled once the syscall exits - the go scheduler may need to spawn additional OS threads if all are blocked in syscalls - which can also be expensive This change introduces the use of a line-buffered logger. It has a buffer of [256](https://gist.github.com/dannykopping/0704db32c0b08751d1d2494efaa734c2) entries, and once that buffer is filled it will flush to disk. However, a situation will arise in which that buffer remains somewhat empty for a period of time, so there is a periodic flush mechanism, configured to flush every 100ms. There is also a preallocated bytes slice of 10MB which is reused, to avoid excessive slice resizing & garbage collection. This does mean that we could lose up to 256 log messages in case of an ungraceful termination of the process, but this would need to be precisely timed within the 100ms flushes - in other words, the likelihood is low, and generally we shouldn't `kill -9` any Loki process.		3 years ago
..
cloud.google.com/go	promtail: GCPLog fixing `failed to receive pubsub messages` error (#6965)	3 years ago
github.com	Use a line-buffered logger to deamplify write syscalls (#6954)	3 years ago
go.etcd.io	update dskit and fix (#6868)	3 years ago
go.mongodb.org/mongo-driver	Bump prometheus dependency (#6403)	3 years ago
go.opencensus.io	Update to go 1.17; go mod vendor required (#4487)	4 years ago
go.opentelemetry.io	Bump prometheus dependency (#6403)	3 years ago
go.uber.org	Configurable TLS MinVersion and CipherSuites (#7227)	3 years ago
go4.org	Support building against Go 1.19 (#6833)	3 years ago
golang.org/x	Configurable TLS MinVersion and CipherSuites (#7227)	3 years ago
google.golang.org	promtail: GCPLog fixing `failed to receive pubsub messages` error (#6965)	3 years ago
gopkg.in	Update gopkg.in/yaml.v3 (#6284)	3 years ago
inet.af/netaddr	Upgrade inet.af/netaddr (#6046)	3 years ago
k8s.io	Bump prometheus dependency (#6403)	3 years ago
rsc.io/binaryregexp	Update to go 1.17; go mod vendor required (#4487)	4 years ago
sigs.k8s.io	Bump prometheus dependency (#6403)	3 years ago
modules.txt	Use a line-buffered logger to deamplify write syscalls (#6954)	3 years ago