Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Danny Kopping 6bf5b5d8c1 Use a line-buffered logger to deamplify write syscalls (#6954) ... We initialise a global logger in `pkg/util/log/log.go` and use it extensively throughout the Loki codebase. Every time we write a log message, a `write` syscall is invoked. Syscalls are problematic because they transition the process from userspace to kernelspace, which means: - a context-switch is incurred, which is inherently expensive ([1-2 microseconds](https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/)) - the goroutine executing the code is **blocked** - the underlying OS thread (_M_ in the go scheduler model) is **also blocked** - the goroutine has to be rescheduled once the syscall exits - the go scheduler may need to spawn additional OS threads if all are blocked in syscalls - which can also be expensive This change introduces the use of a line-buffered logger. It has a buffer of [256](https://gist.github.com/dannykopping/0704db32c0b08751d1d2494efaa734c2) entries, and once that buffer is filled it will flush to disk. However, a situation will arise in which that buffer remains somewhat empty for a period of time, so there is a periodic flush mechanism, configured to flush every 100ms. There is also a preallocated bytes slice of 10MB which is reused, to avoid excessive slice resizing & garbage collection. This does mean that we could lose up to 256 log messages in case of an ungraceful termination of the process, but this would need to be precisely timed within the 100ms flushes - in other words, the likelihood is low, and generally we shouldn't `kill -9` any Loki process.		3 years ago
..
Dockerfile	Update alpine and golang base images (#6897)	3 years ago
Dockerfile.arm32	Update alpine and golang base images (#6897)	3 years ago
Dockerfile.cross	Update alpine and golang base images (#6897)	3 years ago
Dockerfile.debug	update to alpine:3.6.2 (#6979)	3 years ago
main.go	Use a line-buffered logger to deamplify write syscalls (#6954)	3 years ago
promtail-cloudflare.yaml	Promtail pull cloudflare logs (#4813)	4 years ago
promtail-docker-config.yaml	Loki/Promtail: Client Refactor (#3623)	5 years ago
promtail-gelf.yaml	Add GELF support for Promtail. (#4744)	4 years ago
promtail-journal.yaml	Promtail: Add metrics for journal target (#6105)	4 years ago
promtail-kafka-sasl-plain.yaml	Add SASL&mTLS authentication support for Kafka in Promtail (#4663)	4 years ago
promtail-kafka-sasl-scram.yaml	Add SASL&mTLS authentication support for Kafka in Promtail (#4663)	4 years ago
promtail-kafka-sasl-ssl.yaml	Add SASL&mTLS authentication support for Kafka in Promtail (#4663)	4 years ago
promtail-kafka-ssl.yaml	Add SASL&mTLS authentication support for Kafka in Promtail (#4663)	4 years ago
promtail-kafka.yaml	Promtail Kafka target (#4568)	4 years ago
promtail-local-config.yaml	Loki/Promtail: Client Refactor (#3623)	5 years ago
promtail-local-limit-config.yaml	Promtail: Rename config name limit_config to limits_config (#5707)	4 years ago
promtail-local-pubsub-config.yaml	Loki/Promtail: Client Refactor (#3623)	5 years ago
promtail-windows.yaml	Loki/Promtail: Client Refactor (#3623)	5 years ago