Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Danny Kopping 6bf5b5d8c1 Use a line-buffered logger to deamplify write syscalls (#6954) ... We initialise a global logger in `pkg/util/log/log.go` and use it extensively throughout the Loki codebase. Every time we write a log message, a `write` syscall is invoked. Syscalls are problematic because they transition the process from userspace to kernelspace, which means: - a context-switch is incurred, which is inherently expensive ([1-2 microseconds](https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/)) - the goroutine executing the code is **blocked** - the underlying OS thread (_M_ in the go scheduler model) is **also blocked** - the goroutine has to be rescheduled once the syscall exits - the go scheduler may need to spawn additional OS threads if all are blocked in syscalls - which can also be expensive This change introduces the use of a line-buffered logger. It has a buffer of [256](https://gist.github.com/dannykopping/0704db32c0b08751d1d2494efaa734c2) entries, and once that buffer is filled it will flush to disk. However, a situation will arise in which that buffer remains somewhat empty for a period of time, so there is a periodic flush mechanism, configured to flush every 100ms. There is also a preallocated bytes slice of 10MB which is reused, to avoid excessive slice resizing & garbage collection. This does mean that we could lose up to 256 log messages in case of an ungraceful termination of the process, but this would need to be precisely timed within the 100ms flushes - in other words, the likelihood is low, and generally we shouldn't `kill -9` any Loki process.		3 years ago
..
index	avoid using bloomfilters for chunks in stats calls by avoiding duplicates (#7209)	3 years ago
indexshipper	Use a line-buffered logger to deamplify write syscalls (#6954)	3 years ago
series	refactor index storage code to index reader/writer interfaces for better monitoring (#7154)	3 years ago
shipper	Use a line-buffered logger to deamplify write syscalls (#6954)	3 years ago
tsdb	reintroduce mmap for tsdb indices (#7321)	3 years ago
composite_store.go	refactor index storage code to index reader/writer interfaces for better monitoring (#7154)	3 years ago
composite_store_entry.go	refactor index storage code to index reader/writer interfaces for better monitoring (#7154)	3 years ago
composite_store_test.go	refactor index storage code to index reader/writer interfaces for better monitoring (#7154)	3 years ago
series_store_write.go	refactor index storage code to index reader/writer interfaces for better monitoring (#7154)	3 years ago