Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Danny Kopping 6bf5b5d8c1 Use a line-buffered logger to deamplify write syscalls (#6954) ... We initialise a global logger in `pkg/util/log/log.go` and use it extensively throughout the Loki codebase. Every time we write a log message, a `write` syscall is invoked. Syscalls are problematic because they transition the process from userspace to kernelspace, which means: - a context-switch is incurred, which is inherently expensive ([1-2 microseconds](https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/)) - the goroutine executing the code is **blocked** - the underlying OS thread (_M_ in the go scheduler model) is **also blocked** - the goroutine has to be rescheduled once the syscall exits - the go scheduler may need to spawn additional OS threads if all are blocked in syscalls - which can also be expensive This change introduces the use of a line-buffered logger. It has a buffer of [256](https://gist.github.com/dannykopping/0704db32c0b08751d1d2494efaa734c2) entries, and once that buffer is filled it will flush to disk. However, a situation will arise in which that buffer remains somewhat empty for a period of time, so there is a periodic flush mechanism, configured to flush every 100ms. There is also a preallocated bytes slice of 10MB which is reused, to avoid excessive slice resizing & garbage collection. This does mean that we could lose up to 256 log messages in case of an ungraceful termination of the process, but this would need to be precisely timed within the 100ms flushes - in other words, the likelihood is low, and generally we shouldn't `kill -9` any Loki process.		3 years ago
..
bucket	feat: Add support for Azure user assigned identity (#5891)	4 years ago
chunk	Add support for username to redis cache configuration (#7270)	3 years ago
config	tsdb supports data deletion, this PR takes care of todo to enable delete request client for tsdb (#7223)	3 years ago
errors	Refactoring of the Loki storage (#5833)	4 years ago
hack	Refactoring of the Loki storage (#5833)	4 years ago
stores	Use a line-buffered logger to deamplify write syscalls (#6954)	3 years ago
async_store.go	[TSDB] Ingester Index Sampling (#6852)	3 years ago
async_store_test.go	Revert "Hide duplicates caused by stream sharding (#7005)" (#7031)	3 years ago
batch.go	Revert "Hide duplicates caused by stream sharding (#7005)" (#7031)	3 years ago
batch_test.go	Revert "Hide duplicates caused by stream sharding (#7005)" (#7031)	3 years ago
factory.go	Make TSDB store singleton resettable (#6950)	3 years ago
factory_test.go	Refactoring of the Loki storage (#5833)	4 years ago
lazy_chunk.go	Refactoring of the Loki storage (#5833)	4 years ago
lazy_chunk_test.go	Revert "Hide duplicates caused by stream sharding (#7005)" (#7031)	3 years ago
store.go	add index requests panel to reads/writes dashboard (#7163)	3 years ago
store_test.go	Revert "Hide duplicates caused by stream sharding (#7005)" (#7031)	3 years ago
util_test.go	Revert "Hide duplicates caused by stream sharding (#7005)" (#7031)	3 years ago