apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Danny Kopping	7c78d7ea44	Removing "replace" directive for line-buffered logger (#8162 )	3 years ago
Danny Kopping	6bf5b5d8c1	Use a line-buffered logger to deamplify write syscalls (#6954 ) We initialise a global logger in `pkg/util/log/log.go` and use it extensively throughout the Loki codebase. Every time we write a log message, a `write` syscall is invoked. Syscalls are problematic because they transition the process from userspace to kernelspace, which means: - a context-switch is incurred, which is inherently expensive ([1-2 microseconds](https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/)) - the goroutine executing the code is blocked - the underlying OS thread (_M_ in the go scheduler model) is also blocked - the goroutine has to be rescheduled once the syscall exits - the go scheduler may need to spawn additional OS threads if all are blocked in syscalls - which can also be expensive This change introduces the use of a line-buffered logger. It has a buffer of [256](https://gist.github.com/dannykopping/0704db32c0b08751d1d2494efaa734c2) entries, and once that buffer is filled it will flush to disk. However, a situation will arise in which that buffer remains somewhat empty for a period of time, so there is a periodic flush mechanism, configured to flush every 100ms. There is also a preallocated bytes slice of 10MB which is reused, to avoid excessive slice resizing & garbage collection. This does mean that we could lose up to 256 log messages in case of an ungraceful termination of the process, but this would need to be precisely timed within the 100ms flushes - in other words, the likelihood is low, and generally we shouldn't `kill -9` any Loki process.	3 years ago
Jesus Vazquez	db9dbe8e1d	Bump prometheus dependency (#6403 ) * Bump prometheus dependency We had to add a replace of prometheus in go mod because thanos is importing an older version and then replacing it. * Undo textparser.New variable mismatch * modtimevfs path update * Fix labels withlabels and withoutlabels usages WithoutLabels and WithLabels methods have dissappeared. Now labels.NewBuilder has to be used. * Fix RuleManager interface for Update() Update now accepts a new param called ruleGroupPostProcessFunc which is an optional function to be executed after rule group processing. In this case I've set it to nil in the usages i've found. * NewQueryRange now accepts QueryOpts The NewQueryRange function now accepts QueryOptions so I had to update all the calls. I've set the opts to nil since we're not using them. * gofmt	4 years ago
Susana Ferreira	2d24e2ea64	Migration to dskit/ring (#4641 )	4 years ago
Callum Styan	6da518c728	Update to go 1.17; go mod vendor required (#4487 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Danny Kopping	cff8324f24	Replacing go-kit/kit/log with go-kit/log (#4484 ) * Replacing go-kit/kit/log with go-kit/log Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the f&^$ing linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding depguard Cleaning go mod Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating weaveworks after https://github.com/weaveworks/common/pull/226 was merged Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the mod checker Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Arve Knudsen	10c92a901e	Chore: Upgrade to latest Cortex (#4188 ) * Chore: Upgrade to latest Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix formatting Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Arve Knudsen	3e385464c6	Chore: Upgrade Prometheus and Cortex (#4017 ) * Chore: Upgrade Prometheus Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Upgrade Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Upgrade Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build errors Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build errors Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build errors Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	5 years ago
Aditya C S	199e334126	Update loki to cortex master (#2030 )	6 years ago
Joe Elliott	fc6c5c063f	dep => go mod (#1062 ) * go mod files added. dep removed Signed-off-by: Joe Elliott <number101010@gmail.com> * Magically got prometheus version to stick Signed-off-by: Joe Elliott <number101010@gmail.com> * Cortex updated and prometheus updated Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated prometheus dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added additional deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Added two replaces from Gopkg.toml. Tests passing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated build image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golangci-lint to use vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod makefile step and referenced in drone and circle ci configs Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated loki-build-image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed linting error. Force go111module on for linting Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod vendor Signed-off-by: Joe Elliott <number101010@gmail.com> * Forced the inclusion of modtimevfs Signed-off-by: Joe Elliott <number101010@gmail.com> * Pin client-go due to issue with v12 go.mod Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod to drone Signed-off-by: Joe Elliott <number101010@gmail.com> * Re-readded correct golang client Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Pinned golang/x/net to avoid proxy errors Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed check-mod from all. Not necessary for tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go.sum to match new pin Signed-off-by: Joe Elliott <number101010@gmail.com> * Upgraded proto to match build image Signed-off-by: Joe Elliott <number101010@gmail.com> * Force check-mod to wait til after test and lint are successful Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to go builds to force usage of vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Turn on gomodules on all builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Added mod vendor to go builds to force usage of vendored dependencies" This reverts commit 65865a24c9a23133e0fa52942f2828ead7c22147. * Moved builds out of the gopath to enforce vendor usage Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Turn on gomodules on all builds" This reverts commit b5847f0158e928e935e0b3c1b1d4eaba840ca3dc. * Explicitly choose build image for docker driver Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golang.org/x/sys to match prometheus's version to fix promtail windows compilation Signed-off-by: Joe Elliott <number101010@gmail.com> * Added fluentbit dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added dependency management clause Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go version in contributing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added phony makefile targets Signed-off-by: Joe Elliott <number101010@gmail.com> * Circle CI is increasingly failing linting Signed-off-by: Joe Elliott <number101010@gmail.com> * Force vendored deps on builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Added logcli to gitignore Signed-off-by: Joe Elliott <number101010@gmail.com> * Reduced concurrency to help CircleCI Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved drone builds out of GOPATH to force use of go module Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to prevent go clean from redownloading all packages Signed-off-by: Joe Elliott <number101010@gmail.com> * Added to test as well Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod-vendor to final go commands Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved fluent-bit out of GO_PATH to force go modules Signed-off-by: Joe Elliott <number101010@gmail.com> * Pass mod vendor to the last holdout: go generate Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated ugorji/go to 1.17 to avoid version regression Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy and go mod vendor for ugorji/go Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Adam Shannon	eb390523e3	all: update dependencies (#211 ) * all: update dependencies * Update some more dependancies and pin the right versions for Prometheus deps. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	a1d66f42c0	Add vendor stuff.	8 years ago

12 Commits (b46e392adbbb59f40fecd16c9ff925cb40456e0c)