apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Đurica Yuri Nikolić	37ec68eebb	Remove unnecessary code after dskit log.Interface has been dereleased (#10434 ) This PR removes unnecessary code used to convert the dereleased `dslog.Interface` (see https://github.com/grafana/dskit/pull/359) into `go-kit.Logger`. Additionally, it modifies the signature and implementation of `pkg/util/log/log.go`'s `InitLogger` as follows: - previous signature ``` InitLogger(server.Config, prometheus.Registerer, bool, bool) ``` has been replaced with ``` InitLogger(server.Config, prometheus.Registerer, bool, bool) log.Logger ``` Namely, the new `InitLogger` initialises the global logger and returns it. - the new implementation initializes the global logger, but it does not override the default logger for the server. This is now done by the callers of `InitLogger`. Previously, it was done inside `InitLogger` for convenience, since the types of the 2 loggers were not the same. Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com>	2 years ago
Shantanu Alsi	b57d260ddc	Upgrade alpine 3.16.5 -> 3.16.7 (#10188 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #10181 Fixes Open SSL vulnerabilities [CVE-2023-1255](https://security.alpinelinux.org/vuln/CVE-2023-1255), [CVE-2023-2650](https://security.alpinelinux.org/vuln/CVE-2023-2650), [CVE-2023-2975](https://security.alpinelinux.org/vuln/CVE-2023-2975), [CVE-2023-3446](https://security.alpinelinux.org/vuln/CVE-2023-3446), [CVE-2023-3817](https://security.alpinelinux.org/vuln/CVE-2023-3817) Alpine Release Notes: [3.16.7](https://www.alpinelinux.org/posts/Alpine-3.15.10-3.16.7-3.17.5-3.18.3-released.html) [3.16.6](https://www.alpinelinux.org/posts/Alpine-3.15.9-3.16.6-3.17.4-3.18.2-released.html) Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Charles Korn	bde65667f7	Upgrade dskit to use packages migrated from weaveworks/common (#10164 ) What this PR does / why we need it: This PR upgrades dskit and replaces use of packages from weaveworks/common with their migrated equivalents in dskit. See https://github.com/grafana/dskit/pull/342 for more details. Note that Loki still uses some packages from weaveworks/common that I haven't migrated (`aws` and `test`) - I'll migrate these separately. If this PR needs to be rebuilt, I used `rewrite.sh` ([source](https://gist.github.com/charleskorn/48efe62a09d6d70f3de30327003df5c5#file-rewrite-sh)) to generate most of these changes. Which issue(s) this PR fixes: (none) Special notes for your reviewer: (none) Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	2 years ago
Michel Hollands	d484d31856	Add support for cross arch for fluent-bit-plugin-loki image (#10081 ) Copied from https://github.com/grafana/loki/pull/9991 which has problems running CI. What this PR does / why we need it: Currently the `fluent-bit-plugin-loki` image is only build and published for amd64. With more use of arm-based processors, it would be good to add support for those as well. I've build the image using `docker buildx` on my M1 Mac and pushed it to Dockerhub, should you wish to try it first: https://hub.docker.com/r/ebosveld/fluent-bit-plugin-loki/ On AWS ECS Graviton based images it works well with the above mentioned image. I've also fixed #9932 as it needed the full bullseye support. Which issue(s) this PR fixes: Fixes #6738 Fixes #9932 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com> Co-authored-by: Edwin Bosveld <edwin.bosveld@persgroep.net> Co-authored-by: Travis Patterson <travis.patterson@grafana.com> Co-authored-by: Edwin Bosveld <ebosveld@users.noreply.github.com>	2 years ago
Michel Hollands	bad691b509	Update go build image (#9914 ) Update the go build image --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Michel Hollands	7be9deacf3	Update go version to 1.20.6 (#9913 ) Update the go version. Another PR will follow that uses the new build image --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Trevor Whitney	4a56445686	Upgrade `golangci-lint` and fix linting errors (#9601 ) What this PR does / why we need it: Upgrade `golangci-lint` and fixes all the errors. The upgrade includes some stricter linting.	2 years ago
Danny Kopping	9495b6cc56	Update loki-build-image with go1.20.4 (#9372 ) Continuation of #9370 Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Danny Kopping	f5c0fdf028	Upgrade to latest security release: go1.20.4 (#9370 ) What this PR does / why we need it: https://groups.google.com/g/golang-announce/c/MEb0UyuSMsU Also updated the build image. --------- Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Jeff Cantrill	3d7060bf73	fluent-plugin-grapha-loki: Add config to support tls: ciphers, min_version (#9289 )	2 years ago
Trevor Whitney	a11921bbb8	Upgrade alpine 3.16.4 -> 3.16.5 (#9264 ) What this PR does / why we need it: Upgrade alpine version to address [openssl CVEs](https://www.alpinelinux.org/posts/Alpine-3.17.3-released.html) Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Trevor Whitney	38c9c6c460	Bump go from 1.20.1 -> 1.20.3 (#9106 ) What this PR does / why we need it: Bumps our go dependency from 1.20.1 to 1.20.3 to address [security fixes](https://groups.google.com/g/golang-announce/c/Xdv6JL9ENs8) Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Bryan Boreham	6fd4b5e89b	Update prometheus/prometheus from 2.41 to 2.42 (#8571 ) What this PR does / why we need it: Brings in the latest updates from upstream. These open up some opportunities for optimisations in TSDB indexing. Dependencies updated: * github.com/Azure/go-autorest/autorest/adal v0.9.21 -> v0.9.22 (comment-only change) * github.com/docker/docker v20.10.21 -> v20.10.23 (fixing filters bug) * golang.org/x/exp fae10dda9338 -> d38c7dcee874 (optimisations in `BinarySearch` function) Indirect dependencies also updated: * github.com/digitalocean/godo v1.91.1 -> v1.95.0 (nothing alarming in [release notes](https://github.com/digitalocean/godo/releases)) * github.com/google/pprof aee1124e3a93 -> 76d1ae5aea2b (no changes pulled into vendor) * golang.org/x/tools v0.4.0 -> v0.5.0 (relating to Go compiler utilities) * google.golang.org/genproto 76db0878b65f -> 31e0e69b6fc2 * k8s.io/api v0.26.0 -> v0.26.1 (comment-only change) * k8s.io/apimachinery v0.26.0 -> v0.26.1 (no changes pulled into vendor) * k8s.io/client-go v0.26.0 -> v0.26.1 (small fixes) Special notes for your reviewer: A couple of interfaces changed; these have required matching changes in Loki code. Those changes are split into separate commits. I also note that most calls to `relabel` ignore when the rule says "drop". Maybe this is wrong?	2 years ago
Michel Hollands	4f8d3240f6	Use 0.28.1 build image and update go and alpine versions (#8583 ) What this PR does / why we need it: - Use the 0.28.1 build image - Update Go version to 1.20.1 - Use Alpine 3.16.4 - Fix linter issues due to updated govet in 1.20.1 - Modify images to not create `/etc/nsswitch.conf` file anymore (available by default on Alpine 3.16.4) - Remove impossible test cases for `TestMemChunk_IteratorBounds`(see https://github.com/grafana/loki/pull/8591#discussion_r1115020111 for context) Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: DylanGuedes <djmgguedes@gmail.com>	2 years ago
Christian Haudum	f52ae115fe	Bump version of `fluent-plugin-grafana-loki` to 1.2.20 (#8426 )	2 years ago
Christian Haudum	1cbedec0b6	Update fluent-plugin-grafana-loki to Ruby 3.2 (#8421 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Etran-H	5447541e2e	Fix open "/tmp/*" the system cannot find the path specified on windows (#8170 ) Instead of hard-coding `/tmp` directory, use the OS default directory for temporary files	2 years ago
Bryan Boreham	c3c9ac7740	Cosmetic lint warning fixes (#8363 )	2 years ago
Christian Haudum	1728d09cba	Use Loki build image 0.27.1 in CI (#8279 ) Second part of updating the Loki build image Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	58e29de988	Update golang to 1.19.5 (#8278 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
nicoche	80ea621ff3	promtail: Add `max-line-size-truncate` (#8233 )	2 years ago
Danny Kopping	d0fd4e8ad1	Do not exit when using `stream_lag_labels` setting (#8243 )	2 years ago
Callum Styan	399eb097fd	Remove the stream lag metric. (#6324 ) Removes the stream lag metric since it's not possible to get a correct value and also not cause issues with metrics registration or collection until we have the WAL Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Kaviraj Kanagaraj	2597429c89	promtail: Add `max-line-size` limit to drop on client side (#8153 )	2 years ago
Karsten Jeschkies	4504ed2701	Cross build Promtail for ARM with journald support. (#8131 ) This is the follow up to #8111 and adds journald support for Promatil ARM and ARM64 builds. I've tested it locally with my Raspberry Pi and it works. Fixes #1459	2 years ago
Michel Hollands	1e143ace11	Use new build image everywhere (#8063 ) Use the latest build image everywhere. Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Rutger	17517bfea8	Promtail: Fix broken promtail-debug docker image (#7852 ) What this PR does / why we need it: The 'grafana/loki-build-image' version was not specified for the promtail-debug docker image, resulting in a ~6 month old 'latest' image as a build image. Updating this resulted in one more change: the newer version has a different location for the 'dlv' binary. The promtail-debug image would also hang waiting for a connection when ran, this adds the same flag, `--continue`, as added to the loki debug file in #1372. Special notes for your reviewer: I was interested in using the `-server.profiling_enabled=true` flag, to enable pprof endpoints, and scrape with phlare. This flag is not available in the non-debug version. Therefore I do not really care for attaching a debugger right away. Open to removing that change though. Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Robert Fratto	bd58352439	Update build image references to 0.25.0 (#7848 ) cc @slim-bean	3 years ago
Paul Wheeler	85d6794d2b	fluent-bit: Update the Fluent Bit plugin README with correct golang minimum version (#7302 ) What this PR does / why we need it: The Loki plugin for Fluent Bit will not build with golang < 1.17 because of the use of net.IP.IsPrivate in the referenced version of github.com/grafana/dskit/netutil/netutil.go Which issue(s) this PR fixes: No issue exists (this is a minor documentation issue). Special notes for your reviewer: Checklist - [X] Reviewed the `CONTRIBUTING.md` guide - [N/A] Documentation added - [N/A] Tests updated - [N/A] `CHANGELOG.md` updated - [N/A] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Danny Kopping	6c8d04fe78	Completing step 2 of updating loki-build-image (#7796 )	3 years ago
Christian Haudum	d5a86e0ebe	Bump new version 1.2.19 of fluentd-plugin-grafana-loki Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Christian Haudum	e04fb0073b	Add make target for building and pushing the fluentd plugin The fluentd plugin can now easily be published using the `make fluentd-plugin-push` make target. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Irina	bdb0aa0bf4	Promtail: Add go build tag promtail_journal_enabled (#7587 ) What this PR does / why we need it: This PR introduces the new go build tag `promtail_journal_enabled`. This tag is passed to go build if the environment variable `PROMTAIL_JOURNAL_ENABLED=true` is present. It is useful for users with Linux OS and CGO enabled. Currently, if the user has Linux and CGO enabled he has to install `libsystemd-dev` library (`sudo apt install -y libsystemd-dev`) even when he doesn't need journald support.	3 years ago
Furkan Türkal	28d6a97991	fluent-bit: clarify docs, use 1.9.9 (#6464 ) What this PR does / why we need it: * Pins digests for security enhancements * Pass `-ldflags` using `--build-args` * Use fluent-bit image `1.9.9` * Use golang image `1.19.2` * Clarify fluent-bit readme Signed-off-by: Furkan <furkan.turkal@trendyol.com> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Christian Haudum	0f5dd2bceb	Revert "fluentd: Add un-escaping of control characters in JSON" (#7468 ) Reverts grafana/loki#7396 according to discussing on PR	3 years ago
Pablo	5f665651d5	Promtail: Add basic tracing support (#7414 ) We've been using Promtail's HTTP/gRPC server-based targets with real workloads for some weeks now. The server that these spin out is based on [weaveworks commons](https://github.com/weaveworks/common) HTTP/gRPC servers. These include by default some middlewares that if enabled, inject tracing support. We found this useful when tracking down request handling time, or even more, if the caller also support Jaeger tracing support, cross-correlated requests between the upstream server and Promtail's. This PR adds support for enabling tracing the same way Loki does. At the moment, this will just generate traces for the before-mentioned targets. In the future, the quality and completeness of tracing support can grow organically to enrich spans.	3 years ago
Richard Treu	5fe714e466	fluentd: Add un-escaping of control characters in JSON (#7396 )	3 years ago
Danny Kopping	33bf36aa21	Promtail: flush logs on exit (#7400 ) Promtail is using the line-buffered logger (https://github.com/grafana/loki/pull/6954), and log messages were not being printed on exit	3 years ago
李国忠	fb26baa5b1	[new feature] promtail: Add config reload endoint / signal to promtail (#7247 )	3 years ago
Hemanth Krishna	0e54477309	area(promtail): add -check-syntax functionality (#7363 ) Adds `-check-syntax` to promtail which will exit just by validating the config file. Fixes #6862. Signed-off-by: Hemanth Krishna <hkpdev008@gmail.com>	3 years ago
Danny Kopping	c3e92b876c	Restoring lint checks that were previously disabled (#7340 ) I removed these checks in https://github.com/grafana/loki/pull/7337 to not make https://github.com/grafana/loki/pull/7243 overly complex to review. Refactored code to fix valid lint issues, ignored the irrelevant/dangerous ones. I also removed `unused` and `unparam` as these produce low-quality results that are largely useless.	3 years ago
Danny Kopping	355175ff89	Upgrade to Go 1.19 (#7243 ) Upgrade Loki and all related components to Go 1.19.2.	3 years ago
Danny Kopping	6bf5b5d8c1	Use a line-buffered logger to deamplify write syscalls (#6954 ) We initialise a global logger in `pkg/util/log/log.go` and use it extensively throughout the Loki codebase. Every time we write a log message, a `write` syscall is invoked. Syscalls are problematic because they transition the process from userspace to kernelspace, which means: - a context-switch is incurred, which is inherently expensive ([1-2 microseconds](https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/)) - the goroutine executing the code is blocked - the underlying OS thread (_M_ in the go scheduler model) is also blocked - the goroutine has to be rescheduled once the syscall exits - the go scheduler may need to spawn additional OS threads if all are blocked in syscalls - which can also be expensive This change introduces the use of a line-buffered logger. It has a buffer of [256](https://gist.github.com/dannykopping/0704db32c0b08751d1d2494efaa734c2) entries, and once that buffer is filled it will flush to disk. However, a situation will arise in which that buffer remains somewhat empty for a period of time, so there is a periodic flush mechanism, configured to flush every 100ms. There is also a preallocated bytes slice of 10MB which is reused, to avoid excessive slice resizing & garbage collection. This does mean that we could lose up to 256 log messages in case of an ungraceful termination of the process, but this would need to be precisely timed within the 100ms flushes - in other words, the likelihood is low, and generally we shouldn't `kill -9` any Loki process.	3 years ago
Eng Zer Jun	e2842c69c5	Replace deprecated `io/ioutil` with `io` and `os` packages (#7212 ) The `io/ioutil` package has been deprecated in Go 1.16 (See https://pkg.go.dev/io/ioutil). This PR replaces the existing `io/ioutil` functions with their new definitions in `io` and `os` packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	3 years ago
李国忠	a290549a59	Promtail: Add support for max stream limit (#7101 ) What this PR does / why we need it: Add Promtail support to a new feature/configuration named `max_stream_limit` * This new feature limits the max number of streams Promtail can materialize. The new feature is also useful to avoid OOMing Promtail Which issue(s) this PR fixes: Fixes #6968	3 years ago
Callum Styan	4a21923913	update to alpine:3.6.2 (#6979 ) Updates the alpine image version we use to 3.16.2 and will also build/publish a new loki build image Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Christian Simon	88e2705034	Update alpine and golang base images (#6897 ) * Upgrade base images to alpine 3.15.6 everywhere https://alpinelinux.org/posts/Alpine-3.13.12-3.14.8-3.15.6-3.16.2-released.html * Update golang images to 1.18.5	3 years ago
Kaviraj Kanagaraj	0d99d80acd	upgrade `go:1.17.9` to `go:1.18.4` (#6726 )	3 years ago
Rutger	1794a76613	Promtail: Add metrics for journal target (#6105 ) * Add metrics for journal target There are no metrics for the journal target yet, this adds two basic ones: - Total number of lines processed - Total number of errors processing lines (with a reason) Because of the way the journal works, tailing one file with many possible process writing to it, I chose note to expose counts per unit. * Add changelog entry * Fix compilation of (unsupported) journalmanager target on windows * Extract possible errors to constants & add tests	3 years ago
Trevor Wood	23cc93862d	chore(fluentd): resolve rubocop failures (#6359 ) Signed-off-by: Trevor Wood <Trevor.G.Wood@gmail.com>	3 years ago

1 2

94 Commits (7f6c9104e28ae75fb119254773204a0ff0f55bd0)