apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
renovate[bot]	d43b2de1b4	fix(deps): update module github.com/fsouza/fake-gcs-server to v1.47.7 (#13935 ) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	9 months ago
Marco Pracucci	b88583da7d	feat: upgrade prometheus (#13671 ) Signed-off-by: Marco Pracucci <marco@pracucci.com> Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	10 months ago
Paulin Todev	d54e0871d4	chore(deps): Update Prometheus in Loki and Promtail (#12245 ) This PR updates the Prometheus dependency in Loki and Promtail. The PR is required so that we can also update Grafana Agent to the latest Prometheus. Unfortunately, Promtail and Loki share the same go.mod file.	1 year ago
Sandeep Sukhani	6069df8f7d	ingestion: native otlp ingestion support (#10727 ) What this PR does / why we need it: Add support for natively supporting logs ingestion in OTLP format. `/otlp/v1/logs` is the new endpoint where users can push logs in OTLP format. It accepts logs serialized in JSON or proto format. Since OTEL format is very different than what Loki storage model, here is how data in OTEL format will be mapped to Loki data model: * Index labels: The Resource Attributes map quite well to Index labels in Loki since both usually identify the source of the logs. The problem however is that Resource attributes in OTLP can have an unbounded number of values while Loki has a default limit of having up to 30 labels. Since Index labels in Loki can largely drive the kind of querying experience the users are going to have, we have chosen select attributes which would be picked as Index Labels. The ones that are not picked up as Index labels would be stored as Structured Metadata with each log entry. * Timestamp: LogRecord.TimeUnixNano * LogLine: LogRecord.Body holds the body of the log. However, since Loki only supports Log body in string format, we will stringify non-string values using [AsString method from OTEL collector lib](`ab3d6c5b64/pdata/pcommon/value.go (L353)`). * Structured Metadata: Anything which can’t be stored in Index labels and LogLine. Here is a non-exhaustive list of what will be stored in Structured Metadata to give a sense of what it will hold: * Resource Attributes not stored as Index labels is replicated and stored with each log entry. * Everything under InstrumentationScope is replicated and stored with each log entry. * Everything under LogRecord except LogRecord.Body, LogRecord.TimeUnixNano and sometimes LogRecord.ObservedTimestamp. NOTES: * Since Loki does not support `.` or any other special characters other than `_` in label names, we replace all non-supported characters with `_`. * Since Loki only supports string in values of Index Labels and Structured Metadata, all the complex types are converted as follows: * Map would be flattened into label keys using `_` as separator, same as how we do it in [json parser in LogQL](https://grafana.com/docs/loki/latest/query/log_queries/#json). * Everything else is stringified using [AsString method from OTEL collector lib](`ab3d6c5b64/pdata/pcommon/value.go (L353)`) Special notes for your reviewer: I will open follow-up PRs for: * Documentation * Make blessed attributes list configurable per tenant. Checklist - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label	2 years ago
Shantanu Alsi	586b9042c6	Upgrade Thanos objstore, dskit and other modules (#10366 ) What this PR does / why we need it: Upgrades various common modules including dskit, thanos objstore, prometheus, etc. Fixes errors/ changes resulting due to module upgrades. This is required in order to eventually upgrade BEL dependency in GEL to point to master. Which issue(s) this PR fixes: None Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
dependabot[bot]	89d00fbccd	Bump golang.org/x/oauth2 from 0.7.0 to 0.10.0 (#9903 ) Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.7.0 to 0.10.0. <details> <summary>Commits</summary> <ul> <li><a href="`ec5679f607`"><code>ec5679f</code></a> go.mod: update golang.org/x dependencies</li> <li><a href="`989acb1bfe`"><code>989acb1</code></a> all: update dependencies to their latest versions</li> <li><a href="`2323c81c8d`"><code>2323c81</code></a> go.mod: update golang.org/x dependencies</li> <li><a href="`839de2255f`"><code>839de22</code></a> google: don't check for IsNotExist for well-known file</li> <li><a href="`0690208dba`"><code>0690208</code></a> go.mod: update golang.org/x dependencies</li> <li><a href="`451d5d662f`"><code>451d5d6</code></a> internal: remove repeated definite articles</li> <li><a href="`cfe200d5bb`"><code>cfe200d</code></a> oauth2: parse RFC 6749 error response</li> <li>See full diff in <a href="https://github.com/golang/oauth2/compare/v0.7.0...v0.10.0">compare view</a></li> </ul> </details> <br />	2 years ago
Peter Štibraný	90a1d4593e	Update Prometheus dependency (#9205 )	2 years ago
Travis Patterson	71979f0f42	Fix pubsub pull target (#8281 )	2 years ago
dependabot[bot]	fd72aa8908	Bump cloud.google.com/go/storage from 1.27.0 to 1.29.0 (#8276 ) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Robert Fratto	85392a9728	Update Prometheus dependency to latest release (v2.40.4) (#7826 ) Closes #7811, which is needed for Grafana Agent to update to v2.40 and add support for native histograms. I did not add support for native histograms to Loki, sorry :)	3 years ago
Robert Fratto	2208ea7147	Update Prometheus dependency to latest release (v2.39.1) (#7425 )	3 years ago
dmitri-lerko	e06c0790ba	promtail: GCPLog fixing `failed to receive pubsub messages` error (#6965 ) 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: Upgrades PubSub to the latest. Myself and other users observe frequent disconnects from PubSub which was addressed upstream. High load production testing has proven that this fixes the problem. As an extra bonus, PubSub seems to ingest ~33% faster with the latest library. Which issue(s) this PR fixes: Fixes #6505 Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Arve Knudsen	d87dbf9f8a	Chore: Upgrade to latest Cortex (#5085 ) * Chore: Upgrade to latest Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Chore: Upgrade to Prometheus@692a54649ed7 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build after upgrading dskit Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 years ago
Marco Pracucci	c53457feb9	Upgrade Cortex, Prometheus and Thanos (#4830 ) * Upgrade Cortex, Prometheus and Thanos Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed imported ordering Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed data type used for series ref when recovering from checkpoint/WAL Signed-off-by: Marco Pracucci <marco@pracucci.com> * Upgraded Thanos to fix arm build Signed-off-by: Marco Pracucci <marco@pracucci.com> * Run go mod tidy Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Mauro Stettler	4b59556342	Update cortex to newer version (#4478 ) * update cortex and thanos to newer version * pin k8s.io/apimachinery * fixes for newer libs * add missing method to mock wal data collector * fixing test by using different gokit level pkg * make check-mod	4 years ago
Nick Pillitteri	79a7bdbabc	Chore: Use services and modules from grafana/dskit (#4196 ) * Update Cortex vendor to bcf15611345da65e03604a08a0213aaaaac49e54 Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Use services and modules packages from grafana/dskit Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Update promtail test to set default weaveworks server network type Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Punctuation Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	4 years ago
Arve Knudsen	3e385464c6	Chore: Upgrade Prometheus and Cortex (#4017 ) * Chore: Upgrade Prometheus Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Upgrade Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Upgrade Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build errors Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build errors Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build errors Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Sandeep Sukhani	d765921735	revendor cortex to latest master (#3771 )	4 years ago
Michel Hollands	9137ef921b	Update cortex to 1.8 (#3627 ) * Update Cortex to 1.8.1 Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Tidy up Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	4 years ago
Owen Diehl	306cc72438	Revendor/cortex (#3446 ) * Upgrade Cortex in Loki Signed-off-by: Marco Pracucci <marco@pracucci.com> * proto compat * gofmt * changelog for deprecated querier.split-queries-by-day Co-authored-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Jacob Lisi	c3d1937eab	update vendored cortex and add new replace overrides (#3256 ) * update vendored cortex and add new replace overrides Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update upstream function to new package Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update all moved functions Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update math to util_math Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>	4 years ago
Kaviraj	6cc41f92f8	Gcplog targetmanager (#3083 ) * Add new target manager(PubsubTarget) for promtail Lets you to scrape log entries from pubsub topic and send it to lok - Basic `Target` and `TargetManager` - Minimum config to make it work * make the receive loop on pubsub manager work * Add mmock pubsub server for testing * Remove debugging logs * Minor changes to messages pushing into loki * Basic formatter to convert GCP log entry to loki log entry * Reverting go.mod, go.sum and vendors to master branch * only pubsub dependencies * rebuild proto * Fix protobuf diffs * Remove debug printfs * Add basic formatter for GCP log entry - Create uniq label to create stream without out-of-order timestamp - extract instance id and timestamp from log entry * - Small fixes to formatter - Config changes on pubsub manager for testing * - Formatting changes - Rewrite timestamp to avoid out-of-order errors on loki - Minor config changes to Pubsub Target manager * Fix target formatter tests * PR remarks * Minor refactoring and following changes - Rename Pubsub* entities to Gcplog* - Rename pubsub package into gcplog - Add flag `KeepIncomingTimestamp` to toggle between either keeping incoming timestamps or rewriting with current timestamp - Tests for gcplog target - Docs and basic metrics to gcplog target * Add docs for cloud configure for gcplog. * Add docs for cloud configure for gcplog. * Review feedback * Checkin generated files * Making linters happy Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Sandeep Sukhani	071a20803b	Revendor cortex to latest master (#2794 )	5 years ago
Peter Štibraný	e4a5360586	Update vendored Cortex to master (1.1+) (#2149 ) * Updated Cortex to master (post 1.1) Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use new modules system from Cortex. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Treat query errors reported by chunk store as client errors. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed obsolete tests. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix race for ct variable. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix test. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Peter Štibraný	d4cae667ae	loki: use new runtimeconfig package from Cortex (#1484 ) * loki: use new runtimeconfig package from Cortex runtimeconfig is a generalization of previous "overrides" mechanism, and can be used not for per-tenant limits, but other parts of Loki too (only one additional usage for now is "multi" KV client). Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy, go mod vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed RuntimeConfig from Ingester dependencies. Ingester already depends on RuntimeConfig via Store and Overrides mechanism. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed extra new line. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Remove unused method. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix broken code after rebasing. Removed incorrect comment. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Updated docs with section about runtime configuration. Mostly copied from Cortex. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy, go mod vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Updated Cortex to current master. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Go mod tidy, vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Joe Elliott	a48230d343	Reverted cloud.google.com/go to 0.44.1 (#1097 ) Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Joe Elliott	fc6c5c063f	dep => go mod (#1062 ) * go mod files added. dep removed Signed-off-by: Joe Elliott <number101010@gmail.com> * Magically got prometheus version to stick Signed-off-by: Joe Elliott <number101010@gmail.com> * Cortex updated and prometheus updated Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated prometheus dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added additional deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Added two replaces from Gopkg.toml. Tests passing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated build image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golangci-lint to use vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod makefile step and referenced in drone and circle ci configs Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated loki-build-image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed linting error. Force go111module on for linting Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod vendor Signed-off-by: Joe Elliott <number101010@gmail.com> * Forced the inclusion of modtimevfs Signed-off-by: Joe Elliott <number101010@gmail.com> * Pin client-go due to issue with v12 go.mod Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod to drone Signed-off-by: Joe Elliott <number101010@gmail.com> * Re-readded correct golang client Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Pinned golang/x/net to avoid proxy errors Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed check-mod from all. Not necessary for tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go.sum to match new pin Signed-off-by: Joe Elliott <number101010@gmail.com> * Upgraded proto to match build image Signed-off-by: Joe Elliott <number101010@gmail.com> * Force check-mod to wait til after test and lint are successful Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to go builds to force usage of vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Turn on gomodules on all builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Added mod vendor to go builds to force usage of vendored dependencies" This reverts commit 65865a24c9a23133e0fa52942f2828ead7c22147. * Moved builds out of the gopath to enforce vendor usage Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Turn on gomodules on all builds" This reverts commit b5847f0158e928e935e0b3c1b1d4eaba840ca3dc. * Explicitly choose build image for docker driver Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golang.org/x/sys to match prometheus's version to fix promtail windows compilation Signed-off-by: Joe Elliott <number101010@gmail.com> * Added fluentbit dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added dependency management clause Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go version in contributing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added phony makefile targets Signed-off-by: Joe Elliott <number101010@gmail.com> * Circle CI is increasingly failing linting Signed-off-by: Joe Elliott <number101010@gmail.com> * Force vendored deps on builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Added logcli to gitignore Signed-off-by: Joe Elliott <number101010@gmail.com> * Reduced concurrency to help CircleCI Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved drone builds out of GOPATH to force use of go module Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to prevent go clean from redownloading all packages Signed-off-by: Joe Elliott <number101010@gmail.com> * Added to test as well Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod-vendor to final go commands Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved fluent-bit out of GO_PATH to force go modules Signed-off-by: Joe Elliott <number101010@gmail.com> * Pass mod vendor to the last holdout: go generate Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated ugorji/go to 1.17 to avoid version regression Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy and go mod vendor for ugorji/go Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Adam Shannon	eb390523e3	all: update dependencies (#211 ) * all: update dependencies * Update some more dependancies and pin the right versions for Prometheus deps. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	9f257c591a	[WIP] Persistence (#44 ) Persistence chunks using the Cortex chunk store.	7 years ago

29 Commits (b5ac6a0258be51a6d6c3a7743e498dc40014b64b)