apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Owen Diehl	f53498c4aa	allows filter-only matchers in series endpoint (#10269 ) We already allow empty matchers `{}` as a special case in series queries to return all series. This PR also allows specifying only filter-style matchers. This is specifically helpful when a tenant is trying to understand all streams which do _not_ have a corresponding label. ``` $ logcli series '{does_not_exist=""}' --analyze-labels --org-id="fake" 2023/08/16 10:13:32 http://localhost:3100/loki/api/v1/series?end=1692206012889997000&match=%7Bdoes_not_exist%3D%22%22%7D&start=1692202412889997000 Total Streams: 6 Unique Labels: 2 ``` Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Shantanu Alsi	b57d260ddc	Upgrade alpine 3.16.5 -> 3.16.7 (#10188 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #10181 Fixes Open SSL vulnerabilities [CVE-2023-1255](https://security.alpinelinux.org/vuln/CVE-2023-1255), [CVE-2023-2650](https://security.alpinelinux.org/vuln/CVE-2023-2650), [CVE-2023-2975](https://security.alpinelinux.org/vuln/CVE-2023-2975), [CVE-2023-3446](https://security.alpinelinux.org/vuln/CVE-2023-3446), [CVE-2023-3817](https://security.alpinelinux.org/vuln/CVE-2023-3817) Alpine Release Notes: [3.16.7](https://www.alpinelinux.org/posts/Alpine-3.15.10-3.16.7-3.17.5-3.18.3-released.html) [3.16.6](https://www.alpinelinux.org/posts/Alpine-3.15.9-3.16.6-3.17.4-3.18.2-released.html) Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Ed Welch	8ca1d1e0a4	Loki: simplify the FetchChunks method and optimize the experimental l2 cache a little more (#10160 ) What this PR does / why we need it: This started as a simple PR to optimize the new L2 chunks cache code such that we don't look up chunks in L1 that we know won't be there, however, the complexity of the FetchChunks method taking both a slice of requested Chunks as well as a slice of requested Keys made this really challenging. Looking at how we use this method, I saw no reason for there to be a requirement to pass both a list of chunks and keys, we only ever want the Chunks and everywhere we used this we were generating the list of keys external to this function from the list of chunks being passed in. The second change was to the implementation of how we process the cache results to look for found vs missing keys. The current implementation required the chunks to be sorted by external key so that it could iterate through them, I tested out a different version of this which uses a map instead of sorting and iterating. The map performance was slightly better in terms of reduced allocations, not enough to make a meaningful difference, but I think this code is easier to understand and reason about, and it removes any requirement for sorting of the passed in Chunks by External key. Benchmarks: Previous: ``` 100 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 3445 334642 ns/op 478074 B/op 7536 allocs/op BenchmarkFetch-32 2947 344439 ns/op 478534 B/op 7537 allocs/op BenchmarkFetch-32 3489 339239 ns/op 478092 B/op 7536 allocs/op BenchmarkFetch-32 3523 340550 ns/op 478003 B/op 7536 allocs/op BenchmarkFetch-32 2935 346810 ns/op 478590 B/op 7537 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 23.948s 1000 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 363 3225519 ns/op 5305481 B/op 78133 allocs/op BenchmarkFetch-32 358 3308456 ns/op 5308299 B/op 78143 allocs/op BenchmarkFetch-32 336 3379067 ns/op 5322534 B/op 78191 allocs/op BenchmarkFetch-32 357 3330118 ns/op 5308854 B/op 78145 allocs/op BenchmarkFetch-32 349 3390415 ns/op 5313813 B/op 78162 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 25.607s 10,000 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 3 334471200 ns/op 639282104 B/op 2630192 allocs/op BenchmarkFetch-32 2 503434741 ns/op 922833480 B/op 3412973 allocs/op BenchmarkFetch-32 1 1064668769 ns/op 1771502096 B/op 5741173 allocs/op BenchmarkFetch-32 1 1027755910 ns/op 1771419128 B/op 5749049 allocs/op BenchmarkFetch-32 1 1027270578 ns/op 1774711872 B/op 5752726 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 23.251s ``` Now ``` 100 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 3856 300569 ns/op 462932 B/op 6214 allocs/op BenchmarkFetch-32 4016 305387 ns/op 462803 B/op 6214 allocs/op BenchmarkFetch-32 3382 303078 ns/op 463201 B/op 6215 allocs/op BenchmarkFetch-32 3319 303367 ns/op 463308 B/op 6215 allocs/op BenchmarkFetch-32 3289 356986 ns/op 463341 B/op 6215 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 22.879s 10,000 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 393 2641470 ns/op 5193282 B/op 65956 allocs/op BenchmarkFetch-32 433 2684287 ns/op 5174832 B/op 65894 allocs/op BenchmarkFetch-32 427 2778092 ns/op 5177334 B/op 65902 allocs/op BenchmarkFetch-32 418 2714370 ns/op 5181193 B/op 65915 allocs/op BenchmarkFetch-32 415 2822163 ns/op 5183436 B/op 65920 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 25.053s 10,000 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 3 362233430 ns/op 638097202 B/op 2510070 allocs/op BenchmarkFetch-32 2 611639304 ns/op 921128060 B/op 3292828 allocs/op BenchmarkFetch-32 1 1022180867 ns/op 1770465496 B/op 5621431 allocs/op BenchmarkFetch-32 1 1253448115 ns/op 1771278456 B/op 5630224 allocs/op BenchmarkFetch-32 1 1189564672 ns/op 1774053064 B/op 5628113 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 23.960s ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Charles Korn	bde65667f7	Upgrade dskit to use packages migrated from weaveworks/common (#10164 ) What this PR does / why we need it: This PR upgrades dskit and replaces use of packages from weaveworks/common with their migrated equivalents in dskit. See https://github.com/grafana/dskit/pull/342 for more details. Note that Loki still uses some packages from weaveworks/common that I haven't migrated (`aws` and `test`) - I'll migrate these separately. If this PR needs to be rebuilt, I used `rewrite.sh` ([source](https://gist.github.com/charleskorn/48efe62a09d6d70f3de30327003df5c5#file-rewrite-sh)) to generate most of these changes. Which issue(s) this PR fixes: (none) Special notes for your reviewer: (none) Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	2 years ago
Michel Hollands	7be9deacf3	Update go version to 1.20.6 (#9913 ) Update the go version. Another PR will follow that uses the new build image --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Paul Rogers	40d0536e45	Migrate tool: Fetch chunks as big batch, only go one by one in case of an error (#9867 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Vladyslav Diachenko	00cf3133e3	disabled indexGateway client for boltdbShipper in migration tool. (#9809 ) What this PR does / why we need it: Without this fix, the migration tool tries to initialize the ingexGateway client, which is unreachable. This leads to the error: `Failed to create source store: error creating index client: index gateway grpc dial: failed to build resolver: passthrough: received empty target in Build()`	2 years ago
Paul Rogers	0da462558f	Migrate individual chunks as opposed to all chunks. (#9780 ) This way, if an error occurs, we get as much as we can. What this PR does / why we need it: In some situations, not all blob data may be accessible. For instance, in the case where we have started migration between object storage buckets, some data may be in the old bucket. If we wanted to migrate data from the new bucket, those chunks will error with a 404, as they are in the old bucket. This PR is so that we continue on and get all of the chunks that we can, rather than stopping with whatever error may arise. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Danny Kopping	f5c0fdf028	Upgrade to latest security release: go1.20.4 (#9370 ) What this PR does / why we need it: https://groups.google.com/g/golang-announce/c/MEb0UyuSMsU Also updated the build image. --------- Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Trevor Whitney	a11921bbb8	Upgrade alpine 3.16.4 -> 3.16.5 (#9264 ) What this PR does / why we need it: Upgrade alpine version to address [openssl CVEs](https://www.alpinelinux.org/posts/Alpine-3.17.3-released.html) Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Trevor Whitney	38c9c6c460	Bump go from 1.20.1 -> 1.20.3 (#9106 ) What this PR does / why we need it: Bumps our go dependency from 1.20.1 to 1.20.3 to address [security fixes](https://groups.google.com/g/golang-announce/c/Xdv6JL9ENs8) Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Ed Welch	7264e0ef53	Loki: Remove global singleton of the tsdb.Store and keep it scoped to the storage.Store (#8928 ) What this PR does / why we need it: I made this change to fix an issue with the cmd/migrate tool which I'm also sneaking a few fixes on in this PR too. We had a global singleton of the tsdb.Store which was leading to the migrate tool reading from and writing to the `source` storage.Store because it created the singleton with an object store client only pointing to the source store. This PR changes the scoping of that singleton to not be global but instead local to each instance of storage.Store Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Michel Hollands	4f8d3240f6	Use 0.28.1 build image and update go and alpine versions (#8583 ) What this PR does / why we need it: - Use the 0.28.1 build image - Update Go version to 1.20.1 - Use Alpine 3.16.4 - Fix linter issues due to updated govet in 1.20.1 - Modify images to not create `/etc/nsswitch.conf` file anymore (available by default on Alpine 3.16.4) - Remove impossible test cases for `TestMemChunk_IteratorBounds`(see https://github.com/grafana/loki/pull/8591#discussion_r1115020111 for context) Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: DylanGuedes <djmgguedes@gmail.com>	2 years ago
Christian Haudum	58e29de988	Update golang to 1.19.5 (#8278 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Danny Kopping	355175ff89	Upgrade to Go 1.19 (#7243 ) Upgrade Loki and all related components to Go 1.19.2.	3 years ago
Callum Styan	4a21923913	update to alpine:3.6.2 (#6979 ) Updates the alpine image version we use to 3.16.2 and will also build/publish a new loki build image Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Christian Simon	88e2705034	Update alpine and golang base images (#6897 ) * Upgrade base images to alpine 3.15.6 everywhere https://alpinelinux.org/posts/Alpine-3.13.12-3.14.8-3.15.6-3.16.2-released.html * Update golang images to 1.18.5	3 years ago
Kaviraj Kanagaraj	0d99d80acd	upgrade `go:1.17.9` to `go:1.18.4` (#6726 )	3 years ago
Ed Welch	3e77423f91	Migrate Tool: Update to work with boltdb-shipper, improve logging output. (#6237 ) * update migrate tool Signed-off-by: Edward Welch <edward.welch@grafana.com> * improve logging don't use any caches Signed-off-by: Edward Welch <edward.welch@grafana.com> * comment out some more log lines Signed-off-by: Edward Welch <edward.welch@grafana.com> * change stat types Signed-off-by: Edward Welch <edward.welch@grafana.com> * update docs and comments as well as set some lower times on a few boltdb-shipper timers Signed-off-by: Edward Welch <edward.welch@grafana.com> * remove changes to the Makefile which weren't meant to make it to the PR Signed-off-by: Edward Welch <edward.welch@grafana.com> * remove some commented code Signed-off-by: Edward Welch <edward.welch@grafana.com> * fix tests Signed-off-by: Edward Welch <edward.welch@grafana.com> * lint Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Christian Simon	9d0fd3a7e3	Upgrade golang version to v1.17.9 (#5899 ) ``` git ls-files '[Dd]ockerfile' \| grep -v ^vendor/ \| xargs sed -i "s/golang:[0-9\\.]*/golang:1.17.9/g" \| ag 'golang:' ```	3 years ago
Cyril Tovena	8f02495453	Refactoring of the Loki storage (#5833 ) * Extract index interface from the series store * moving package around * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * moving package around * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve composite store for more generics index store * fixes first round of tests and ensure coverage is still good * implement series method * Fixes storage tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add test for GetSeries Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Make Loki compile Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint the whole project Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * tsdb util package usage Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more information of why SetChunkFilterer exists for the index Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes ruler/base package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes o2 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Christian Simon	3d941ccccb	Update alpine images to 3.15.4 (#5780 )	3 years ago
Karsten Jeschkies	46956d0bf6	Use dskit/tenant intead of loki/pkg/tenant. (#5724 )	3 years ago
Christian Simon	ea62af58e9	Introduce a ChunkRef type as part of logproto (#5643 ) * Change protobuf definitions * Adapt to an embedded ChunkRef struct * Move the Checksum to the ChunkRef	3 years ago
Cyril Tovena	79f0e349f5	Refactor LogQL syntax into its own package (#5539 ) * Refactor LogQL syntax into its own package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Merge upstream Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/loghttp/series.go Co-authored-by: Susana Ferreira <ssncferreira@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	3 years ago
Cyril Tovena	c8bdb37ef3	update loki go version to 1.17.8 (#5543 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Callum Styan	bfef7bad44	Start refactor of storage (chunks) clients metrics into structs (#5057 ) * Refactor azure metrics to be a struct (+ add an overall client metrics struct), add an azure client metric for total egress bytes. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add changelog entry for new azure metric. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fixes based on Kavi's review. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix one more merge conflict that I missed in the merge commit. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix failing tests. Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Arve Knudsen	a2dd731586	Chore: Add pkg/util/log package (#5187 ) * Chore: Add pkg/util/log package Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix documentation of loki_log_messages_total Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 years ago
JordanRushing	291360ddcb	Simpler new chunk key v12 (#5054 ) * starts hacking new chunk paths * Create new chunk key for S3; update parsing and add basic test and benchmark Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Fix `TestGrpcStore` to work with new chunk key structure Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Add a function to SchemaConfig that returns the correct PeriodConfig for a given time. Signed-off-by: Callum Styan <callumstyan@gmail.com> Add schema config to object client so it we can use the right external key function based on the schema version. Signed-off-by: Callum Styan <callumstyan@gmail.com> Fix failing compactor tests by passing SchemaConfig to chunkClient Remove empty conditional from SchemaForTime Define schema v12; wire-in ChunkPathShardFactor and ChunkPathPeriod as configurable values Add PeriodConfig test steps for new values ChunkPathShardFactor and ChunkPathPeriod in v12 Update test for chunk.NewExternalKey(); remove completed TODO Set defaults for new ChunkPathPeriod and ChunkPathShardFactor SchemaConfig values; change FSObjectClient to use IdentityEncoder instead of Base64Encoder Use IdentityEncoder everywhere we use FSObjectClient for Chunks Add ExternalKey() function to SchemaConfig; update ObjectClient for Chunks * Finish plumbing through the chunk.ExternalKey -> schemaConfig.ExternalKey change. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Clean up lint failures Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up chunk.go and fix tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Break SchemaConfig.ExternalKey() into smaller functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Quickly fix variable name in SchemaConfig.newerExternalKey() Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up ExternalKey conditional logic; add better comments; add tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix a bug where we are prepending userID to legacy Chunk keys but never parsing it; refactor key tests and benchmarks Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Add small SchemaConfig test Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Update docs and CHANGELOG.md Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Correctly return an error when failing to parse a chunk external key Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Revert IdentityEncoder to Base64Encoder after testing Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix assignment in test for linter Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove leftover comments from development Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Change v12 version comment format; remove redundant login in `ParseExternalKey()` Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Change remaining v12 comment style Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Pass chunk.SchemaConfig to new parallel chunk client * Simplify chunk external key prefixes for schema v12 * Fix broken benchmark; add benchmarks for old parsing conditional Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove unneeded lines from upgrading doc * Add benchmarks for root chunk external key parsing functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * memoizes schema number calculations * Resolve linter issue with PeriodConfig receiver name Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Kaviraj	38c42a26c6	Move `cortex/pkg/tenant` dependency to Loki. (#4944 )	3 years ago
Marco Pracucci	c53457feb9	Upgrade Cortex, Prometheus and Thanos (#4830 ) * Upgrade Cortex, Prometheus and Thanos Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed imported ordering Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed data type used for series ref when recovering from checkpoint/WAL Signed-off-by: Marco Pracucci <marco@pracucci.com> * Upgraded Thanos to fix arm build Signed-off-by: Marco Pracucci <marco@pracucci.com> * Run go mod tidy Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Callum Styan	096cf0cfaf	Update golang and loki-build-image image versions. (#4481 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Ed Welch	11e215d79f	Loki: Common Config (#4347 ) * refactor the pkg/util/cfg package to simplify and make it accessable outside of this package. create a pkg/cfg which handles building the Loki config, this is created separately from pkg/util (which is Apache 2 licensed) to maintain proper separations of AGPL and Apache 2 code Update the main.go for several of the apps with the refactoring * refactor how dynamic/common config is loaded Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Add common config logic for path prefix Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Missed a file, update common config with prefix property Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * fix test, remove loki from args Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Clean up test names Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * fix typo in filename, make linter happy Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Update pkg/loki/config_wrapper.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/loki/config_wrapper.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/loki/config_wrapper_test.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * remove temp files in test Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Cyril Tovena	32e55c3043	Fork cortex chunk storage into Loki. (#3842 ) The chunk package has been forked from this commit for future reference and licensing. `09fe17c0e6`	4 years ago
Nick Pillitteri	b830d65bdd	Use the Cortex wrapper for getting tenant ID from a context (#3973 ) This change replaces all uses of the weaveworks/common method for getting tenant/org ID with the Cortex wrapper which performs some extra validation. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	4 years ago
Cyril Tovena	490c19e38c	Deprecate max_look_back_period in the chunk storage. (#3677 ) * Deprecate max_look_back_period in the chunk storage. From now on it will be applied at the frontend, ruler and querier level. The querier validation was missing so I added it. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Ed Welch	6e67d1ea5d	Loki/Promtail: Client Refactor (#3623 ) * move docker driver * move fluent-bit output plugin * move fluentd plugin * move logstash plugin * move promtail * match stage uses simple logql label matcher and filter expressions only * remove remaining logql imports * remove helpers package and move contents to util package * move validation and runtime out of util package * fix imports * move build and cfg packages into util * move ParseRequest function to util package * create a new package for logql models and move marshal/unmarshal code to util folder * move ErrorLabel to logqlmodel * move stats package to logqlmodel * decouple loghttp from logql by moving validation to logql package * move gcplog to tools * move PackedEntryKey constant to logqlmodel * fix stats proto import and rebuild protos * fix windows promtail tests * update drone to new dockerfile locations	4 years ago
Ed Welch	36ec92ce5c	Update Go and Alpine versions (#3481 ) * Upgrade go to 1.16 and alpine to 3.13 * update drone to use new build image version * gomod Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	306cc72438	Revendor/cortex (#3446 ) * Upgrade Cortex in Loki Signed-off-by: Marco Pracucci <marco@pracucci.com> * proto compat * gofmt * changelog for deprecated querier.split-queries-by-day Co-authored-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Jacob Lisi	923671a17e	chore: update cortex to latest and fix refs (#3295 ) * chore: update cortex to latest and fix refs Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * tidy mod file Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>	4 years ago
Ed Welch	b4eeddc710	Loki migrate-tool (#3151 ) * basics working, needs some more cleanup and more flags for config and a Dockerfile Signed-off-by: Edward Welch <edward.welch@grafana.com> * add label matchers start end time params and ability to remap the chunk to a different user Signed-off-by: Edward Welch <edward.welch@grafana.com> * adding dockerfile and makefile instructions Signed-off-by: Edward Welch <edward.welch@grafana.com> * push out some of the limits show errors from index query * for testing have docker container not run migrate command but instead stay running by tailing /dev/null * big overhaul to add parallelization * adding better stats * adding retry logic * tweaking retry logic * tweaking retry logic * work around changes in latest cortex Signed-off-by: Ed Welch <edward.welch@grafana.com> * fix up imports and changes since the last time this was merged added a readme fixed up the makefile added some comments * fixing some of my garbage * fixing more of my garbage * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * tabs not spaces Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago

41 Commits (85e2e52279ecac6dc111d5c113c54d6054d2c922)