apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Paul Rogers	5f670a4b18	chore(deps): Move from base-nossl to static (#15203 )	6 months ago
Paul Rogers	f2c2a22bdc	chore: Preparation for incoming static code analysis CI check (#15164 ) Co-authored-by: Danny Cooper <danny.cooper@grafana.com>	6 months ago
Paul Rogers	6875c872a6	chore: Move to Go 1.23.1, along with golangci update (#14294 )	8 months ago
Paul Rogers	29110feb99	build: Loki build and Docker updates (#13325 )	10 months ago
Paul Rogers	ca2f11d93e	chore: Update Docker images to Go 1.22.5 (#13398 )	11 months ago
Owen Diehl	bc78d13d9b	feat(blooms): compute chunks once (#12664 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Callum Styan	71a8f2c2b1	fix: updated all dockerfiles go1.22 (#12708 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	1 year ago
Periklis Tsirakidis	131144bcab	chore(deps): Bump golang to 1.21.9 (#12467 )	1 year ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Trevor Whitney	8b48a18d77	Bump build image (#11608 ) What this PR does / why we need it: Bumps alpine in the loki build image and ass the build image to the `scan-vulnerabilities` target to make it easier to scan in the future --------- Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	1 year ago
Christian Haudum	62176339c4	Bloom querying: Propagate line filters from querier to bloom filter (#11492 ) In order to query the bloom filters with the search strings from the user query, we need to propagate the line filters through the different implementations of the store, namely through the GetChunks function. In order to avoid converting data structures, the line filter from the syntax tree is used as custom protobuf type, because it is passed from querier to index gateway and from index gateway to bloom gateway. So far, the line filters request have not been tokenized to n-grams. This PR adds the ability to read the n-gram length from the block schema at query time and tokenizes the line filters from the input just ahead of searching the block. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
renovate[bot]	10fe48b815	chore(deps): update alpine docker tag to v3.18.4 (main) (#11167 ) [![Mend Renovate logo banner](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: \| Package \| Type \| Update \| Change \| \|---\|---\|---\|---\| \| alpine \| final \| patch \| `3.18.3` -> `3.18.4` \| \| alpine \| final \| minor \| `3.16.7` -> `3.18.4` \| --- ### Configuration 📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied. ♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 Ignore: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/grafana/loki). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40Ni4wIiwidXBkYXRlZEluVmVyIjoiMzcuNTkuOCIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2 years ago
Michel Hollands	380f902c30	Change default of metrics.namespace (#11110 ) What this PR does / why we need it: Change the default for a subset of metrics. After this all Loki metrics should start with "loki_". This changes the following metrics: cortex_distributor_ingester_clients cortex_dns_failures_total cortex_dns_lookups_total cortex_dns_provider_results cortex_frontend_query_range_duration_seconds_bucket cortex_frontend_query_range_duration_seconds_count cortex_frontend_query_range_duration_seconds_sum cortex_ingester_flush_queue_length cortex_kv_request_duration_seconds_bucket cortex_kv_request_duration_seconds_count cortex_kv_request_duration_seconds_sum cortex_member_consul_heartbeats_total cortex_prometheus_last_evaluation_samples cortex_prometheus_notifications_alertmanagers_discovered cortex_prometheus_notifications_dropped_total cortex_prometheus_notifications_errors_total cortex_prometheus_notifications_latency_seconds cortex_prometheus_notifications_latency_seconds_count cortex_prometheus_notifications_latency_seconds_sum cortex_prometheus_notifications_queue_capacity cortex_prometheus_notifications_queue_length cortex_prometheus_notifications_sent_total cortex_prometheus_rule_evaluation_duration_seconds cortex_prometheus_rule_evaluation_duration_seconds_count cortex_prometheus_rule_evaluation_duration_seconds_sum cortex_prometheus_rule_evaluation_failures_total cortex_prometheus_rule_evaluations_total cortex_prometheus_rule_group_duration_seconds cortex_prometheus_rule_group_duration_seconds_count cortex_prometheus_rule_group_duration_seconds_sum cortex_prometheus_rule_group_interval_seconds cortex_prometheus_rule_group_iterations_missed_total cortex_prometheus_rule_group_iterations_total cortex_prometheus_rule_group_last_duration_seconds cortex_prometheus_rule_group_last_evaluation_timestamp_seconds cortex_prometheus_rule_group_rules cortex_query_frontend_connected_schedulers cortex_query_frontend_queries_in_progress cortex_query_frontend_retries_bucket cortex_query_frontend_retries_count cortex_query_frontend_retries_sum cortex_query_scheduler_connected_frontend_clients cortex_query_scheduler_connected_querier_clients cortex_query_scheduler_inflight_requests cortex_query_scheduler_inflight_requests_count cortex_query_scheduler_inflight_requests_sum cortex_query_scheduler_queue_duration_seconds_bucket cortex_query_scheduler_queue_duration_seconds_count cortex_query_scheduler_queue_duration_seconds_sum cortex_query_scheduler_queue_length cortex_query_scheduler_running cortex_quota_cgroup_cpu_max cortex_quota_cgroup_cpu_period cortex_quota_cpu_count cortex_quota_gomaxprocs cortex_ring_member_heartbeats_total cortex_ring_member_tokens_owned cortex_ring_member_tokens_to_own cortex_ring_members cortex_ring_oldest_member_timestamp cortex_ring_tokens_total cortex_ruler_client_request_duration_seconds_bucket cortex_ruler_client_request_duration_seconds_count cortex_ruler_client_request_duration_seconds_sum cortex_ruler_clients cortex_ruler_config_last_reload_successful cortex_ruler_config_last_reload_successful_seconds cortex_ruler_config_updates_total cortex_ruler_managers_total cortex_ruler_ring_check_errors_total cortex_ruler_sync_rules_total Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [X] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com>	2 years ago
Michel Hollands	8628b15624	Use metrics namespace for more metrics (#11025 ) What this PR does / why we need it: Use the metrics namespace setting instead of hardcoding to `cortex`. This is a follow up to (and based on) https://github.com/grafana/loki/pull/11014. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. <!-- TODO(salvacorts): Add example PR --> --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com>	2 years ago
Ashwanth	98551ceb9a	use go 1.21.3 and go-grpc v1.56.3 (#10869 ) What this PR does / why we need it: Fixes CVE-2023-39325 / CVE-2023-44487 https://github.com/grpc/grpc-go/releases/tag/v1.56.3 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Michel Hollands	eb3cff4625	Update Go version to 1.21.2 in Dockerfiles (#10805 ) What this PR does / why we need it: Update the go version in the Dockerfiles to 1.21.2 Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com>	2 years ago
Christian Haudum	2e775f3f44	chore(storage): Restructure shipper packages (#10729 ) What this PR does / why we need it: This PR restructures the code for the shipper component of the Loki storage layer. New package layout: ```console $ tree -d pkg/storage/stores/shipper pkg/storage/stores/shipper └── indexshipper ├── boltdb │ └── compactor ├── compactor │ ├── client │ │ └── grpc │ ├── deletion │ ├── deletionmode │ ├── generationnumber │ └── retention ├── downloads ├── gatewayclient ├── index ├── indexgateway ├── storage ├── testutil ├── tsdb │ ├── index │ ├── testdata │ └── testutil ├── uploads └── util 23 directories ``` * TSDB and BoltDB specific code is under `./pkg/storage/stores/shipper/indexshipper/tsdb` and `./pkg/storage/stores/shipper/indexshipper/boltdb` respectively. * Common code for both TSDB and BoltDB is directly under `./pkg/storage/stores/shipper/indexshipper` and subdirectories, such as `uploads/`, `downloads/`, `compactor/`, ... Special notes for your reviewer: This PR is identical to https://github.com/grafana/loki/pull/10724 except of the package `pkg/storage/stores/indexshipper` the package is `pkg/storage/stores/shipper/indexshipper` (one level deeper). Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Ashwanth	2902f986a4	cache: Enable embedded cache if no other cache is configured. (#10620 ) What this PR does / why we need it: FIFO cache is replaced by embedded-cache in https://github.com/grafana/loki/pull/6821 This PR enables embedded cache for query range results, chunks cache, index_stats_results and volume_results if no other cache is explicitly configured (redis, memcache). Special notes for your reviewer: Also removes a deprecated flag `cache-split-interval` Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Dylan Guedes	4681386775	Update alpine versions (#10573 ) What this PR does / why we need it: Update alpine versions used by our images to `3.18.3` Which issue(s) this PR fixes: N/A Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Christian Haudum	2b6bdc2a29	chore(storage): Simplify store interfaces and abstractions (pt 2) (#10454 ) What this PR does / why we need it: * Remove preemptive interface from indexReaderWriter and make it an exported struct * Compose IngesterStore and QuerierStore The ingester and querier do not require a store that implements the full interface, but can accept stores that are a subset of the interfaces. Therefore we can make specific interfaces that are accepted by the ingester and querier constructors. * Split up ChunkFetcher interface This commit splits the ChunkFetcher interface into a ChunkFetcher and a ChunkFetcherProvider interface. This has the advantage that methods can allow accepting smaller interfaces. This commit also renames the `GetChunkRefs()` method of the ChunkFetcher interface into `GetChunks()`, because it returns a `[][]chunk.Chunk`, not a `[]chunk.ChunkRef` type. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Owen Diehl	f53498c4aa	allows filter-only matchers in series endpoint (#10269 ) We already allow empty matchers `{}` as a special case in series queries to return all series. This PR also allows specifying only filter-style matchers. This is specifically helpful when a tenant is trying to understand all streams which do _not_ have a corresponding label. ``` $ logcli series '{does_not_exist=""}' --analyze-labels --org-id="fake" 2023/08/16 10:13:32 http://localhost:3100/loki/api/v1/series?end=1692206012889997000&match=%7Bdoes_not_exist%3D%22%22%7D&start=1692202412889997000 Total Streams: 6 Unique Labels: 2 ``` Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Shantanu Alsi	b57d260ddc	Upgrade alpine 3.16.5 -> 3.16.7 (#10188 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #10181 Fixes Open SSL vulnerabilities [CVE-2023-1255](https://security.alpinelinux.org/vuln/CVE-2023-1255), [CVE-2023-2650](https://security.alpinelinux.org/vuln/CVE-2023-2650), [CVE-2023-2975](https://security.alpinelinux.org/vuln/CVE-2023-2975), [CVE-2023-3446](https://security.alpinelinux.org/vuln/CVE-2023-3446), [CVE-2023-3817](https://security.alpinelinux.org/vuln/CVE-2023-3817) Alpine Release Notes: [3.16.7](https://www.alpinelinux.org/posts/Alpine-3.15.10-3.16.7-3.17.5-3.18.3-released.html) [3.16.6](https://www.alpinelinux.org/posts/Alpine-3.15.9-3.16.6-3.17.4-3.18.2-released.html) Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Ed Welch	8ca1d1e0a4	Loki: simplify the FetchChunks method and optimize the experimental l2 cache a little more (#10160 ) What this PR does / why we need it: This started as a simple PR to optimize the new L2 chunks cache code such that we don't look up chunks in L1 that we know won't be there, however, the complexity of the FetchChunks method taking both a slice of requested Chunks as well as a slice of requested Keys made this really challenging. Looking at how we use this method, I saw no reason for there to be a requirement to pass both a list of chunks and keys, we only ever want the Chunks and everywhere we used this we were generating the list of keys external to this function from the list of chunks being passed in. The second change was to the implementation of how we process the cache results to look for found vs missing keys. The current implementation required the chunks to be sorted by external key so that it could iterate through them, I tested out a different version of this which uses a map instead of sorting and iterating. The map performance was slightly better in terms of reduced allocations, not enough to make a meaningful difference, but I think this code is easier to understand and reason about, and it removes any requirement for sorting of the passed in Chunks by External key. Benchmarks: Previous: ``` 100 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 3445 334642 ns/op 478074 B/op 7536 allocs/op BenchmarkFetch-32 2947 344439 ns/op 478534 B/op 7537 allocs/op BenchmarkFetch-32 3489 339239 ns/op 478092 B/op 7536 allocs/op BenchmarkFetch-32 3523 340550 ns/op 478003 B/op 7536 allocs/op BenchmarkFetch-32 2935 346810 ns/op 478590 B/op 7537 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 23.948s 1000 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 363 3225519 ns/op 5305481 B/op 78133 allocs/op BenchmarkFetch-32 358 3308456 ns/op 5308299 B/op 78143 allocs/op BenchmarkFetch-32 336 3379067 ns/op 5322534 B/op 78191 allocs/op BenchmarkFetch-32 357 3330118 ns/op 5308854 B/op 78145 allocs/op BenchmarkFetch-32 349 3390415 ns/op 5313813 B/op 78162 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 25.607s 10,000 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 3 334471200 ns/op 639282104 B/op 2630192 allocs/op BenchmarkFetch-32 2 503434741 ns/op 922833480 B/op 3412973 allocs/op BenchmarkFetch-32 1 1064668769 ns/op 1771502096 B/op 5741173 allocs/op BenchmarkFetch-32 1 1027755910 ns/op 1771419128 B/op 5749049 allocs/op BenchmarkFetch-32 1 1027270578 ns/op 1774711872 B/op 5752726 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 23.251s ``` Now ``` 100 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 3856 300569 ns/op 462932 B/op 6214 allocs/op BenchmarkFetch-32 4016 305387 ns/op 462803 B/op 6214 allocs/op BenchmarkFetch-32 3382 303078 ns/op 463201 B/op 6215 allocs/op BenchmarkFetch-32 3319 303367 ns/op 463308 B/op 6215 allocs/op BenchmarkFetch-32 3289 356986 ns/op 463341 B/op 6215 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 22.879s 10,000 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 393 2641470 ns/op 5193282 B/op 65956 allocs/op BenchmarkFetch-32 433 2684287 ns/op 5174832 B/op 65894 allocs/op BenchmarkFetch-32 427 2778092 ns/op 5177334 B/op 65902 allocs/op BenchmarkFetch-32 418 2714370 ns/op 5181193 B/op 65915 allocs/op BenchmarkFetch-32 415 2822163 ns/op 5183436 B/op 65920 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 25.053s 10,000 chunks ❯ go test -bench=Fetch -count=5 goos: linuxbench=Fetch -count=5 ─╯ goarch: amd64 pkg: github.com/grafana/loki/pkg/storage/chunk/fetcher cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkFetch-32 3 362233430 ns/op 638097202 B/op 2510070 allocs/op BenchmarkFetch-32 2 611639304 ns/op 921128060 B/op 3292828 allocs/op BenchmarkFetch-32 1 1022180867 ns/op 1770465496 B/op 5621431 allocs/op BenchmarkFetch-32 1 1253448115 ns/op 1771278456 B/op 5630224 allocs/op BenchmarkFetch-32 1 1189564672 ns/op 1774053064 B/op 5628113 allocs/op PASS ok github.com/grafana/loki/pkg/storage/chunk/fetcher 23.960s ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Charles Korn	bde65667f7	Upgrade dskit to use packages migrated from weaveworks/common (#10164 ) What this PR does / why we need it: This PR upgrades dskit and replaces use of packages from weaveworks/common with their migrated equivalents in dskit. See https://github.com/grafana/dskit/pull/342 for more details. Note that Loki still uses some packages from weaveworks/common that I haven't migrated (`aws` and `test`) - I'll migrate these separately. If this PR needs to be rebuilt, I used `rewrite.sh` ([source](https://gist.github.com/charleskorn/48efe62a09d6d70f3de30327003df5c5#file-rewrite-sh)) to generate most of these changes. Which issue(s) this PR fixes: (none) Special notes for your reviewer: (none) Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	2 years ago
Michel Hollands	7be9deacf3	Update go version to 1.20.6 (#9913 ) Update the go version. Another PR will follow that uses the new build image --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Paul Rogers	40d0536e45	Migrate tool: Fetch chunks as big batch, only go one by one in case of an error (#9867 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Vladyslav Diachenko	00cf3133e3	disabled indexGateway client for boltdbShipper in migration tool. (#9809 ) What this PR does / why we need it: Without this fix, the migration tool tries to initialize the ingexGateway client, which is unreachable. This leads to the error: `Failed to create source store: error creating index client: index gateway grpc dial: failed to build resolver: passthrough: received empty target in Build()`	2 years ago
Paul Rogers	0da462558f	Migrate individual chunks as opposed to all chunks. (#9780 ) This way, if an error occurs, we get as much as we can. What this PR does / why we need it: In some situations, not all blob data may be accessible. For instance, in the case where we have started migration between object storage buckets, some data may be in the old bucket. If we wanted to migrate data from the new bucket, those chunks will error with a 404, as they are in the old bucket. This PR is so that we continue on and get all of the chunks that we can, rather than stopping with whatever error may arise. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Danny Kopping	f5c0fdf028	Upgrade to latest security release: go1.20.4 (#9370 ) What this PR does / why we need it: https://groups.google.com/g/golang-announce/c/MEb0UyuSMsU Also updated the build image. --------- Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Trevor Whitney	a11921bbb8	Upgrade alpine 3.16.4 -> 3.16.5 (#9264 ) What this PR does / why we need it: Upgrade alpine version to address [openssl CVEs](https://www.alpinelinux.org/posts/Alpine-3.17.3-released.html) Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Trevor Whitney	38c9c6c460	Bump go from 1.20.1 -> 1.20.3 (#9106 ) What this PR does / why we need it: Bumps our go dependency from 1.20.1 to 1.20.3 to address [security fixes](https://groups.google.com/g/golang-announce/c/Xdv6JL9ENs8) Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Ed Welch	7264e0ef53	Loki: Remove global singleton of the tsdb.Store and keep it scoped to the storage.Store (#8928 ) What this PR does / why we need it: I made this change to fix an issue with the cmd/migrate tool which I'm also sneaking a few fixes on in this PR too. We had a global singleton of the tsdb.Store which was leading to the migrate tool reading from and writing to the `source` storage.Store because it created the singleton with an object store client only pointing to the source store. This PR changes the scoping of that singleton to not be global but instead local to each instance of storage.Store Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Michel Hollands	4f8d3240f6	Use 0.28.1 build image and update go and alpine versions (#8583 ) What this PR does / why we need it: - Use the 0.28.1 build image - Update Go version to 1.20.1 - Use Alpine 3.16.4 - Fix linter issues due to updated govet in 1.20.1 - Modify images to not create `/etc/nsswitch.conf` file anymore (available by default on Alpine 3.16.4) - Remove impossible test cases for `TestMemChunk_IteratorBounds`(see https://github.com/grafana/loki/pull/8591#discussion_r1115020111 for context) Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: DylanGuedes <djmgguedes@gmail.com>	2 years ago
Christian Haudum	58e29de988	Update golang to 1.19.5 (#8278 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Danny Kopping	355175ff89	Upgrade to Go 1.19 (#7243 ) Upgrade Loki and all related components to Go 1.19.2.	3 years ago
Callum Styan	4a21923913	update to alpine:3.6.2 (#6979 ) Updates the alpine image version we use to 3.16.2 and will also build/publish a new loki build image Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Christian Simon	88e2705034	Update alpine and golang base images (#6897 ) * Upgrade base images to alpine 3.15.6 everywhere https://alpinelinux.org/posts/Alpine-3.13.12-3.14.8-3.15.6-3.16.2-released.html * Update golang images to 1.18.5	3 years ago
Kaviraj Kanagaraj	0d99d80acd	upgrade `go:1.17.9` to `go:1.18.4` (#6726 )	3 years ago
Ed Welch	3e77423f91	Migrate Tool: Update to work with boltdb-shipper, improve logging output. (#6237 ) * update migrate tool Signed-off-by: Edward Welch <edward.welch@grafana.com> * improve logging don't use any caches Signed-off-by: Edward Welch <edward.welch@grafana.com> * comment out some more log lines Signed-off-by: Edward Welch <edward.welch@grafana.com> * change stat types Signed-off-by: Edward Welch <edward.welch@grafana.com> * update docs and comments as well as set some lower times on a few boltdb-shipper timers Signed-off-by: Edward Welch <edward.welch@grafana.com> * remove changes to the Makefile which weren't meant to make it to the PR Signed-off-by: Edward Welch <edward.welch@grafana.com> * remove some commented code Signed-off-by: Edward Welch <edward.welch@grafana.com> * fix tests Signed-off-by: Edward Welch <edward.welch@grafana.com> * lint Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Christian Simon	9d0fd3a7e3	Upgrade golang version to v1.17.9 (#5899 ) ``` git ls-files '[Dd]ockerfile' \| grep -v ^vendor/ \| xargs sed -i "s/golang:[0-9\\.]*/golang:1.17.9/g" \| ag 'golang:' ```	3 years ago
Cyril Tovena	8f02495453	Refactoring of the Loki storage (#5833 ) * Extract index interface from the series store * moving package around * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * moving package around * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve composite store for more generics index store * fixes first round of tests and ensure coverage is still good * implement series method * Fixes storage tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add test for GetSeries Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Make Loki compile Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint the whole project Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * tsdb util package usage Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more information of why SetChunkFilterer exists for the index Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes ruler/base package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes o2 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Christian Simon	3d941ccccb	Update alpine images to 3.15.4 (#5780 )	3 years ago
Karsten Jeschkies	46956d0bf6	Use dskit/tenant intead of loki/pkg/tenant. (#5724 )	3 years ago
Christian Simon	ea62af58e9	Introduce a ChunkRef type as part of logproto (#5643 ) * Change protobuf definitions * Adapt to an embedded ChunkRef struct * Move the Checksum to the ChunkRef	3 years ago
Cyril Tovena	79f0e349f5	Refactor LogQL syntax into its own package (#5539 ) * Refactor LogQL syntax into its own package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Merge upstream Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/loghttp/series.go Co-authored-by: Susana Ferreira <ssncferreira@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	3 years ago
Cyril Tovena	c8bdb37ef3	update loki go version to 1.17.8 (#5543 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Callum Styan	bfef7bad44	Start refactor of storage (chunks) clients metrics into structs (#5057 ) * Refactor azure metrics to be a struct (+ add an overall client metrics struct), add an azure client metric for total egress bytes. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add changelog entry for new azure metric. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fixes based on Kavi's review. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix one more merge conflict that I missed in the merge commit. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix failing tests. Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Arve Knudsen	a2dd731586	Chore: Add pkg/util/log package (#5187 ) * Chore: Add pkg/util/log package Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix documentation of loki_log_messages_total Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 years ago
JordanRushing	291360ddcb	Simpler new chunk key v12 (#5054 ) * starts hacking new chunk paths * Create new chunk key for S3; update parsing and add basic test and benchmark Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Fix `TestGrpcStore` to work with new chunk key structure Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Add a function to SchemaConfig that returns the correct PeriodConfig for a given time. Signed-off-by: Callum Styan <callumstyan@gmail.com> Add schema config to object client so it we can use the right external key function based on the schema version. Signed-off-by: Callum Styan <callumstyan@gmail.com> Fix failing compactor tests by passing SchemaConfig to chunkClient Remove empty conditional from SchemaForTime Define schema v12; wire-in ChunkPathShardFactor and ChunkPathPeriod as configurable values Add PeriodConfig test steps for new values ChunkPathShardFactor and ChunkPathPeriod in v12 Update test for chunk.NewExternalKey(); remove completed TODO Set defaults for new ChunkPathPeriod and ChunkPathShardFactor SchemaConfig values; change FSObjectClient to use IdentityEncoder instead of Base64Encoder Use IdentityEncoder everywhere we use FSObjectClient for Chunks Add ExternalKey() function to SchemaConfig; update ObjectClient for Chunks * Finish plumbing through the chunk.ExternalKey -> schemaConfig.ExternalKey change. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Clean up lint failures Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up chunk.go and fix tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Break SchemaConfig.ExternalKey() into smaller functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Quickly fix variable name in SchemaConfig.newerExternalKey() Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up ExternalKey conditional logic; add better comments; add tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix a bug where we are prepending userID to legacy Chunk keys but never parsing it; refactor key tests and benchmarks Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Add small SchemaConfig test Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Update docs and CHANGELOG.md Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Correctly return an error when failing to parse a chunk external key Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Revert IdentityEncoder to Base64Encoder after testing Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix assignment in test for linter Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove leftover comments from development Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Change v12 version comment format; remove redundant login in `ParseExternalKey()` Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Change remaining v12 comment style Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Pass chunk.SchemaConfig to new parallel chunk client * Simplify chunk external key prefixes for schema v12 * Fix broken benchmark; add benchmarks for old parsing conditional Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove unneeded lines from upgrading doc * Add benchmarks for root chunk external key parsing functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * memoizes schema number calculations * Resolve linter issue with PeriodConfig receiver name Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Kaviraj	38c42a26c6	Move `cortex/pkg/tenant` dependency to Loki. (#4944 )	3 years ago

1 2

61 Commits (ccf3e4e5ecd5f237d22c82c7e3623aafd216245a)