apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Christian Haudum	60551dacc4	Remove unused sharding code from bloom gateway (#11906 ) The sharding by fingerprint range is done on the client side, not on the server side. The sharding related code was not used in the gateway. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Periklis Tsirakidis	5914df77b9	operator: Add support for running with Google Workload Identity (#11869 ) Co-authored-by: Robert Jacob <rojacob@redhat.com>	1 year ago
Owen Diehl	52167c4d32	logs query & hashedquery separately to minimize stacktraces (#11901 ) Rather than call all our values together, this PR precomputes a few in `metrics.go` so any issues will result in smaller stack traces.	1 year ago
Owen Diehl	fe53eb6817	fix type errors after introduction of typedcache into bloomshipper (#11902 ) Recent changes causing compilation failures -- this fixes type signatures after refactoring. Also removes test coverage comparisons because they prevent us from fixing `main`	1 year ago
Owen Diehl	3a1bd41ef5	[Blooms] Routing & wiring for bloom building (#11894 ) A few changes wiring up bloom building logic: * A `Router` as the controlling mechanism for iterating tenants+periods, queueing work for parallel computation, determining which fingerprint ranges and tenants are owned by a replica, and waiting for it's completion * Updates the `SimplerBloomController` to accept ownership ranges, periods, and tenants as arguments for use by the `Router` * Builds a `BloomTSDBStore` struct around the `indexshipper/storage.Client` interface in order to iterate relevant TSDBs/etc	1 year ago
Christian Haudum	3b122e4be1	Bloom gateway: Wire up cache config with implementation (#11898 ) Until now, the cache configuration in the bloom gateway did not initialise the cache. This PR wires the config with the actual implementation and passes them to the bloom store. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	9128036145	Bloom gateway: Integrate processor into worker (#11895 ) This PR replaces the iteration of the bloom gateway worker with the code encapsulated in the previously established processor. Since the processor only relies on the BloomStore, rather than on the BloomShipper, the PR also removes unused code from the shipper. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Karsten Jeschkies	483ee56139	Inject downstream accumulator. (#11863 ) What this PR does / why we need it: We inject the downstream accumulator into the downstreamer instead of constructing it on the first arrival of the first result. Since we know the query type before executing the first query we can pass the correct accumulator. This will allow us to define special `topk` or `sum by` accumulators in the future. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Derek Cadzow	b26bd6d77b	changed first person to comply with style guide (#11890 ) What this PR does / why we need it: changed first person to comply with style guide Which issue(s) this PR fixes: Fixes #<issue number> No issue Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Owen Diehl	ba0538c2a5	[Blooms] tsdb iterator implementation for bloom construction (#11886 ) Builds a lazy iterator over a tsdb file for bloom construction. This is just the library utility, not the integration.	1 year ago
J Stickler	46c6118eb2	Update docs to support OTEL GA (#11858 ) What this PR does / why we need it: Updating the docs in advance of OTEL release to remove "experimental" notes. Also restructured release notes so that most recent content is first.	1 year ago
Christian Haudum	c4ac168a8c	BloomStore: Change signature of `FetchBlocks()` to return `[]CloseableBlockQuerier` (#11884 ) What this PR does / why we need it: In order to encapsulate the reference counting on the `BlockDirectory`, the `BloomStore` now returns a slice of `CloseableBlockQuerier` instead of the directory itself. The caller is responsible for closing the returned querier in order to release the reader resource and decrement the ref counter. The PR also renames `ClosableBlockQuerier` to `CloseableBlockQuerier`. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	7bbbf232d0	Extend BloomStore with `Client(model.Time)` function (#11881 ) What this PR does / why we need it: For certain operations, we want direct access to the `BloomClient`. Since every schema period has its own client, the `Client()` function accepts a `model.Timestamp` for which the client should be returned. The function may return an error, if no client for the given time could be found. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	69d152b618	(chore) BloomStore: Clean up `FetchBlocks()` (#11876 ) What this PR does / why we need it: * Removes `FetchBlocksWithQueue` and replace it with `FetchBlocks` * Operate on single `BlockRef` when loading a block directory from cache/fs/storage. * Ensure order of responses from the FetchBlocks request --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	af66ecec6a	[Blooms] bloomshipper meta integration (#11874 ) Builds on top of https://github.com/grafana/loki/pull/11872, standardizing on the `bloomshipper` types `Meta` and `MetaRef`.	1 year ago
Robert Jacob	6b46e373fb	operator: Integrate support for OpenShift-managed credentials in Azure (#11868 )	1 year ago
Christian Haudum	1623609263	Use correct header name in docs for query fairness in scheduler (#11875 )	1 year ago
Owen Diehl	3a92834de0	[Blooms] Stateless archiving (#11872 ) Moves the archival code to an interface based approach, removing the explicit fs dependency. We can now implement with in-memory solutions. Also refactors to use a tar-file iterator and marks tests as parallelizable.	1 year ago
Christian Haudum	73edf7a943	(chore) Bloomshipper: Separate store and client (#11865 ) What this PR does / why we need it: This PR removes the `StoreAndClient` interface that was accepted by the `BloomShipper`. Since the `BloomStore` had to not only implement the `Store` interface, but also the `Client` interface, it caused re-implementation of the same methods in different ways. Now the shipper solely relies on the `Store` interface. See individual commit messages for more context. Tests have been rewritten from scratch and placed in their own respective test files for store and client. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Callum Styan	2e3fa3b861	quantile sharding: switch to ddsketch sparse constructor and remove quantile vector pooling in Join function (#11844 ) This should reduce our memory usage for our quantile sketches by about half, at least in our current situation we'll OOM after ~160-170 minutes instead of ~90 😢 --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>	1 year ago
Christian Haudum	f97fb093f3	(chore) Bloom store: Rewrite block downloading/caching in bloom store fetcher (#11857 ) What this PR does / why we need it: 1. The `BloomStore` interface gets a new function `FetchBlocks()`, which accepts `[]BlockRef` and returns `[]BlockDirectory`. The fetcher implements a new function `FetchBlocks()` which returns a list of block directories. A block directory represents a local file path that contains the extracted files of a bloom block. It also holds a counter of active readers that access the directory. This is used for safely deleting the directory in case it needs to be removed from disk (eg. max disk/cache size reached). The fetcher resolves the block directory from three places: 1. Cache: The in-memory cache that holds actively accessed directories and keeps track of used disk size. 2. Filesystem: In case the cache was emptied (e.g. when restarting the process) but the block directory is present on disk, it can be re-assambled into a bloom directory that can be put to cache. 3. Storage: If the directory is not present locally, the block archive is downloaded and extracted and the block directory is put to cache. 2. The `{Meta,Block}Client` interfaces are unified: Both clients define the same set of operations: Get{Meta,Block} Get{Meta,Block}s Put{Meta,Block} Delete{Meta,Block}s 3. The `blockDownloader` is replaced by a simple queue powered by a channel. The queue is implemented using generics, so it can be re-used for different request/response types. 4. Code related to "block caching" moved into a separate file `cache.go` and unused code is removed. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Periklis Tsirakidis	24c760a75e	operator: Allow custom audience for managed-auth on STS (#11854 )	1 year ago
Dylan Guedes	9b7e48aca9	Ingesters: Revert "Add metrics.go log line when reading from an ingester (#11571 )" (#11818 ) What this PR does / why we need it: This reverts commit `f04d0db434` Although it is useful, the amount of logging it generates is too massive. In the future we might revisit it but with a different approach.	1 year ago
Salva Corts	0358b6d968	Correctly check chunks in bloom block querier. (#11862 ) What this PR does / why we need it: This PR fixes the block querier to correctly test blooms for chunks. Before this PR, for each chunk we were checking the same set of search terms against the bloom. Therefore, we were not filtering chunks, but only series. This PR uses the chunk ref as a prefix for the search term. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Karsten Jeschkies	b1a0272357	Run GitHub action checks on pull requests against main. (#11864 ) What this PR does / why we need it: The GitHub action `checks` should run on each pull request. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Karsten Jeschkies	486d14572d	Annotate code with lint issues (#11823 ) What this PR does / why we need it: This should make it easier to find and fix lint issues. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Ashwanth	69919a303b	ruler: add filtering support for prometheus rules api (#11817 ) What this PR does / why we need it: Adds `rule_name`, `rule_group`, `file` and `type` query parameters for filtering the response of `/prometheus/api/v1/rules` endpoint. Replicates mimir's functionality: https://github.com/grafana/mimir/pull/5291 - all of them are optional. - `type` paremeter accepts either `alert` or `record` - `rule_name`, `rule_group`, `file` can accept multiple values and they filter the response accordingly. There is a minor change in behavior: `/prometheus/api/v1/rules` endpoint will no longer return empty rule groups which is inline with both [prometheus](https://github.com/prometheus/prometheus/pull/12270/files#diff-315f251cdd7e93fcec1e7e9505744da1d1828f30d2b61d1f4ce963fa26bf1909R1403) and [mimir](https://github.com/grafana/mimir/pull/5291/files#diff-e5424c21c0e827bd1c9d3f669ed605897696bdc27993bc8bfd7113eba787b49dR1120). This is not a breaking change since rule groups with [no rules fail validation](`27fbd62505/pkg/ruler/base/manager.go (L295)`) and cannot be created. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki/issues/9295 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Sandeep Sukhani	c350641a64	otlp: add docs for per tenant otlp config (#11849 ) What this PR does / why we need it: In PR #11143 we added support for per tenant otlp config. This PR adds the relevant documentation to explain how the config looks and works. Checklist - [x] Documentation added --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	1 year ago
Owen Diehl	0f34d9155e	Blooms/block metadata (#11859 ) A few updates to the bloom library: * Uses `FingerprintBounds` in series headers * Encodes `BlockOptions` in the series file so we can later read the target page & block sizes the block was generated with in addition to the schema. * Introduces `BlockMetadata` struct and loads it correctly from blocks. This struct will be used to convert to the `BlockRef`s from the `bloomshipper` pkg and used in the bloom compactor + bloom gateway * Integrates checksums better into block building and XORs the headers metadata from each file (blooms, series) together to generate a final checksum for the block (a combination of both files).	1 year ago
Nikki Ashton	5132f6d2a7	Remove trailing tab character in helm templates (#11805 ) What this PR does / why we need it: Remove trailing tab character in helm templates Which issue(s) this PR fixes: Fixes #11385 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Owen Diehl	d413c28fa3	Blooms: Align compactor to shipper (#11855 ) Does some alignment work between the `bloomcompactor` and the `bloomshipper` pkgs. Notably: * Uses `bloomshipper.BlockRef` everywhere (removes old bloomshipper struct * Integrates `v1.FingerprintBounds` in `Ref` struct * `Location` interface to distinguish local paths vs paths in object storage for certain types (`{Meta,Bloom}Ref`s) * Introduces `KeyResolver` interface to generate locations from these structs * Integrates `KeyResolver` into our bloom store. In the future, this will allow us to change key structures across schema boundaries when we want to change\|improve them. * Removes `BlockPath` from `BlockRef` in favor of the new resolving functionality. This is also beneficial because it lets us _calculate_ locations from the pure Ref objects, rather than tie some arbitrary state to them which can change (or not be populated by accident).	1 year ago
Robert Jacob	009c53a75e	operator: Add support for running with Azure Workload Identity (#11802 )	1 year ago
Owen Diehl	098eef7bd4	Blooms/bounds integration (#11848 ) Replaces `Ref`'s min/max fingerprint fields with `v1.FingerprintBounds` and improves a bunch of checks that use it. Builds on top of https://github.com/grafana/loki/pull/11847	1 year ago
Salva Corts	c335cd200e	Extract shard annotation into new FingerprintFilter interface (#11834 ) What this PR does / why we need it: This PR extracts the `Match` and `Bounds` methods of TSDB's `ShardAnnotation` into a new interface `FingerprintFilter`. This will allow us to query the index for any fingerprint bounds, not just power of 2 shard factors. We now use this in the bloom compactor by calling the index `ForSeries` ([here][1]) passing a fingerprint filter with the bounds of the FP range owned by the compactor. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) [1]: `de4f56e42d/pkg/bloomcompactor/bloomcompactor.go (L408)`	1 year ago
Christian Haudum	1191f88190	(chore) Bloom shipper: Extend `Interval` struct with utility functions (#11841 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	928e3f34f8	Blooms: Prepare compactor refactor (#11847 ) Prepares `bloomcompactor` pkg for integration with new logic, primarily removes/thins out existing code.	1 year ago
Steven Dungan	8289ca7188	Structured metadata docs update (#11843 ) * Rewrote description to highlight primary use case * Added defaults for how much structured metadata can be added to log lines * Rewrote example queries What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Christian Haudum	4411649a0e	(chore) Bloom shipper: Replace `Keyspace` struct with `v1.FingerprintBounds` (#11839 ) The latter struct has more utility functions to compare and operate on bounds. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	509f6f6b66	Bloom/controller wiring (#11831 ) Functionality & testware for building bloom block plans.	1 year ago
Meng Ye	45e5b427b1	docs: fix row_shards doc (#11795 ) for v10 or greater, default is 16 refer to `1002ba00df/pkg/storage/config/schema_config.go (L336)` What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	1 year ago
Sandeep Sukhani	7a04da852a	otlp: fix otlp service.name check when no resource attributes are defined (#11837 ) What this PR does / why we need it: When no resource attributes are defined, check for `service.name` resource attributes fail with `nil pointer dereference`. This PR fixes the issue by checking the bool returned by the `Get` method on resource attributes. Checklist - [x] Tests updated	1 year ago
Christian Haudum	6902130c9a	Bloom shipper: Restructure bloom store (#11828 ) What this PR does / why we need it: This PR changes the structure of how the bloom client * The `BloomStore` is the top component interface which implements the `Store` and the `Client` interfaces. The store holds a store entry for each schema period. * The `bloomStoreEntry` implements the `Store` and `Client` interfaces. It holds a bloom client for a single schema period. Additionally, the store entry also exposes a fetcher for metas (and in the future for blocks), which is responsible for getting data from cache or storage (if not available in cache). * The `BloomClient` implement the `Client` interface. The bloom client uses an object client for the schema period of the bloom client. * The `Fetcher` can fetch and cache metas. This structure is very similar to what we use for the chunk store. Note Before implementing `FetchBlocks()` in the `BloomStore`, I want to implement the new `FingerprintBounds` type also in the shipper and bloom gateway code to be consistent across components, as well as to make use of the new utilities exposed by them. The store implementation probably needs some specific test cases around using the correct store entry for different schema period configs. The code however is mostly taken from the chunk store implementation. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Yogev Levy	5675baecbe	Adding annotations support for statefulset-read.yaml (#11690 ) Adding fixing lake of annotations support for `read` StatefulSet What this PR does / why we need it: Currently there is no use for `read.annotations` at the `statefulset-read.yaml` file. The key exists on `values.yaml` file, with no use. Which issue(s) this PR fixes: Fixes #11688 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	1 year ago
Paul Rogers	d4e359dea8	Change distributor latency to have a regular expression equals (#11827 ) Currently, the distributor latency panel uses a "=" in the code as opposed to a "=~", causing the panel to render as empty. This is to fix that issue. What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Dylan Guedes	3b7ac359bc	Spans: Stop emitting `GetChunks` and `IndexClient.GetChunkRefs` (#11825 ) What this PR does / why we need it: Modify our code to not emit the `GetChunks` and the `IndexClient.GetChunkRefs` spans. We don't worse our observability by getting rid of them because their data is redundant with parent spans. Example: - If an specific index-gateway or ingester struggles, the parent span `/logproto.Querier/GetChunkIDs` will report it just fine - If an specific query causes trouble the `query.Exec` span will show the query name, the `start` and `end`, etc. Also, its children will show the slower parts of the query (in case we have a weak link) Which issue(s) this PR fixes: N/A	1 year ago
Robert Jacob	38d205b967	operator: Improve messages for errors in storage secret (#11824 )	1 year ago
Meng Ye	9d4d101169	docs: fix runtime_config example (#11798 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	1 year ago
Owen Diehl	a56b56c8b0	[Bloom] Keyspace aware bloom creation, library utilities (#11810 ) This PR continues work refactoring the bloom compactor pkg and adds a bunch of utilities for keyspace manipulation/mgmt. A lot of the bloom block logic is contained here as a skeleton. This code is not complete, but is fine to merge as we don't use this yet and should be easier to review before it gets too large.	1 year ago
Vladyslav Diachenko	821c0b1ae0	DequeueMany fix (#11797 ) What this PR does / why we need it: Previously we relied only on the index of the tenant's queue to read the requests from the same tenant's queue. However, as long as the queue is aggressively used by the consumers in parallel, there are a few edge cases when knowing the index of last used tenant's queue is not enough to guarantee that we dequeue items from exactly the same tenant's queue. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	1 year ago
Christian Haudum	ecab33a932	Bloom Gateway: Extract processing logic from bloom worker into separate struct (#11812 ) What this PR does / why we need it: The processor executes a set of tasks. It cleanly separates I/O from logic and therefore is more testable than the current worker implementation. `9602214abf/pkg/bloomgateway/processor.go (L33-L36)` Note: The processor is not used yet in the worker. Certain parts must be refactored when #11810 is merged. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago

1 2 3 4 5 ...

6226 Commits (b7cb85f92b7f903a854bf0f84b9c80c2efaa4334) All Branches Search

6226 Commits (b7cb85f92b7f903a854bf0f84b9c80c2efaa4334)

All Branches