apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Sven Grossmann	aa1ac99f4d	fix(`detected_fields`): return parsed labels when parsers are passed (#14047 ) Co-authored-by: Galen Kistler <109082771+gtk-grafana@users.noreply.github.com> Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	1 year ago
Shantanu Alshi	25234e8348	fix(detected_labels): Add matchers to get labels from store" (#14012 )	1 year ago
Trevor Whitney	ab1caea123	feat: add _extracted suffix to detected fields conflicts (#13993 )	1 year ago
Shantanu Alshi	6f99af6222	fix(detected labels): response when store label values are empty (#13970 )	1 year ago
Trevor Whitney	c6ab6b31e8	chore: remove initial metric aggregation experiment (#13729 )	1 year ago
benclive	3a99b69ae5	feat: Add baseline rf1 querier implementation (#13639 ) Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	2 years ago
Cyril Tovena	5f5fd4e4ab	feat(rf1): Store index ref in metastore (#13613 )	2 years ago
Sven Grossmann	ce02cc254a	feat(structured-metadata-api): add structured metadata to `/detected_fields` API (#13604 )	2 years ago
Trevor Whitney	f6a94d3034	fix: detected fields incorrect type bug (#13515 )	2 years ago
Christian Haudum	d8cc1cea83	chore: Make dedicated iterator package (#13273 ) Iterators of various types are widely used throughout the Loki code base. With the recent code additions of the bloom filters, a new set of utility functions for iterators emerged in `github.com/grafana/loki/v3/pkg/storage/bloom/v1`. The package defines interfaces for various common types, but also provides implementations to create and compose new iterators that implement these interfaces. This new package uses Go Generics. - However, at the current state, there are multiple iterator interfaces and implementations, which adds cognitive overhead in understanding how they work and which ones should be used. The idea is to unify them into a single iterator lib for Loki. - As a first step towards a single iterator library for the Loki code base, this PR moves the utilities from the `pkg/storage/bloom/v1` package to the `pkg/iter/v2` package. - Second, it changes the existing `EntryIterator` and `SampleIterator` iterators to "inherit" `v2.CloseIterator` in order to expose the same function names. - And lastly, naming conventions of iterator interfaces and structs are unified. The basic iterator interface (defined in `pkg/iter/v2/interface.go`) looks like so: ```go // Usage: // // for it.Next() { // curr := it.At() // // do something // } // if it.Err() != nil { // // do something // } type Iterator[T any] interface { Next() bool Err() error At() T } ``` --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Dylan Guedes	4eb45cc588	refactor: Introduce context cause to our code (#13224 ) Introduce the usage of Go new `WithCause` functions to make our context cancellation errors more descriptive.	2 years ago
Dylan Guedes	c1fada9af0	refactor: Remove unnecessary spanlogger usage (#13255 ) Historically, we've been using `spanlogger` to enrich our spans. That isn't the right usage for it (as of now), as they'll be emitting log lines. Our current usage is causing: - Noisy/important log lines less visible - Unnecessary logging volume - Spread of the bad practice. With this PR I'm moving away from `spanlogger` all places where the log message is too bad or nonexistent. That's because these cases are indicating we don't care about that log line at all, and only about the data injected in the span.	2 years ago
Trevor Whitney	467eb1bb1b	feat: collect and serve pre-aggregated bytes and counts (#13020 ) Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	2 years ago
Yuri Kotov	808425953f	feat: API: Expose optional label matcher for label names API (#11982 ) Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	2 years ago
Shantanu Alshi	e7fdeb974a	perf: Improve Detected labels API (#12816 )	2 years ago
Shantanu Alshi	3cc28aaf0e	fix: panics when ingester response is nil (#12946 )	2 years ago
jackyin	a46d14fb05	fix: Optimize regular initialization (#12926 )	2 years ago
Sven Grossmann	19fef9355f	feat(detectedFields): Support multiple parsers to be returned for a single field (#12899 )	2 years ago
Sven Grossmann	2b3ae48d9b	feat(detectedFields): add parser to response (#12872 )	2 years ago
Shantanu Alshi	587a6d20e9	feat: Detected labels from store (#12441 )	2 years ago
Trevor Whitney	f68d1f7faf	fix: make detected fields work for both json and proto (#12682 )	2 years ago
Trevor Whitney	6c33809015	feat: split detected fields queries (#12491 )	2 years ago
Christian Haudum	2f24b670cb	chore: Restructure `indexgateway` component package (#12630 ) This PR moves code for the index gateway server and client components into the `pkg/indexgateway` directory, so it that the index gateway has its own top-level package like other components, such as ingester, distributor, querier, ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Cyril Tovena	491d2513e7	feat: add log patterns ingestion and query API. (#12403 ) Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Anton Kolesnikov <anton.e.kolesnikov@gmail.com> Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Shantanu Alshi	2c878c8301	fix: Bugfixes for detected labels API (#12466 )	2 years ago
Shantanu Alshi	643f353cef	feat: Detected labels - get and filter labels from ingesters (#12432 )	2 years ago
Trevor Whitney	a53a0cc792	feat: return real data in detected fields endpoint (#12421 ) parse log lines in the querier to return data about detected fields. this is not sustainable long term, but a short term solution to validate the data we want for the frontend.	2 years ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Shantanu Alshi	a331746c2f	feat(detected_labels): Add pod as a detected label to demo (#12422 )	2 years ago
Shantanu Alshi	5190dda64b	feat(detected_labels): Initial skeleton for the API (#12390 ) Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	2 years ago
Trevor Whitney	5b4bfa54b6	feat(detected_fields): initial plumbing of new endpoint (#12385 )	2 years ago
Owen Diehl	a36483b731	feat(blooms): bloom integration in query planning (#12208 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Karsten Jeschkies	9aae25008d	Optimize series response format by using repeated fileds. (#11498 ) What this PR does / why we need it: The Protobuf map type is encodied as a repeated field of map entries. Decoding them to a slice is much faster than decoding them into a map. Since Loki is not using the fast key check for a map we can use the slice decoding. This change also allows us to decode the JSON directly into the right protobuf struct. This doulbes the JSON decoding speed and reduces the memory pressure by ~40%. ``` › go test -bench=. -run=^$ -count=10 ./pkg/querier/queryrange > before.log › go test -bench=. -run=^$ -count=10 ./pkg/querier/queryrange > after.log › benchstat before.log after.log goos: darwin goarch: arm64 pkg: github.com/grafana/loki/pkg/querier/queryrange │ before.log │ after.log │ │ sec/op │ sec/op vs base │ ResponseMerge/mergeStreams_unlimited-10 32.36m ± 0% 32.63m ± 2% ~ (p=0.393 n=10) ResponseMerge/mergeOrderedNonOverlappingStreams_unlimited-10 1.050m ± 1% 1.080m ± 3% +2.84% (p=0.005 n=10) ResponseMerge/mergeStreams_limited-10 33.02m ± 0% 32.60m ± 1% -1.29% (p=0.004 n=10) ResponseMerge/mergeOrderedNonOverlappingStreams_limited-10 15.11m ± 0% 15.07m ± 0% ~ (p=0.075 n=10) _CodecDecodeLogs-10 4.395m ± 1% 4.364m ± 0% -0.72% (p=0.005 n=10) _CodecDecodeSamples-10 16.97m ± 0% 16.84m ± 2% -0.77% (p=0.023 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 745.8µ ± 8% 736.8µ ± 12% ~ (p=0.739 n=10) _CodecDecodeSeries/application/json;_charset=utf-8-10 15.37m ± 1% 10.60m ± 0% -31.03% (p=0.000 n=10) _MergeResponses-10 1186.9m ± 2% 149.8m ± 1% -87.38% (p=0.000 n=10) _UnwrapSeries-10 9.399m ± 1% 4.049m ± 0% -56.92% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 666.0m ± 3% 194.6m ± 0% -70.79% (p=0.000 n=10) geomean 18.87m 12.51m -33.70% │ before.log │ after.log │ │ B/op │ B/op vs base │ _CodecDecodeLogs-10 3.649Mi ± 0% 3.649Mi ± 0% ~ (p=0.364 n=10) _CodecDecodeSamples-10 18.12Mi ± 0% 18.12Mi ± 0% ~ (p=0.926 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 7.647Mi ± 0% 7.647Mi ± 0% ~ (p=0.587 n=10) _CodecDecodeSeries/application/json;_charset=utf-8-10 27.94Mi ± 0% 16.99Mi ± 0% -39.18% (p=0.000 n=10) _MergeResponses-10 2.362Mi ± 0% 2.408Mi ± 0% +1.98% (p=0.000 n=10) _UnwrapSeries-10 19.495Mi ± 0% 8.595Mi ± 0% -55.91% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 772.3Mi ± 0% 772.3Mi ± 0% ~ (p=0.912 n=10) geomean 17.50Mi 14.54Mi -16.91% │ before.log │ after.log │ │ allocs/op │ allocs/op vs base │ _CodecDecodeLogs-10 41.10k ± 0% 41.10k ± 0% ~ (p=1.000 n=10) ¹ _CodecDecodeSamples-10 411.9k ± 0% 411.9k ± 0% ~ (p=1.000 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 32.00 ± 0% 32.00 ± 0% ~ (p=1.000 n=10) ¹ _CodecDecodeSeries/application/json;_charset=utf-8-10 304.2k ± 0% 298.1k ± 0% -2.01% (p=0.000 n=10) _MergeResponses-10 100.1k ± 0% 100.1k ± 0% -0.00% (p=0.002 n=10) _UnwrapSeries-10 201.1k ± 0% 198.0k ± 0% -1.54% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 203.1k ± 0% 203.1k ± 0% ~ (p=0.621 n=10) geomean 48.95k 48.70k -0.51% ¹ all samples are equal ``` Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Karsten Jeschkies	f67fff3eb2	Approximate `quantile_over_time` (#10417 ) What this PR does / why we need it: This change shards `quantile_over_time` queries using t-digest or DDSketch approximations. It can be enabled with `querier.shard_aggregations=quantile_over_time`. Outstanding - [x] Replace generic return type of `StepEvaluator` with interface `StepResult`. - [x] Send mapped query with quantile sketch expression from frontend to querier over the wire. - [x] Serialize sketches. See https://github.com/influxdata/tdigest/issues/34 - [x] Add feature flag. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Shantanu Alshi	fb6f6d2e2f	Stop using global logger in modules (#11051 ) What this PR does / why we need it: Refactors Loki to inject logger from the constructors instead of using global logger variable. It is generally considered a good practice. Which issue(s) this PR fixes: [Fixes #<issue number>](https://github.com/grafana/loki-private/issues/810) Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. <!-- TODO(salvacorts): Add example PR -->	2 years ago
Salva Corts	9be3c0863e	Support categorized labels in Tailing (#11079 ) What this PR does / why we need it: This is a follow-up PR for https://github.com/grafana/loki/pull/10419 adding support for tailing. I tested it on a dev cell and works fine. <img width="1296" alt="image" src="https://github.com/grafana/loki/assets/8354290/6177e0ca-02ce-48cd-a17f-0739dc3caa0a"> Note: With these changes, the JSON marshal unmarshal functions for the tail are no longer used ([example][1]) so I think we can remove them. Also, the new Tail response is no longer used, so we can also make it an alias to the _legacy_ one. Let's do it on a follow-up PR to avoid making this one bigger. [1]: `52a3f16039/pkg/loghttp/entry.go (L210-L238)`	2 years ago
Karsten Jeschkies	6948c4a179	Turn frontend Tripperware into a Middleware. (#10688 ) What this PR does / why we need it: Currently, a request to Loki's frontend API goes through these conversions: ``` http.Request ↓ limitedRoundTripper queryrangebase.Request ↓ queryrangebase.Middlware … ↓ queryrangebase.Request ↓ limitedRoundTripper http.Request ↓ grpcRoundTripperAdapter httpgrpc ↓ grpcRoundTripperAdapter http.Response ↓ limitedRoundTripper queryrangebase.Response ↓ limitedRoundtripper http.Response ``` Since `httgrpc` and `queryrangebase.Request` are Protobufs there's no good reason to encode and decode them to HTTP responses/requests. Furthermore, the encoding to HTTP makes it harder for us to encode query plans. Thus the conversions are changed to the following: ``` http.Request ↓ queryrangebase.Request ↓ queryrangebase.Middlware … ↓ queryrangebase.Request ↓ httpgrpc ↓ queryrangebase.Response ↓ http.Response ``` In order to achieve this the `http.RoundTripper` is pushed to the outside. Only the serialization layer from `http.Request` to `queryrangebase.Request` and `http.Response` to `queryrangebase.Response` will be an `http.RoundTripper`. Everything else is either a `queryrangebase.Handler` or `queryrangebase.Middleware`. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Ashwanth	0be191309c	config: better defaults for querier max concurrency (#10785 ) What this PR does / why we need it: To simply loki configuration, this pr removes `querier.worker-parallelism` as it does not offer additional value to already existing `querier.max-concurrent` Also updates the default value of `querier.max-concurrent` to 4. This is a better default for users that are just starting out. Users can consider increasing it as they scale up their loki installation. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [X] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [X] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Ashwanth	91cae03fe1	chore: restructure compactor package (#10775 ) What this PR does / why we need it: This pr moves the compactor pkg from `pkg/storage/stores/shipper/indexshipper/compactor` to `pkg/compactor` - same level as other high level components (querier, distributor...) table compaction code however resides in the respective store pkg closer to other store specific code. `pkg/storage/stores/shipper/indexshipper/boltdb/compactor/table_compactor.go` `pkg/storage/stores/shipper/indexshipper/tsdb/compactor.go` Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Christian Haudum	2e775f3f44	chore(storage): Restructure shipper packages (#10729 ) What this PR does / why we need it: This PR restructures the code for the shipper component of the Loki storage layer. New package layout: ```console $ tree -d pkg/storage/stores/shipper pkg/storage/stores/shipper └── indexshipper ├── boltdb │ └── compactor ├── compactor │ ├── client │ │ └── grpc │ ├── deletion │ ├── deletionmode │ ├── generationnumber │ └── retention ├── downloads ├── gatewayclient ├── index ├── indexgateway ├── storage ├── testutil ├── tsdb │ ├── index │ ├── testdata │ └── testutil ├── uploads └── util 23 directories ``` * TSDB and BoltDB specific code is under `./pkg/storage/stores/shipper/indexshipper/tsdb` and `./pkg/storage/stores/shipper/indexshipper/boltdb` respectively. * Common code for both TSDB and BoltDB is directly under `./pkg/storage/stores/shipper/indexshipper` and subdirectories, such as `uploads/`, `downloads/`, `compactor/`, ... Special notes for your reviewer: This PR is identical to https://github.com/grafana/loki/pull/10724 except of the package `pkg/storage/stores/indexshipper` the package is `pkg/storage/stores/shipper/indexshipper` (one level deeper). Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	2b6bdc2a29	chore(storage): Simplify store interfaces and abstractions (pt 2) (#10454 ) What this PR does / why we need it: * Remove preemptive interface from indexReaderWriter and make it an exported struct * Compose IngesterStore and QuerierStore The ingester and querier do not require a store that implements the full interface, but can accept stores that are a subset of the interfaces. Therefore we can make specific interfaces that are accepted by the ingester and querier constructors. * Split up ChunkFetcher interface This commit splits the ChunkFetcher interface into a ChunkFetcher and a ChunkFetcherProvider interface. This has the advantage that methods can allow accepting smaller interfaces. This commit also renames the `GetChunkRefs()` method of the ChunkFetcher interface into `GetChunks()`, because it returns a `[][]chunk.Chunk`, not a `[]chunk.ChunkRef` type. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	5652a7baf4	chore(storage): Simplify store interfaces and abstractions (pt 1) (#10435 ) In an attempt to simplify the store abstractions/interfaces, this PR does the following cleanups: * Remove duplicate store field in `pkg/storage.store` Remove duplicate index.Filterable interface inheritance * Remove nesting from CompositeStore * Rename/move ChunkReader to ./pkg/storage.SelectStore --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Ashwanth	470cbbf118	querier: remove query_timeout and engine timeout from querier conf (#10302 ) What this PR does / why we need it: Removes `query_timeout` and `engine:timeout` from querier's config section Both these configs have been deprecated in 2.7.0 release in favor of `query_timeout` in `limits_config` which also allows applying per-tenant overrides at runtime. CLI flag `-querier.engine.timeout` is also removed Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [X] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Owen Diehl	f53498c4aa	allows filter-only matchers in series endpoint (#10269 ) We already allow empty matchers `{}` as a special case in series queries to return all series. This PR also allows specifying only filter-style matchers. This is specifically helpful when a tenant is trying to understand all streams which do _not_ have a corresponding label. ``` $ logcli series '{does_not_exist=""}' --analyze-labels --org-id="fake" 2023/08/16 10:13:32 http://localhost:3100/loki/api/v1/series?end=1692206012889997000&match=%7Bdoes_not_exist%3D%22%22%7D&start=1692202412889997000 Total Streams: 6 Unique Labels: 2 ``` Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Charles Korn	bde65667f7	Upgrade dskit to use packages migrated from weaveworks/common (#10164 ) What this PR does / why we need it: This PR upgrades dskit and replaces use of packages from weaveworks/common with their migrated equivalents in dskit. See https://github.com/grafana/dskit/pull/342 for more details. Note that Loki still uses some packages from weaveworks/common that I haven't migrated (`aws` and `test`) - I'll migrate these separately. If this PR needs to be rebuilt, I used `rewrite.sh` ([source](https://gist.github.com/charleskorn/48efe62a09d6d70f3de30327003df5c5#file-rewrite-sh)) to generate most of these changes. Which issue(s) this PR fixes: (none) Special notes for your reviewer: (none) Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	3 years ago
Trevor Whitney	a12311c5c8	Allow volume to be aggregated by label (#9988 ) Add a `aggregateBy` query parameter to the `series_volume` endpoint, and rename that endpoint to just `volume`. This allows users to get volumes aggregated into top level labels, rather label+value pairs. --------- Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	3 years ago
Trevor Whitney	9e19ff006c	Add targetLabels to SeriesVolume requests (#9878 ) Adds optional `targetLabels` parameter to `series_volume` and `series_volume_range` requests that controls how volumes are aggregated. When provided, volumes are aggregated into the intersections of the provided `targetLabels` only.	3 years ago
Trevor Whitney	5a570cb921	Query ingesters for SeriesVolume in single binary (#9888 ) This change is necessary to get recent SeriesVolume data from the ingesters when running without an `AsyncStore` in single binary. It changes the queriers to query both the ingester and the store, rather than relying on the `AsyncStore` to query the ingesters.	3 years ago
Travis Patterson	db97058a84	Series volume endpoint (#9704 ) This changes the `label_volume` endpoint to the `series_volume` endpoint. The new endpoint still returns volumes but now it does it for the requested streams defined by the selector names passed rather than individual labels. All relevant non-requested labels are aggregated into the returned results ex: Assume we have the following streams: ``` {cluster="prod", team="A", component="foo"} {cluster="prod", team="B", component="foo"} {cluster="dev", team="A", component="foo"} {cluster="dev", team="B", component="foo"} ``` - requesting `{cluster="prod"}` returns one result for all streams containing `{cluster="prod"}` - requesting `{cluster=~".+"}` returns two results for the streams containing `{cluster="prod"}` and `{cluster="dev"}` - requesting `{cluster=~".+", team=".+"}` returns four results for the streams containing: ``` {cluster="prod", team="A"} {cluster="prod", team="B"} {cluster="dev", team="A"} {cluster="dev", team="B"} ``` --------- Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	3 years ago
Travis Patterson	065bee7e72	Label Volume Endpoint (#9588 ) For a given set of matchers, returns the top N associated label/value pairs by volume. A query for `{cluster=prod}` will return ``` cluster=prod: size (total logs matching this matcher) . . . nth-label=nth-value ``` This is to service use cases where users want to understand where their log volume has come from by label without making multiple requests to the stats endpoint. Note: This PR is a monster but it's mostly plumbing. I've pointed out the most interesting bits that actually get the volumes from ingesters/indexs	3 years ago

1 2 3 4

160 Commits (bbbd82bc73322d662ba81efeda3884efcdc09708)