165 Commits (c8d2c3200288e60d6ef7d0f0967ecfa76d593734)

Author	SHA1	Message	Date
Travis Patterson	c8d2c32002	feat: Introduce special header that tells Loki not to modify query results (#12327)	2 years ago
Travis Patterson	d175184fe5	chore: remove shards from wrapper signature (#12322)	2 years ago
Travis Patterson	fed7566ba9	fix: Pass query to pipeline wrappers from the AST (#12305)	2 years ago
Ashwanth	3b5ed8c3ec	chore: pass shard count to injected pipelines and extractors (#12213)	2 years ago
Karsten Jeschkies	243e6f813a	refactor: Pass context to usage tracker. (#12272)	2 years ago
Karsten Jeschkies	6578a00481	feat: Support usage trackers for received and discarded bytes. (#11840)	2 years ago
Dylan Guedes	9b7e48aca9	Ingesters: Revert "Add metrics.go log line when reading from an ingester (#11571)" (#11818) ... **What this PR does / why we need it**: This reverts commit f04d0db434 Although it is useful, the amount of logging it generates is too massive. In the future we might revisit it but with a different approach.	2 years ago
Michel Hollands	f04d0db434	Add metrics.go log line when reading from an ingester (#11571) ... **What this PR does / why we need it**: Add a metrics.go log line when a querier reads from an ingester. The output is like this: ``` level=info ts=2024-01-03T13:59:06.919965546Z caller=metrics.go:275 component=ingester org_id=fake traceID=0b216e7d014c5f87 latency=fast query_type=ingester_series start=2024-01-03T13:00:00Z end=2024-01-03T13:59:06.834Z start_delta=59m6.91996413s end_delta=85.964255ms length=59m6.834s duration=110.792µs status=200 query=<omitted> query_hash=2166136261 total_entries=9 ``` **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [X] Tests updated - [X] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0) --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com>	2 years ago
Travis Patterson	f668a1b80b	Add context and tenant to pipeline/extractor wrappers (#11528) ... Adding the context here means the wrapping pipeline can know when the query has stopped or been cancelled.	2 years ago
Karsten Jeschkies	9aae25008d	Optimize series response format by using repeated fileds. (#11498) ... **What this PR does / why we need it**: The Protobuf map type is encodied as a repeated field of map entries. Decoding them to a slice is much faster than decoding them into a map. Since Loki is not using the fast key check for a map we can use the slice decoding. This change also allows us to decode the JSON directly into the right protobuf struct. This doulbes the JSON decoding speed and reduces the memory pressure by ~40%. ``` › go test -bench=. -run=^$ -count=10 ./pkg/querier/queryrange > before.log › go test -bench=. -run=^$ -count=10 ./pkg/querier/queryrange > after.log › benchstat before.log after.log goos: darwin goarch: arm64 pkg: github.com/grafana/loki/pkg/querier/queryrange │ before.log │ after.log │ │ sec/op │ sec/op vs base │ ResponseMerge/mergeStreams_unlimited-10 32.36m ± 0% 32.63m ± 2% ~ (p=0.393 n=10) ResponseMerge/mergeOrderedNonOverlappingStreams_unlimited-10 1.050m ± 1% 1.080m ± 3% +2.84% (p=0.005 n=10) ResponseMerge/mergeStreams_limited-10 33.02m ± 0% 32.60m ± 1% -1.29% (p=0.004 n=10) ResponseMerge/mergeOrderedNonOverlappingStreams_limited-10 15.11m ± 0% 15.07m ± 0% ~ (p=0.075 n=10) _CodecDecodeLogs-10 4.395m ± 1% 4.364m ± 0% -0.72% (p=0.005 n=10) _CodecDecodeSamples-10 16.97m ± 0% 16.84m ± 2% -0.77% (p=0.023 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 745.8µ ± 8% 736.8µ ± 12% ~ (p=0.739 n=10) _CodecDecodeSeries/application/json;_charset=utf-8-10 15.37m ± 1% 10.60m ± 0% -31.03% (p=0.000 n=10) _MergeResponses-10 1186.9m ± 2% 149.8m ± 1% -87.38% (p=0.000 n=10) _UnwrapSeries-10 9.399m ± 1% 4.049m ± 0% -56.92% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 666.0m ± 3% 194.6m ± 0% -70.79% (p=0.000 n=10) geomean 18.87m 12.51m -33.70% │ before.log │ after.log │ │ B/op │ B/op vs base │ _CodecDecodeLogs-10 3.649Mi ± 0% 3.649Mi ± 0% ~ (p=0.364 n=10) _CodecDecodeSamples-10 18.12Mi ± 0% 18.12Mi ± 0% ~ (p=0.926 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 7.647Mi ± 0% 7.647Mi ± 0% ~ (p=0.587 n=10) _CodecDecodeSeries/application/json;_charset=utf-8-10 27.94Mi ± 0% 16.99Mi ± 0% -39.18% (p=0.000 n=10) _MergeResponses-10 2.362Mi ± 0% 2.408Mi ± 0% +1.98% (p=0.000 n=10) _UnwrapSeries-10 19.495Mi ± 0% 8.595Mi ± 0% -55.91% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 772.3Mi ± 0% 772.3Mi ± 0% ~ (p=0.912 n=10) geomean 17.50Mi 14.54Mi -16.91% │ before.log │ after.log │ │ allocs/op │ allocs/op vs base │ _CodecDecodeLogs-10 41.10k ± 0% 41.10k ± 0% ~ (p=1.000 n=10) ¹ _CodecDecodeSamples-10 411.9k ± 0% 411.9k ± 0% ~ (p=1.000 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 32.00 ± 0% 32.00 ± 0% ~ (p=1.000 n=10) ¹ _CodecDecodeSeries/application/json;_charset=utf-8-10 304.2k ± 0% 298.1k ± 0% -2.01% (p=0.000 n=10) _MergeResponses-10 100.1k ± 0% 100.1k ± 0% -0.00% (p=0.002 n=10) _UnwrapSeries-10 201.1k ± 0% 198.0k ± 0% -1.54% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 203.1k ± 0% 203.1k ± 0% ~ (p=0.621 n=10) geomean 48.95k 48.70k -0.51% ¹ all samples are equal ``` **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0)	2 years ago
Travis Patterson	be71a80b15	Implement hooks to instrument query pipelines (#11493) ... This PR implements interfaces to inject new pipelines/extractors into the query path of queriers and ingesters.	2 years ago
Michel Hollands	8628b15624	Use metrics namespace for more metrics (#11025) ... **What this PR does / why we need it**: Use the metrics namespace setting instead of hardcoding to `cortex`. This is a follow up to (and based on) https://github.com/grafana/loki/pull/11014. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. <!-- TODO(salvacorts): Add example PR --> --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com>	2 years ago
Kaviraj Kanagaraj	bbfb13c3e4	Pin `chunk` and `index` format to `schema` version. (#10213) ... We pin all three `Chunk`, `HeadBlock` and `TSDB` Version to `schema` version in period config. This is the following mapping (after being discussed with @owen-d and @sandeepsukhani ) `v12` (current existing schema) - ChunkFormatV3 (UnorderedHeadBlock) + TSDBv2 `v13` (introducing new schema) - ChunkFormatV4 (UnorderedWithNonIndexedLabelsHeadBlockFmt) + TSDBv3 Note the new schema `v13` supports the latest chunk and index format. **NOTES for Reviewer** 1. General approach is we removed the idea of `index.LiveFormat`, `chunkenc.DefaultChunkFormat` and `chunkenc.DefaultHeadBlockFmt` and made following two changes. These variables were used before to tie chunk and tsdb formats specific to Loki versions. This PR remove that coupling and pin these formats to `schema` version instead. 1. These variables were replaced with explicit chunk and index formats within those packages (and it's tests) 2. If these variables were used outside it's own packages say by ingester, compactor, etc. Then we extract correct chunk and index versions from the `schema` config. 2. Add two methods to `periodConfig`. (1) `ChunkFormat()` returning chunk and head format tied to schema (2) `TSDBFormat()` returning tsdb format tied to schema. 2. Other ideas I thought of doing but didn't end up doing is make `ChunkFormat` and `IndexFormat` as separate type (rather than `byte` and `int` currently. Similar to `HeadBlockFmt` type). But didnt' do it eventually to keep the PR small and don't want to complicate with lots of changes. 4. Moved couple of test cases from `chunkenc` to `config` package, because the test case was actually testing methods on `schemaconfig` and it was creating cycling dependencies. --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Bryan Boreham	f89a51fde8	Tracing: elide small traces for Stats call (#10308) ... **What this PR does / why we need it**: At work I see a lot of traces getting data dropped due to being tens of megabytes in size. Looking at what was captured, I see many many repeats of these four spans: <img width="1070" alt="image" src="https://github.com/grafana/loki/assets/8125524/4268e664-6910-49bb-8f68-69b139a5a66b"> Events on the parent trace are cheaper and, for small operations like this, just as useful. Where there was an event line I added a `function` field matching the previous span name. For `storeEntry.Stats()` I didn't add an event line as it doesn't seem to do very much. But please correct me; I'm not hugely familiar with the Loki codebase. **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - NA Documentation added - NA Tests updated - [x] `CHANGELOG.md` updated - NA If the change is worth mentioning in the release notes, add `add-to-release-notes` label - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Owen Diehl	f53498c4aa	allows filter-only matchers in series endpoint (#10269) ... We already allow empty matchers `{}` as a special case in series queries to return all series. This PR also allows specifying only filter-style matchers. This is specifically helpful when a tenant is trying to understand all streams which do _not_ have a corresponding label. ``` $ logcli series '{does_not_exist=""}' --analyze-labels --org-id="fake" 2023/08/16 10:13:32 http://localhost:3100/loki/api/v1/series?end=1692206012889997000&match=%7Bdoes_not_exist%3D%22%22%7D&start=1692202412889997000 Total Streams: 6 Unique Labels: 2 ``` Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Trevor Whitney	4d2a94ebd3	Return intersecting labels when aggregating volume by label (#10248) ... This PR changes the behavior the `index/volume` endpoint to return intersecting labels when requesting volume aggregating by label. Previously it only returned labels that matched selectors in the query. The old behavior is still possible using the `targetLabels` argument.	2 years ago
Charles Korn	bde65667f7	Upgrade dskit to use packages migrated from weaveworks/common (#10164) ... **What this PR does / why we need it**: This PR upgrades dskit and replaces use of packages from weaveworks/common with their migrated equivalents in dskit. See https://github.com/grafana/dskit/pull/342 for more details. Note that Loki still uses some packages from weaveworks/common that I haven't migrated (`aws` and `test`) - I'll migrate these separately. If this PR needs to be rebuilt, I used `rewrite.sh` ([source](https://gist.github.com/charleskorn/48efe62a09d6d70f3de30327003df5c5#file-rewrite-sh)) to generate most of these changes. **Which issue(s) this PR fixes**: (none) **Special notes for your reviewer**: (none) **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	2 years ago
Trevor Whitney	39be4b5e6b	Only aggregate labels or series based on requested (#10076) ... This improves the logic when calculating volumes by only aggregating series or labels based on which was requested. Previously we aggregated both, and then picked the correct set for the response, causing unnecessary additional memory consumption.	2 years ago
Dylan Guedes	122313caf7	Remove redundant spans (#10055) ... **What this PR does / why we need it**: - Stop using `instrument.CollectedRequest` on all Index calls since it emits spans that are redundant with existing ones and aren't improving Loki's monitoring while making it more expensive to run - Add events to `QuerySample` to improve its monitoring so we can better track what is holding down its performance - Remove redundant `Ingester.GetChunkRef` span	2 years ago
Trevor Whitney	a12311c5c8	Allow volume to be aggregated by label (#9988) ... Add a `aggregateBy` query parameter to the `series_volume` endpoint, and rename that endpoint to just `volume`. This allows users to get volumes aggregated into top level labels, rather label+value pairs. --------- Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	3 years ago
Karsten Jeschkies	efac690328	Use series hash to identify uniques in merge. (#9985) ... **What this PR does / why we need it**: A run of the protobuf encoding for the querier showed a spike in memory usage in the `codec.MergeResponse` method. The routine was allocating a lot of memory for the `series.String()` method which is then hashed. Instead, we can use a `uint64` hash. ``` › go test -v -run=^$ -bench "Benchmark_MergeResponses$" -count=10 ./pkg/querier/queryrange ... │ before.txt │ after.txt │ │ sec/op │ sec/op vs base │ _MergeResponses-16 4.563 ± 1% 1.197 ± 1% -73.77% (p=0.000 n=10) │ before.txt │ after.txt │ │ B/op │ B/op vs base │ _MergeResponses-16 6578.715Mi ± 0% 2.362Mi ± 0% -99.96% (p=0.000 n=10) │ before.txt │ after.txt │ │ allocs/op │ allocs/op vs base │ _MergeResponses-16 40500.1k ± 0% 100.1k ± 0% -99.75% (p=0.000 n=10) ``` **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Ed Welch	fa6f9c638a	Loki: add 'post_filter_lines' stat for tracking how many lines are match a queries filter expression(s) (#9983) ... **What this PR does / why we need it**: To better understand query behavior and labeling strategies of users, logging the number of log lines "post filtering" can be very useful, we already log the total_lines processed in a query so this will allow us to see how many lines in the label selector matched what the query was looking for. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Trevor Whitney	9e19ff006c	Add targetLabels to SeriesVolume requests (#9878) ... Adds optional `targetLabels` parameter to `series_volume` and `series_volume_range` requests that controls how volumes are aggregated. When provided, volumes are aggregated into the intersections of the provided `targetLabels` only.	3 years ago
Travis Patterson	8ca035ffbf	Log Volume: Do the prometheus-format conversion in dedicated middleware (#9776) ... This PR moves the Prometheus-format conversion to it's own middleware. This keeps it all in one place and consolidates logic that was in many places. It also sets us up to make range queries as a next step.	3 years ago
Trevor Whitney	dbc3040739	Convert SeriesVolume response to prometheus-style (#9703) ... Changes the response type of the label volume stats endpoint to return volumes as prometheus-style timeseries metrics. It currently only supports instant queries, but is a necessary step to eventually supporting range queries.	3 years ago
Dylan Guedes	c0d9a2b4ba	Log push failures per-entry instead of in batches (#9720) ... **What this PR does / why we need it**: Modify how we log push errors to do it per-entry instead of accumulating all errors and then logging them all in a single pass. **Which issue(s) this PR fixes**: N/A	3 years ago
Travis Patterson	db97058a84	Series volume endpoint (#9704) ... This changes the `label_volume` endpoint to the `series_volume` endpoint. The new endpoint still returns volumes but now it does it for the requested streams defined by the selector names passed rather than individual labels. All relevant non-requested labels are aggregated into the returned results ex: Assume we have the following streams: ``` {cluster="prod", team="A", component="foo"} {cluster="prod", team="B", component="foo"} {cluster="dev", team="A", component="foo"} {cluster="dev", team="B", component="foo"} ``` - requesting `{cluster="prod"}` returns one result for all streams containing `{cluster="prod"}` - requesting `{cluster=~".+"}` returns two results for the streams containing `{cluster="prod"}` and `{cluster="dev"}` - requesting `{cluster=~".+", team=".+"}` returns four results for the streams containing: ``` {cluster="prod", team="A"} {cluster="prod", team="B"} {cluster="dev", team="A"} {cluster="dev", team="B"} ``` --------- Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	3 years ago
Travis Patterson	065bee7e72	Label Volume Endpoint (#9588) ... For a given set of matchers, returns the top N associated label/value pairs by volume. A query for `{cluster=prod}` will return ``` cluster=prod: size (total logs matching this matcher) . . . nth-label=nth-value ``` This is to service use cases where users want to understand where their log volume has come from by label without making multiple requests to the stats endpoint. Note: This PR is a monster but it's mostly plumbing. I've pointed out the most interesting bits that actually get the volumes from ingesters/indexs	3 years ago
Periklis Tsirakidis	69392de8b6	Fix duplicate label values from ingester streams (#9629)	3 years ago
Travis Patterson	6d8a184334	Revert "revert count unflused chunks (#9504)" (#9615) ... Now that the bug where huge values is fixed and known not to be this. Let's readd it. Original: https://github.com/grafana/loki/pull/9479	3 years ago
Salva Corts	87a659a6db	Add span events for index stats and result cache (#9552) ... **What this PR does / why we need it**: This PR adds events to the traces to have some extra observability for how we compute the index stats. We also add some trace events to the results cache.      **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Travis Patterson	a4ef06890b	revert count unflused chunks (#9504) ... revert this commit: aeba51ab0f We seem to be dramatically overcounting bytes now which is causing oversharding This is the same query before and after: Before: ~20 GB  After: ~8 EB 	3 years ago
Travis Patterson	aeba51ab0f	Count unflushed chunks in instance stats (#9479) ... The instance wasn't counting unflushed chunks in the stats. This change checks that the chunk's `flushed time` is 0 which means it hasn't been flushed yet.	3 years ago
Owen Diehl	64c6f6b8cc	Better combines stats in ingesters (#9474) ... * Reduces double counting streams when possible * proportionally add stats based on chunk overlap range * code reuse	3 years ago
Dylan Guedes	358b8965c4	Distributor: Add org_id to log error (#9475) ... **What this PR does / why we need it**: Add `org_id` to push error when the error is "max active streams limit exceeded". Apparently all other push errors correctly report the `org_id`, but not this one. **Which issue(s) this PR fixes**: N/A	3 years ago
Trevor Whitney	50bc1cee71	Rename usagestats to analytics (#9212) ... **What this PR does / why we need it**: Rename usagestats to analytics. The config section in the yaml is already called analytics, this makes things more consistent. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
李国忠	322783e3d8	LogQL: [optimization] syntax: Replace "panic" in "/pkg/logql/syntax" with "error" (#7208)	3 years ago
Pablo	fda14ca156	Re-license and export `pkg/ingester` WAL code to be used in Promtail's WAL (#8315)	3 years ago
Kaviraj Kanagaraj	e18b4fca6f	ingester: Add `ingester_memory_streams_labels_bytes` metric (#8212) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> This is to give visibility on `labels` size of in-memory streams on the ingesters. It caculate the size of labels via `len(labels.String())`. Meaning if you have ```go lbs := labels.Labels{ {Name: "foo", Value: "bar"}, {Name: "instance", Value: "loki"}, } ``` size would be number of bytes in `{foo="bar", instance="loki"}` which is `28`. NOTE: `len(string)` gives [bytes length in Go](https://pkg.go.dev/builtin#len)	3 years ago
Travis Patterson	92fca94bb5	Revert "Update Ingester Rate Calculations (#7652)" (#7991) ... This change was supposed to smooth out the per-second rate of streams so distributors could react more uniformly to streams with variable rates. In practice, it resulted in drastically more resource usage on distributors because `GetStreamRates` now responds with all streams on an ingester rather than only the streams seen in the last second. This change also didn't seem to affect sharding precision one way or another. If we'd still like to implement exponential smoothing for stream rates, it should be implemented in distributors to keep communication overhead to a minimum.	3 years ago
Travis Patterson	343e39f67b	Update Ingester Rate Calculations (#7652) ... - Use weighted moving average for rates - Keep Streams around until they are removed Previously, the rate calculator only kept the most recent second. This led to issues where bursty streams that didn't get pushes every second resulted in undersharding those streams	3 years ago
李国忠	6848ea37dd	fix ingester: grpc method=/logproto.Querier/Query fail. (#5970) ... When ingester's ctx times out, don't try to use grpc to return data to the querier again. This PR fixes the ingester error rate > 0 for our loki cluster. We rely on this error rate indicator for alarming Fixes #5964	3 years ago
Travis Patterson	45d59ba571	Improve rate store refresh latency (#7492) ... Rather than iterating streams and getting rates per stream, keep all necessary information in the rate calculator and just hand back the current rates when asked. This change took p99 latencies from ~1.8s to ~400ms and p50 latences from ~1s to 6ms.	3 years ago
Travis Patterson	3e7adb2c34	Measure Per-Stream rates (#7370) ... This PR implements the ability for ingesters to measure per-stream rates at 1/s resolution. It also fixes bugs in the kind of Limits used in the RateStore and the Ingester API	3 years ago
Dylan Guedes	b1d4efab12	Loki: Per-tenant stream sharding (#7311) ... **What this PR does / why we need it**: - Move stream sharding configuration to its own package to avoid cyclic imports - Change stream sharding to be a per-tenant configuration - Change ingesters to reject whole streams due to rate-limit based on per-tenant stream sharding - Change stream sharding flags prefix from `distributor.shard-stream` to `shard-stream`	3 years ago
Travis Patterson	e3e4e27c8c	Implement ingester API to return stream rates (#7176) ... This PR introduces an API on the Ingester that reports all current streams in memory and their current rate in bytes/second. The reponse from the API contains a slice of `StreamRate` structs: ```go type StreamRate struct { StreamHash uint64 `protobuf:"varint,1,opt,name=streamHash,proto3" json:"streamHash,omitempty"` StreamHashNoShard uint64 `protobuf:"varint,2,opt,name=streamHashNoShard,proto3" json:"streamHashNoShard,omitempty"` Rate int64 `protobuf:"varint,3,opt,name=rate,proto3" json:"rate,omitempty"` } ``` In the interest of space on the caller and in transmission, only hashes and rates are sent. - `StreamHash` will be used to deduplicate replicas - `StreamHashNoShard` will be used to aggregate a sharded stream's total rate. Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	3 years ago
Danny Kopping	c3e92b876c	Restoring lint checks that were previously disabled (#7340) ... I removed these checks in https://github.com/grafana/loki/pull/7337 to not make https://github.com/grafana/loki/pull/7243 overly complex to review. Refactored code to fix valid lint issues, ignored the irrelevant/dangerous ones. I also removed `unused` and `unparam` as these produce low-quality results that are largely useless.	3 years ago
Travis Patterson	071118a090	revert stream push error response (#7160) ... We've changed approaches to how we want to do per-stream rate limiting. The new approach will rely on creating an api on the ingester so the distributors can decide how streams should be sharded. As a result, the work to return rate-limited streams in the error body is no longer needed.	3 years ago
Travis Patterson	bc16450349	Revert "Hide duplicates caused by stream sharding (#7005)" (#7031) ... @owen-d had a suggestion that was dramatically dimpler. Revert this whole thing and the simplified PR will be forthcoming	3 years ago
Travis Patterson	a98b7ebaf4	Hide duplicates caused by stream sharding (#7005) ... As streams are sharded and resharded across different ingesters, the likelihood of duplicate logs being ingested increases dramatically. This removes duplicates at query time by dynamically removing the `__shard_stream__` label for both log and sample queries so the `stream hash` used by the `MergeEntryIterator` will match when doing comparisons for duplicates. The `stream hash` is created in either the `store` or `ingester instance` at query time so the labels are removed there. Additionally, the `MergeEntryIterator` has been modified to remove duplicates in a stream rather than just across streams. For completeness, `__shard_stream__` has also been removed from Label and Series Queries.	3 years ago