apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Owen Diehl	e2a63e582c	adds tsdb-max-bytes-per-shard limit (#9811 ) Adds the per tenant limit `tsdb-max-bytes-per-shard` which is used in configuring the shard size for tsdb subqueries. This effectively gives control over how big subqueries should be (assuming they're shardable). The default is no different (`600MB`) than the previously hardcoded initial value. This should help us iterate to find optimal shard sizes to improve operations in the long term.	2 years ago
Vladyslav Diachenko	00cf3133e3	disabled indexGateway client for boltdbShipper in migration tool. (#9809 ) What this PR does / why we need it: Without this fix, the migration tool tries to initialize the ingexGateway client, which is unreachable. This leads to the error: `Failed to create source store: error creating index client: index gateway grpc dial: failed to build resolver: passthrough: received empty target in Build()`	2 years ago
Christian Haudum	a65c99d9bf	Add per-tenant request counter metric to index gateway server (#9797 ) This commit add a counter metric `loki_index_gateway_requests_total` with labels `operation`, `tenant`, `status` for gRPC requests that are served by the index gateway. What for? The per-tenant RPS on the index gateway is used to derive the per-tenant shard factor. Why tracking on the server? Unlike tracking index gateway RPS on the client side, tracking on the server side does not yield that many series, even in multi-tenant installations with a lot of tenants, because the amount of index gateway instances is relatively small compared to the amount of queriers and frontends. Special notes for your reviewer: The previous approach of tracking requests on the client https://github.com/grafana/loki/pull/9781 has been abandoned. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Travis Patterson	8ca035ffbf	Log Volume: Do the prometheus-format conversion in dedicated middleware (#9776 ) This PR moves the Prometheus-format conversion to it's own middleware. This keeps it all in one place and consolidates logic that was in many places. It also sets us up to make range queries as a next step.	2 years ago
dependabot[bot]	af71030e30	Bump anchore/sbom-action from 0.14.2 to 0.14.3 (#9793 ) Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.14.2 to 0.14.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/anchore/sbom-action/releases">anchore/sbom-action's releases</a>.</em></p> <blockquote> <h2>v0.14.3</h2> <h2>Changes in v0.14.3</h2> <ul> <li>update Syft to v0.84.0 (<a href="https://redirect.github.com/anchore/sbom-action/issues/418">#418</a>) [<a href="https://github.com/anchore-actions-token-generator">anchore-actions-token-generator</a>]</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="`78fc58e266`"><code>78fc58e</code></a> chore(deps): update Syft to v0.84.0 (<a href="https://redirect.github.com/anchore/sbom-action/issues/418">#418</a>)</li> <li>See full diff in <a href="https://github.com/anchore/sbom-action/compare/v0.14.2...v0.14.3">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=anchore/sbom-action&package-manager=github_actions&previous-version=0.14.2&new-version=0.14.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Victor Login	30fff3cd65	upgrade version nginxinc/nginx-unprivileged to 1.23 (#8984 ) What this PR does / why we need it: This PR upgrade nginxinc/nginx-unprivileged to 1.23 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Ward Loos	c88a3688e0	Helm - Loki: Fix empty starting line in runtimeConfig (#8657 ) What this PR does / why we need it: The `runtime` ConfigMap in the Loki Helm chart (introduced in `7aa596752d`) starts with an empty line because it improperly removes the current line. ```yaml --- # Source: loki/templates/runtime-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: loki-runtime data: runtime-config.yaml: \| {} ``` While this generates technically valid YAML, it looks weird and interacts badly with some tools. E.g.: - Python ruamel.yaml library: https://stackoverflow.com/questions/75584262/ruamel-yaml-adds-incorrect-indentation-indicator - VS Code which is set to trim whitespace on save (yellow highlighting is for objects, not strings): <img width="213" alt="Screenshot 2023-02-28 at 21 13 10" src="https://user-images.githubusercontent.com/4059732/221967899-719622c8-7393-4336-8c07-934e85a94a30.png"> This PR removes this empty start line: ```yaml --- # Source: loki/templates/runtime-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: loki-runtime data: runtime-config.yaml: \| {} ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
pixiono	7a11a1642b	loki chart: fix service/ingress mapping (#8708 ) What this PR does / why we need it: If you currently deploy the Loki chart under a release name other than "loki" errors occur which is why loki is not accessible via Ingress. The problem looks like this: > $ k describe ingress xyz-loki-app > Name: xyz-loki-app > Labels: app.kubernetes.io/instance=xyz-loki-app > app.kubernetes.io/managed-by=Helm > app.kubernetes.io/name=loki > [...] > Rules: > Host Path Backends > ---- ---- -------- > loki.example.com > /api/prom/tail xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /loki/api/v1/tail xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /loki/api xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /api/prom/rules xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /loki/api/v1/rules xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /prometheus/api/v1/rules xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /prometheus/api/v1/alerts xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /api/prom/push xyz-loki-app-write:3100 (<error: endpoints "xyz-loki-app-write" not found>) > /loki/api/v1/push xyz-loki-app-write:3100 (<error: endpoints "xyz-loki-app-write" not found>) The name of the services is: loki-read & loki-write. I found [another PR with the same problem](https://github.com/grafana/loki/pull/8234) but with a different approach, so I would suggest here to solve the name problem without the question if .name or .fullname should be used. Which issue(s) this PR fixes: Fixes # Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
René Scheibe	954df433e9	Promtail: Fix file descriptor leak (#8987 ) This reduces the amount of open file descriptors per followed journal file from 3 to 2. It also releases file descriptors for removed files (journal vacuuming). The `defaultJournalEntryFunc` is only called once upon startup. It creates a `Journal` instance to only read a single entry to determine the age of the entry referenced in the positions file. Right after, this instance is not required anymore. The actually used `Journal` instance used to follow the journal is created inside `journalTargetWithReader`.	2 years ago
Alexandru Gologan	f785faa820	Helm: add additionalRuleLabels to PrometheusRule alerts (#9020 ) What this PR does / why we need it: Adds the ability to specify other labels on the Prometheus alerts. Specifying: ```yaml monitoring: rules: additionalRuleLabels: custom_label: custom_value ``` will add it to all alerts: ```yaml - alert: LokiRequestErrors annotations: ... expr: ... for: 15m labels: severity: critical custom_label: custom_value ``` This approach is inspired by: [kube-prometheus-stack/rules](https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/templates/prometheus/rules-1.14/general.rules.yaml) Though in their case they run [sync_prometheus_rules.py](https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/hack/sync_prometheus_rules.py) to create the `tpl`. I can understand if this is not the approach (manual) you want to take as editing the tpl adding the labels everywhere and escaping go tpl `{{ }}` is quite painful. Which issue(s) this PR fixes: N/A Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Jim Gustavsson	9eca39601d	Helm: Add PDB when gateway autoscaling is enabled (#8439 ) What this PR does / why we need it: The logic for adding PDB (PodDisruptionBudget) for gateway is not taking autoscaling into account. It is only added when `replicas` > 1. Instead, use the following logic: - When autoscaling is disabled, PDB shall be added when `replicas` > 1 - When autoscaling is enabled, PDB shall be adedd when `autoscaling.minReplicas` > 1 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Nyaundi Brian	5add792487	Update _index.md (#9028 ) What this PR does / why we need it: Typo fix Which issue(s) this PR fixes: None	2 years ago
Bryan Boreham	6928aac776	loser tree: allow sequences to contain maxVal (#9057 ) What this PR does / why we need it: The tree stores the loser in each node, or maxVal if the sequence has ended. If sequences can contain maxVal, we need to take care not to let an ended sequence overtake an active one. Some test cases from Charles who pointed out the issue in #9053. Note this situation cannot happen for real in Loki, since we don't expect log lines with a timestamp that far into the future, but this is supposed to be a re-usable library. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Charles Korn <charles.korn@grafana.com>	2 years ago
Ryo Shindo	ed0d15a6bf	loki: enable PathPrefix for queryHandler endpoints (#8406 ) What this PR does / why we need it: Hi, loki developers. I would like to use the option `http_path_prefix` as described in the official documentation. This setting is valid for endpoints defined at `4d55a24ed3/pkg/loki/loki.go (L477-L494)` but not for the endpoints defined at `4d55a24ed3/pkg/loki/modules.go (L366-L392)` . The current latest version with `http_path_prefix` will return the expected response for the former endpoint, but will return a 404 not found error when accessing the latter endpoint. For example, if you define `http_path_prefix = /xxx`, requests for `/xxx/ready` will work as expected, but requests for `/xxx/loki/api/v1/query` will result in 404 errors. I've created a pull request for this issue because I've been experiencing some inconvenience when using loki. Please check it out. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki/issues/4756 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Tests updated --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Sandeep Sukhani	3e1f2fc273	caching: do not try to fill the gap in log results cache when the new query interval does not overlap the cached query interval (#9757 ) What this PR does / why we need it: Currently, when we find a relevant cached negative response for a logs query, we do the following: * If the cached query completely covers the new query: * return back an empty response. * else: * fill the gaps on either/both sides of the cached query. The problem with filling the gaps is that when the cached query does not overlap at all with the new query, we have to extend the query beyond what the query requests for. However, with the logs query, we have a limit on the number of lines we can send back in the response. So, this could result in the query response having logs which were not requested by the query, which then get filtered out by the [response extractor](`b78d3f0552/pkg/querier/queryrange/log_result_cache.go (L299)`), unexpectedly resulting in an empty response. For example, if the query was cached for start=15, end=20 and we get a `backwards` query for start=5, end=10. To fill the gap, the query would be executed for start=5, end=15. Now, if we have logs more than the query `limit` in the range 10-15, we would filter out all the data in the response extractor and send back an empty response to the user. This PR fixes the issue by doing the following changes when handling cache hit: * If the cached query completely covers the new query: * return back an empty response[_existing_]. * else if the cached query does not overlap with the new query: * do the new query as requested. * If the new query results in an empty response and has a higher interval than the cached query: * update the cache * else: * query the data for missing intervals on both/either side[_existing_] * update the cache with extended intervals if the new queries resulted in an empty response[_existing_] Special notes for your reviewer: We could do further improvements in the handling of queries not overlapping with cached query by selectively extending the queries based on query direction and cached query lying before/after the new query. For example, if the new query is doing `backwards` query and the `cachedQuery.End` < `newQuery.Start`, it should be okay to extend the query and do `cachedQuery.End` to `newQuery.End` to fill the cache since query would first fill the most relevant data before hitting the limits. I did not want to complicate the fix so went without implementing this approach. We can revisit later if we feel we need to improve our caching. Checklist - [x] Tests updated - [x] `CHANGELOG.md` updated --------- Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Paul Rogers	0da462558f	Migrate individual chunks as opposed to all chunks. (#9780 ) This way, if an error occurs, we get as much as we can. What this PR does / why we need it: In some situations, not all blob data may be accessible. For instance, in the case where we have started migration between object storage buckets, some data may be in the old bucket. If we wanted to migrate data from the new bucket, those chunks will error with a 404, as they are in the old bucket. This PR is so that we continue on and get all of the chunks that we can, rather than stopping with whatever error may arise. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Travis Patterson	cc33749b73	fix integration test for multiple periods (#9782 ) The integration test is writing logs across two index periods but there's a chance that the log `lineB` is written after the beginning of the second period after it's added. This means we never get `lineB` in the results. This change pushes back the write time of the first logs so they're always stay in the first index period.	2 years ago
Ashwanth	10e55a6359	store: fix incorrect usage of period config in table range (#9754 ) What this PR does / why we need it: We are incorrectly taking reference to a loop variable in store init, I overlooked this part as the reference was being taken in the call to `GetIndexTableNumberRange` As a result of this, `TableRange` of each period (which is used by index-shipper) points to the latest `periodConfig`. This in itself is not a problem if the index prefix doesn't change as we use the `periodConfig` only to generate the `tableName`. But if the index prefix changes across periods, we end up losing access to the indexes using the old prefix. Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	2 years ago
Susana Ferreira	35465d0297	Fix instant query summary split stats (#9773 ) What this PR does / why we need it: Fix instant query summary statistic's `splits` corresponding to the number of subqueries a query is split into based on `split_queries_by_interval`. * Update rangemapper with a statistics structure to include the number of split queries a query is mapped into. * In the `split_by_range` middleware once the mapped query is returned update the middleware statistics with the number of split queries. This value will then be merged with the statistics of the Loki response. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Hans Kristian Flaatten	86d943d8f4	helm: add updateStategy for canary pods (#9760 ) What this PR does / why we need it: Allows configuring Canary DaemonSet rollout. Which issue(s) this PR fixes: N/A Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Steven Miller	e1002624bd	Helm: Add priorityClassName for Grafana Agent and Loki Canary (#8506 ) What this PR does / why we need it: - Allow configuring PriorityClassName for the GrafanaAgent daemonset - Allow configuring PriorityClassName for the loki canary daemonset without also configuring for read deployment - Helpful when an administrator wants to prioritize logging on their nodes Special notes for your reviewer: - Used _coalese_ style like it was already in the case of loki canary - For loki canary, specifying on the canary takes higher precedence than on the reader, but lower than global - For grafana agent, global takes precedence over grafana agent to match the existing style, but maintainers might consider flipping that around so that most-specific is also most-prioritized Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Travis Patterson	4da0f63789	Remove unused Value field (#9774 ) We didn't end up needing the `Value` field because we can express everything we need to as selectors	2 years ago
Susana Ferreira	a894c3ccf0	Bugfix: Fix logic of offset operator in instant query splitting (#9763 ) What this PR does / why we need it: Fix the logic of the `offset` operator for downstream queries on instant query splitting of (range) vector aggregation expressions already containing an offset. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Susana Ferreira	b611641fea	Add comments to clarify Grouping struct in Aggregation expressions (#9761 ) What this PR does / why we need it: Add comments to clarify the Grouping struct in Vector Aggregation and Range Vector Aggregations expressions. Related with PR: https://github.com/grafana/loki/pull/9515 Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Ashwanth	bbbbe2440f	changelog: moving unreleased changes to the main section (#9769 ) What this PR does / why we need it: noticed that a couple of unreleased changes are incorrectly showing under 2.8.0 release, moving them to the right place. also adds the missing changelog entry for #7447 Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	2 years ago
Travis Patterson	806674fdaa	Add log-volume feature flag (#9762 ) Adds a feature flag for use with the new log-volume endpoints so associated features can be rolled out incrementally.	2 years ago
Trevor Whitney	dbc3040739	Convert SeriesVolume response to prometheus-style (#9703 ) Changes the response type of the label volume stats endpoint to return volumes as prometheus-style timeseries metrics. It currently only supports instant queries, but is a necessary step to eventually supporting range queries.	2 years ago
Trevor Whitney	d74c67a9d0	Add version to TSDB index builder struct (#9566 ) Add version as a property to the TSDB Index Builder, forcing users of the builder to specify a version for each builder instance. This is the first of many PRs that will add the TSDB index version to the schema version config.	2 years ago
Christian Haudum	ce81895241	Add shuffle sharding to index gateway (#9710 ) This PR introduces shuffle sharding of the index files across index gateways where the shuffle sharding key is the tenant ID. Instead of having a global replication factor on the index gateways which affects all tenants the same, whether they are small or large tenants, the shuffle sharding allows to specify a shard factor per tenant (on top of the replication factor). This a) reduces the amount of indexes that are downloaded on individual index gateways in case of a high replication factor (which is currently the de-factor standard to deal with high load) and b) allows to scale tenants individually. The shuffle sharding algorithm is part of dskit's ring implementation and is not part of this commit. If you have set a high RF for the index gateways, it is required to reduce the RF and instead set the shard factor for the tenants. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Joao Marcal	3509edd98d	operator: fix replica count for 1x.ExtraSmall (#9751 )	2 years ago
Periklis Tsirakidis	89a094c5fe	operator: Add rules labels filters for openshift application tenant (#9600 ) Co-authored-by: Mohamed-Amine Bouqsimi <bouqsimiamine@gmail.com>	2 years ago
Dylan Guedes	24e9efb907	Loki: Report event for `GetChunkRef` instead of SpanLogging (#9748 ) What this PR does / why we need it: Modify our calls to `index_client.GetChunkRef` to report invocations through a span event instead of through span logging. This way we won't log a new log line for every GetChunkRef call. Which issue(s) this PR fixes: N/A	2 years ago
Tim Collins	4bb622c54d	helm: table_manager configmap volume (#9742 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #9740 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Dylan Guedes	c6e18a2efb	Loki: Humanize downloaded file size (bytes format) (#9746 ) What this PR does / why we need it: Humanize reported size of downloaded file. Which issue(s) this PR fixes: N/A	2 years ago
Marcin Dufrat	ba8388b8fd	Fix config as a secret mount for single binary statefulset (#9733 ) What this PR does / why we need it: With https://github.com/grafana/loki/pull/8526 came support for config stored as a secret, but `single-binary` statefulset was forgotten. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Joao Marcal	636f3843b7	operator: adjusts 1x.extra-small resources according to findings (#9735 )	2 years ago
Danny Kopping	ccf44f8840	Adding 3 existing maintainers, and our newest one: Ashwanth! (#9741 ) Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	2 years ago
Dylan Guedes	c0d9a2b4ba	Log push failures per-entry instead of in batches (#9720 ) What this PR does / why we need it: Modify how we log push errors to do it per-entry instead of accumulating all errors and then logging them all in a single pass. Which issue(s) this PR fixes: N/A	2 years ago
dependabot[bot]	62bb5b27ae	Bump cachix/install-nix-action from 18 to 22 (#9731 ) Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 18 to 22. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/cachix/install-nix-action/releases">cachix/install-nix-action's releases</a>.</em></p> <blockquote> <h2>install-nix-action-v22</h2> <ul> <li>Nix 2.16.1</li> <li>Fix issues with System Integrity Protection when using macos-12</li> </ul> <h2>install-nix-action-v21</h2> <ul> <li>pin Nix to 2.15.1 (recent releases broke too many things)</li> <li>fix the action to work on custom containers</li> </ul> <h2>install-nix-action-v20</h2> <p><a href="https://discourse.nixos.org/t/nix-2-14-0-released/25900/4Fix">https://discourse.nixos.org/t/nix-2-14-0-released/25900/4Fix</a> support for Nix 2.14 new profile location: <a href="https://redirect.github.com/NixOS/nix/pull/5226">NixOS/nix#5226</a></p> <h2>install-nix-action-v19</h2> <ul> <li>expose github_access_token option and use the default github token by default to prevent rate limiting</li> <li>support <code>--no-daemon</code> installation</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="`6ed004b9cc`"><code>6ed004b</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/184">#184</a> from cachix/macos-bump</li> <li><a href="`e27879448e`"><code>e278794</code></a> Nix: 2.15.1 -> 2.16.1</li> <li><a href="`8ab3881720`"><code>8ab3881</code></a> use system certs</li> <li><a href="`16b951426e`"><code>16b9514</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/182">#182</a> from l0b0/feat/configure-editors</li> <li><a href="`2c203fd87b`"><code>2c203fd</code></a> feat: Configure editors</li> <li><a href="`4b933aa7eb`"><code>4b933aa</code></a> Nix: 2.15.1</li> <li><a href="`35806937f1`"><code>3580693</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/179">#179</a> from joergdw/fix-action-path</li> <li><a href="`3eb7a24508`"><code>3eb7a24</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/178">#178</a> from cachix/docs/149</li> <li><a href="`840ed7ce9a`"><code>840ed7c</code></a> Document how to pass env vars to modern nix commands</li> <li><a href="`b2f4229533`"><code>b2f4229</code></a> Fix action to make it work on custom containers;</li> <li>Additional commits viewable in <a href="https://github.com/cachix/install-nix-action/compare/v18...v22">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cachix/install-nix-action&package-manager=github_actions&previous-version=18&new-version=22)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
dependabot[bot]	4c32ff83fc	bump github.com/heroku/x from 0.0.55 to 0.0.59 (#9732 ) [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/heroku/x&package-manager=go_modules&previous-version=0.0.55&new-version=0.0.59)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Kaviraj Kanagaraj	74a4dca54e	fix(async_store): Fix the `storageconfig.EnableAsyncStore` config (#9725 ) What this PR does / why we need it: During reverting this [PR](https://github.com/grafana/loki/pull/9715). I missed one important flag switch. Which this PR fixes it. Which issue(s) this PR fixes: Fixes NA Special notes for your reviewer: We found a bug on querier due to async_store and we tracked down to[ this PR](https://github.com/grafana/loki/pull/9476). So I reverted the PR in [this PR ](https://github.com/grafana/loki/pull/9715)(manually. Auto revert was not possible as things changed a lot meanwhile) But during my revert PR I missed switching this async store config. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Bryan Boreham	695ea32226	stores/series tests: use abstractions over Labels (#9613 ) What this PR does / why we need it: Stop assuming `labels.Labels` is a slice. Special notes for your reviewer: This PR only touches tests. Upstream Prometheus is using a different implementation of `labels.Labels`, so I plan to create a series of PRs which make Loki compatible with that. #9532 was the first. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Bryan Boreham	904ea0586a	Remove some unused code (#9611 ) What this PR does / why we need it: Remove some functions I came across while making Loki compatible with different `Labels` structures. Move `MetricToLabels` to the only place it is now used, and make it call abstractions instead of assuming `Labels` is a slice. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - NA Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
sarah-spang	6157e28998	Loki ECS Client: Update Name in Options config (#9616 ) Configuration options show "Name": "grafana-loki", however, fluentbit specifies this should just be "loki" A few users have reported receiving an error when using the documented "Name" option, but it works as expected when using the Name specified in the fluentbit docs: Reference: https://docs.fluentbit.io/manual/pipeline/outputs/loki#fluent-bit-+-grafana-cloud	2 years ago
zirain	8bf4e12edf	helm: fix resource not respect namespace (#9678 ) What this PR does / why we need it: this PR make all resources have some behavior when running `helm template loki grafana/loki --name istio-system` istio use it here: https://github.com/istio/istio/blob/master/manifests/addons/gen.sh#L89 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: hejianpeng <hejianpeng2@huawei.com>	2 years ago
Salva Corts	61d407b4bb	Add keep stage to LogQL (#9693 ) What this PR does / why we need it: This PR adds a new `keep` stage to LogQL. It allows filtering specific labels and helps to avoid hitting the _max streams per query_ limit by keeping only specific labels in the label set. It works the opposite of [drop stage](https://grafana.com/docs/loki/latest/logql/log_queries/#drop-labels-expression). Special notes for your reviewer: We do not drop [special error labels](`edc6b0bff7/pkg/logqlmodel/error.go (L18-L20)`) added by Loki at query time. Users will have to explicitly drop them by using the drop stage. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Justin Roberson	262e9d02e1	helm: make ServiceMonitor cluster label respect clusterLabelOverride (#9690 ) What this PR does / why we need it: As of #9218 `clusterLabelOverride` can be used to set the `cluster` label, however the ServiceMonitor still uses `loki.fullname`. This fixes that so it will use `loki.clusterLabel` to be consistent with other components. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Trevor Whitney	7696938928	Update PR template to include add to release notes label (#9691 ) What this PR does / why we need it: Includes a checklist option to the PR template about the `add-to-release-notes` label. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Ed Welch	e4654c9773	Jsonnet: add dns+ to join_members in memberlist config (#9723 ) What this PR does / why we need it: We use a headless service called `gossip-ring` which contains all the memberlist members, previously we passed this headless service into our memberlist code as ``` join_members: - gossip-ring.loki-ops.svc.cluster.local:7946 ``` What this resulted in was the dskit wrapper around the memberlist library thinking there was only 1 join_node, so it passed this directly to the join method in memberlist. The memberlist lib would then do a DNS resolution on this and it would find all the members behind this service. This is generally not noticeable until you get to very large clusters with thousands of memberlist members and it results in new joiners attempting to join to ALL the existing members instead of just a few. This PR prefixes the url with `dns+` which changes the behavior inside dskit in an important way. ``` join_members: - dns+gossip-ring.loki-ops.svc.cluster.local:7946 ``` Now dskit will resolve the address to find all the memberlist members, first, then apply code which selects a subset of all the nodes to be used for first join (based on some logarithmic math) This subset is then passed to the memberlist library which will join just this set instead of all members. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
J Stickler	32320736db	TOC Sync: Move files under Get Started. (#9717 ) Which issue(s) this PR fixes: Fixes #8711 Special notes for your reviewer: Part of the Database products table of contents alignment project. Moves Getting Started and Fundamentals files under Get Started in TOC. Updates front matter for the topic (headings, page weights, descriptions, aliases to redirect from old URL to new URL) Fixes links broken by the move.	2 years ago

1 2 3 4 5 ...

5237 Commits (e2a63e582c4251f2afa35263f0fe04d522444288) All Branches Search

5237 Commits (e2a63e582c4251f2afa35263f0fe04d522444288)

All Branches