5851 Commits (b1f9be5171888629b970e94cc3d39a9aab0e78d7)

 

All Branches   

Search

Author	SHA1	Message	Date
iTrooz	f4ab1e3e89	Doc: Fix link typos in documentation (#10722) ... **What this PR does / why we need it**: On https://grafana.com/docs/loki/latest/get-started/overview/ you can see that two links do not render well. I fixed them in this PR. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Ashwanth	b2f30ccd4a	doc: log retention page improvements (#10665) ... **What this PR does / why we need it**: - Marks table manager as deprecated - Use admonition shortcode for block quotes - Adds description to stream_retention fields - minor restructure **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Karsten Jeschkies	9e89e8d6e2	Check for nil StepResult. (#10704)	2 years ago
Kaviraj Kanagaraj	fcd5dce0ee	doc: Add patching vulnerabilities step on release doc (#10710) ... **What this PR does / why we need it**: Follow up to other release related docs #10646 https://github.com/grafana/loki/pull/10676 Goal is make the doc complete for all type of Loki releases. Stable/Patch/Security releases. Changes: 1. Two new docs a. Patch-Vulnerabilities Overview - patching go dependencies, Go compiler and Docker images b. Patch-Go-Version - How to patch Go version on both loki build image and loki binaries. 2. Renamed files to remove number prefix on the file name e.g: `1-create-release-branch` to just `create-release-branch`. Makes it easy to add more docs without much change. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: Follow up to other release related docs #10646 https://github.com/grafana/loki/pull/10676 **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
A C N	44402d1455	Add urls to connect Grafana (on Docker) to Loki (local buid instance) (#10694) ... 10692 - Added urls for Mac and Windows when connecting a Grafana running on Docker to a locally built Loki instance **What this PR does / why we need it**: I've added 2 URLs (mac and windows) that needs to be used if the user is running a Grafana with docker and trying to connect to a locally built Loki running on localhost (not Docker). When trying to figure out how to run the buids, it was very confusing as to why it's not working, I had to research to try to make the connection work. zDocumentation in setting up a local development environemtn can actually use some improvement as well as I had to moved from one section to another. **Which issue(s) this PR fixes**: Fixes #10692 **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Co-authored-by: A C N <145731451+aclassynerd@users.noreply.github.com>	2 years ago
Vladyslav Diachenko	bd63f371ec	Update structured-metadata docs (#10698) ... OSS users tried to use the structured metadata feature and faced an issue because they used previous schema versions (V11, V12) and they were not aware that it was necessary to use V13 because the docs did not cover it. re: https://github.com/grafana/loki/issues/10683	2 years ago
Trevor Whitney	1415a97296	remove queryrange.VolumeRequest as it is not used (#10685) ... This does not appear to be used anywhere, so removing it. I think we only need the `VolumeResponse` wrapper here.	2 years ago
Dylan Guedes	2133702661	Ingester: Change log line level and make jitter potentially longer. (#10689) ... **What this PR does / why we need it**: - Change our sleep log line from `level=debug` to `level=info` to allow visibility of it without having to change the log line level of distributors as a whole - Make the flush jitter to be 20% of flush check period instead of 1%. This means that, for the default 30s flush check period, the jitter will be at most 6s.	2 years ago
Owen Diehl	8e79c3a1a9	[sharding] [bugfix] Correctly account for child groupings in count/min/max sharding (#10687) ... Previously, we didn't always consider that some label reductions from downstream vector|range-vector aggregations come from groupings. This is particularly useful in the min/max/count sharding mapper. edit: also fixes an issue with the range extractor mishandling `without ()` as `by ()`	2 years ago
Christian Haudum	69afb6ffbe	Fix internal reference in API docs (#10686) ... **What this PR does / why we need it**: When I rebased https://github.com/grafana/loki/pull/10654 before merging I did not resolve the merge conflict correctly. I forgot to remove the link to the section that was removed on main already with a previous commit. Resolves https://github.com/grafana/loki/pull/10654#discussion_r1334601729 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	a3b9534f37	Restructure API reference documentation (#10654) ... * Removed unsupported APIs * Change order so it matches the list on top * Write better headings * Make cURL examples consistent with command-response-blocks Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	b353251e47	Docs: Revise components page (#10666) ... **What this PR does / why we need it**: The page was lacking information about components and also did not contain components that were added already a long time ago. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	d3d7ab7d82	Promtail: Remove deprecated stream_lag_labels setting (#10677) ... The setting was deprecated and already caused an error on startup when used. This behaviour was noticed when looking at the reference configuration docs of Promtail. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	760ad8691a	Docs: Remove experimental flag from GELF support in Promtail (#10684) ... GELF support in Promtail is not experimental (any more). **Special notes for your reviewer**: Nothing really worth mentioning in the changelog or release notes. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Ashwanth	2902f986a4	cache: Enable embedded cache if no other cache is configured. (#10620) ... **What this PR does / why we need it**: FIFO cache is replaced by embedded-cache in https://github.com/grafana/loki/pull/6821 This PR enables embedded cache for query range results, chunks cache, index_stats_results and volume_results if no other cache is explicitly configured (redis, memcache). **Special notes for your reviewer**: Also removes a deprecated flag `cache-split-interval` **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	2d7930e6d8	Revise documentation about Loki architecture (#10672) ... Also split out legacy storage into separate page. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Kaviraj Kanagaraj	1b24d2af9a	chore: Missed version updates. (#10676) ... **What this PR does / why we need it**: ``` LOKI_NEW_VERSION=2.9.1 ./tools/release_update_tags.sh ``` This script was introduced as the part of release docs PR. https://github.com/grafana/loki/pull/10646 **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Kaviraj Kanagaraj	04716b5d09	dependabot: Fix accessing secerets via environmental variables. (#10675) ... **What this PR does / why we need it**: Noticed, the dependencies auto-merge is breaking with error. ``` gh: To use GitHub CLI in a GitHub Actions workflow, set the GH_TOKEN environment variable. Example: env: GH_TOKEN: ${{ github.token }} ``` https://github.com/grafana/loki/actions/runs/6222148810/job/16885518452?pr=10623 After some investigation. The suspect is secerets are not properly imported into the GH actions (NOTE from above GH run, the env `GITHUB_TOKEN` is set empty). Also from the docs, this token can be set either by `GITHUB_TOKEN` or `GH_TOKEN`. `GITHUB_TOKEN` will get higher priority. https://cli.github.com/manual/gh_help_environment **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Kaviraj Kanagaraj	a8827380a7	docs: Make `release` steps in `_index.md` (#10667)	2 years ago
Roel Arents	5fb1cdb5dd	azure: add connection-string (#10187) ... **What this PR does / why we need it**: Adds a connection-string option to the azure blob storage config. So the [Azurite](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite) emulator can be used in local test environments (my use case). But also to support the use of a SAS token. See the (future) addition of this option to the Thanos object storage client: https://github.com/thanos-io/objstore/pull/51 **Which issue(s) this PR fixes**: n/a **Special notes for your reviewer**: https://github.com/thanos-io/objstore/pull/51 isn't merged yet. But I'm already creating this PR to see what others think. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Karsten Jeschkies	b9c9394dc1	Define sketches for quantiles. (#10659)	2 years ago
Nicolas Lamirault	c29ba973a0	Fix: Sidecar configuration for Backend (#10603) ... **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) With this values: ```yaml sidecar: image: # -- The Docker registry and image for the k8s sidecar repository: kiwigrid/k8s-sidecar # -- Docker image tag tag: 1.24.3 # -- Docker image sha. If empty, no sha will be used sha: "" # -- Docker image pull policy pullPolicy: IfNotPresent # -- Resource requests and limits for the sidecar resources: #{} limits: cpu: 100m memory: 100Mi requests: cpu: 50m memory: 50Mi # -- The SecurityContext for the sidecar. securityContext: runAsNonRoot: true runAsGroup: 10001 runAsUser: 10001 fsGroup: 10001 # -- Set to true to skip tls verification for kube api calls. skipTlsVerify: false # -- Ensure that rule files aren't conflicting and being overwritten by prefixing their name with the namespace they are defined in. enableUniqueFilenames: false # -- Readiness probe definition. Probe is disabled on the sidecar by default. readinessProbe: httpGet: path: /ready port: http-metrics initialDelaySeconds: 30 timeoutSeconds: 1 # -- Liveness probe definition. Probe is disabled on the sidecar by default. livenessProbe: httpGet: path: /ready port: http-metrics initialDelaySeconds: 30 timeoutSeconds: 1 rules: # -- Whether or not to create a sidecar to ingest rule from specific ConfigMaps and/or Secrets. enabled: true # -- Label that the configmaps/secrets with rules will be marked with. label: loki_rule # -- Label value that the configmaps/secrets with rules will be set to. labelValue: "" # -- Folder into which the rules will be placed. folder: /rules # -- Comma separated list of namespaces. If specified, the sidecar will search for config-maps/secrets inside these namespaces. # Otherwise the namespace in which the sidecar is running will be used. # It's also possible to specify 'ALL' to search in all namespaces. searchNamespace: null # -- Method to use to detect ConfigMap changes. With WATCH the sidecar will do a WATCH request, with SLEEP it will list all ConfigMaps, then sleep for 60 seconds. watchMethod: WATCH # -- Search in configmap, secret, or both. resource: both # -- Absolute path to the shell script to execute after a configmap or secret has been reloaded. script: null # -- WatchServerTimeout: request to the server, asking it to cleanly close the connection after that. # defaults to 60sec; much higher values like 3600 seconds (1h) are feasible for non-Azure K8S. watchServerTimeout: 60 # # -- WatchClientTimeout: is a client-side timeout, configuring your local socket. # If you have a network outage dropping all packets with no RST/FIN, # this is how long your client waits before realizing & dropping the connection. # Defaults to 66sec. watchClientTimeout: 60 # -- Log level of the sidecar container. logLevel: INFO ``` Before the patch: ```yaml --- apiVersion: apps/v1 kind: StatefulSet metadata: name: loki-backend namespace: default labels: helm.sh/chart: loki-5.22.0 app.kubernetes.io/name: loki app.kubernetes.io/instance: release-name app.kubernetes.io/version: "2.9.1" app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: backend app.kubernetes.io/part-of: memberlist spec: replicas: 3 podManagementPolicy: Parallel updateStrategy: rollingUpdate: partition: 0 serviceName: loki-backend-headless revisionHistoryLimit: 10 persistentVolumeClaimRetentionPolicy: whenDeleted: Delete whenScaled: Delete selector: matchLabels: app.kubernetes.io/name: loki app.kubernetes.io/instance: release-name app.kubernetes.io/component: backend template: metadata: annotations: checksum/config: a9f8d008a1b589265e3fa894e3b81ab2a4132bb741d3b6e8db304a1e6b6d3ba5 labels: app.kubernetes.io/name: loki app.kubernetes.io/instance: release-name app.kubernetes.io/component: backend app.kubernetes.io/part-of: memberlist spec: serviceAccountName: loki automountServiceAccountToken: true securityContext: fsGroup: 10001 runAsGroup: 10001 runAsNonRoot: true runAsUser: 10001 terminationGracePeriodSeconds: 300 containers: - name: loki-sc-rules image: "kiwigrid/k8s-sidecar:1.24.3" imagePullPolicy: IfNotPresent env: - name: METHOD value: WATCH - name: LABEL value: "loki_rule" - name: FOLDER value: "/rules" - name: RESOURCE value: "both" - name: WATCH_SERVER_TIMEOUT value: "60" - name: WATCH_CLIENT_TIMEOUT value: "60" - name: LOG_LEVEL value: "INFO" livenessProbe: null readinessProbe: null resources: limits: cpu: 100m memory: 100Mi requests: cpu: 50m memory: 50Mi securityContext: fsGroup: 10001 runAsGroup: 10001 runAsNonRoot: true runAsUser: 10001 volumeMounts: - name: sc-rules-volume mountPath: "/rules" - name: loki image: docker.io/grafana/loki:2.9.1 imagePullPolicy: IfNotPresent args: - -config.file=/etc/loki/config/config.yaml - -target=backend - -legacy-read-mode=false ports: - name: http-metrics containerPort: 3100 protocol: TCP - name: grpc containerPort: 9095 protocol: TCP - name: http-memberlist containerPort: 7946 protocol: TCP securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL readOnlyRootFilesystem: true readinessProbe: httpGet: path: /ready port: http-metrics initialDelaySeconds: 30 timeoutSeconds: 1 volumeMounts: - name: config mountPath: /etc/loki/config - name: runtime-config mountPath: /etc/loki/runtime-config - name: tmp mountPath: /tmp - name: data mountPath: /var/loki - name: sc-rules-volume mountPath: "/rules" resources: {} affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchLabels: app.kubernetes.io/name: loki app.kubernetes.io/instance: release-name app.kubernetes.io/component: backend topologyKey: kubernetes.io/hostname volumes: - name: tmp emptyDir: {} - name: config configMap: name: loki items: - key: "config.yaml" path: "config.yaml" - name: runtime-config configMap: name: loki-runtime - name: sc-rules-volume emptyDir: {} volumeClaimTemplates: - metadata: name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: "10Gi" ``` After the patch: ```yaml --- # Source: loki/templates/backend/statefulset-backend.yaml apiVersion: apps/v1 kind: StatefulSet metadata: name: loki-backend namespace: default labels: helm.sh/chart: loki-5.22.1 app.kubernetes.io/name: loki app.kubernetes.io/instance: release-name app.kubernetes.io/version: "2.9.1" app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: backend app.kubernetes.io/part-of: memberlist spec: replicas: 3 podManagementPolicy: Parallel updateStrategy: rollingUpdate: partition: 0 serviceName: loki-backend-headless revisionHistoryLimit: 10 persistentVolumeClaimRetentionPolicy: whenDeleted: Delete whenScaled: Delete selector: matchLabels: app.kubernetes.io/name: loki app.kubernetes.io/instance: release-name app.kubernetes.io/component: backend template: metadata: annotations: checksum/config: f1823dbb8a5e31ab8d2ad66878f6abb71a9e0f49c8768462d873da21192e208b labels: app.kubernetes.io/name: loki app.kubernetes.io/instance: release-name app.kubernetes.io/component: backend app.kubernetes.io/part-of: memberlist spec: serviceAccountName: loki automountServiceAccountToken: true securityContext: fsGroup: 10001 runAsGroup: 10001 runAsNonRoot: true runAsUser: 10001 terminationGracePeriodSeconds: 300 containers: - name: loki-sc-rules image: "kiwigrid/k8s-sidecar:1.24.3" imagePullPolicy: IfNotPresent env: - name: METHOD value: WATCH - name: LABEL value: "loki_rule" - name: FOLDER value: "/rules" - name: RESOURCE value: "both" - name: WATCH_SERVER_TIMEOUT value: "60" - name: WATCH_CLIENT_TIMEOUT value: "60" - name: LOG_LEVEL value: "INFO" livenessProbe: httpGet: path: /ready port: http-metrics initialDelaySeconds: 30 timeoutSeconds: 1 readinessProbe: httpGet: path: /ready port: http-metrics initialDelaySeconds: 30 timeoutSeconds: 1 resources: limits: cpu: 100m memory: 100Mi requests: cpu: 50m memory: 50Mi securityContext: fsGroup: 10001 runAsGroup: 10001 runAsNonRoot: true runAsUser: 10001 volumeMounts: - name: sc-rules-volume mountPath: "/rules" - name: loki image: docker.io/grafana/loki:2.9.1 imagePullPolicy: IfNotPresent args: - -config.file=/etc/loki/config/config.yaml - -target=backend - -legacy-read-mode=false ports: - name: http-metrics containerPort: 3100 protocol: TCP - name: grpc containerPort: 9095 protocol: TCP - name: http-memberlist containerPort: 7946 protocol: TCP securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL readOnlyRootFilesystem: true readinessProbe: httpGet: path: /ready port: http-metrics initialDelaySeconds: 30 timeoutSeconds: 1 volumeMounts: - name: config mountPath: /etc/loki/config - name: runtime-config mountPath: /etc/loki/runtime-config - name: tmp mountPath: /tmp - name: data mountPath: /var/loki - name: sc-rules-volume mountPath: "/rules" resources: {} affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchLabels: app.kubernetes.io/name: loki app.kubernetes.io/instance: release-name app.kubernetes.io/component: backend topologyKey: kubernetes.io/hostname volumes: - name: tmp emptyDir: {} - name: config configMap: name: loki items: - key: "config.yaml" path: "config.yaml" - name: runtime-config configMap: name: loki-runtime - name: sc-rules-volume emptyDir: {} volumeClaimTemplates: - metadata: name: data spec: accessModes: - ReadWriteOnce resources: requests: storage: "10Gi" ``` --------- Signed-off-by: Nicolas Lamirault <nicolas.lamirault@gmail.com> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Christian Haudum	4704d5e51a	Docs: Fix broken link on WAL page (#10673) ... The reference to a headline on the API page broke, because the headline was removed in https://github.com/grafana/loki/pull/10655 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Anton Lindholm	fe223a21e5	helm: improve gitops compatibility (#10452) ... **What this PR does / why we need it**: In its current form, this helm chart is annoying to deploy using GitOps tools like fluxcd/argocd. This is due to missing fields that get added automatically once the objects are applied in the cluster. A workaround is to just add exceptions for these fields in the gitops tool, but I think this is a more robust solution. Screenshot from ArgoCD app diff:  **Which issue(s) this PR fixes**: **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Christian Haudum	0f5fbb6701	Log process startup and running time (#10656) ... This helps to get insights about slow startup of Loki, e.g. for the index gateway, which may pre-fetch a lot of data before it becomes ready. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com>	2 years ago
Christian Haudum	e03637151b	Remove legacy ingester shutdown handler (#10655) ... The handler was deprecated for a while and is removed in favour or `/ingester/shutdown?flush=true`. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Kaviraj Kanagaraj	8721e41c6c	drone(docker-driver): Fetch tags before pushing image during release (#10664) ... **What this PR does / why we need it**: We realized, loki-docker-driver image was not being pushed after 2.8.2. And looking into it's drone pipeline and comparing it with other steps, docker-driver step is not fetching the tags,[ thus the final hash calculated to push is some HEAD branch.](https://github.com/grafana/loki/blob/main/tools/image-tag#L13-L14) e.g: even if we have tag 2.9.1, it pushes only with latest HEAD commit hash. This PR fixes it.    **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: I would prefer this instead of https://github.com/grafana/loki/pull/10663. Because that would make fetching tags more repetitive in all the drone tasks (can also be expensive in terms of running time if we put all together) **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Kaviraj Kanagaraj	446502bad0	docs/scripts: Update Loki OSS release guide (#10646) ... **What this PR does / why we need it**: Update Loki OSS release guide This is based on the learnings from recent minor version release v2.9.0 Changes: 1. Update the release docs and organise it in a way it's usable for both stable and patch release 2. Scripts updates **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2 years ago
J Stickler	aed39c89ee	[docs] Replace ampersand with the word "and" (#10637) ... Replacing the ampersand character (&) with the word `and` to conform to the Style Guide https://grafana.com/docs/writers-toolkit/write/style-guide/capitalization-punctuation/#punctuation-guidelines Do not abbreviate and with an ampersand (&).	2 years ago
Šimon Lukašík	209b4d83af	fluentd docs: fix broken link (#10640) ... Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Kaviraj Kanagaraj	c636a0f51c	Add missing changelog for sharding optimizations (#10469) ... **What this PR does / why we need it**: https://github.com/grafana/loki/pull/10101 **Which issue(s) this PR fixes**: NA **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] `CHANGELOG.md` updated Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Piotr	a8d5815510	Fix race condition when stopping to tail files (#10631) ... **What this PR does / why we need it**: Users are getting `PromtailFileMissing` alerts, which are defined as: ``` promtail_file_bytes_total{namespace!=""} unless promtail_read_bytes_total{namespace!=""} ``` This appears to be due to a race condition where a file can be deleted while the file target is being stopped.	2 years ago
Travis Patterson	641c9ee48a	Don't allow unbounded parallelism when downloading indices (#10634) ... Index Gateways can stall when certain queries try to download tons of indices because. `ErrGroup` needs an explicit limit set on it. This PR sets the limit provisionally until a more configurable solution can be introduced. --------- Co-authored-by: Dylan Guedes <dylan.guedes@grafana.com>	2 years ago
Šimon Lukašík	f184c5db51	docs: master tag hasn't been pushed for almost 2 years (#10633)	2 years ago
Owen Diehl	02ae366451	tool for building bloom filters from log contents (#10594) ... This adds a tool which builds scalable bloom filters from the contents of logs themselves & records metrics. --------- Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Pablo	222ffc52d3	Add CloudFormation template for lambda + EventBridge setup (#10540) ... **What this PR does / why we need it**: https://github.com/grafana/loki/pull/10449 added support so that lambda-promtail can handle s3 events through AWS EventBridge. This PR is a follow up adding a CloudFormation template to deploy lambda-promtail, with the corresponding EventBridge rule to deliver s3 events to the lambda. **Which issue(s) this PR fixes**: Rel https://github.com/grafana/loki/issues/10209 --------- Co-authored-by: J Stickler <julie.stickler@grafana.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	3a8a5e54dd	Fix object store type for table manager (#10632) ... **What this PR does / why we need it**: This commit fixes a regression in the object store client creation introduced with #10518 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	769979c9f6	chore(storage): Refactor store factory (#10518) ... **What this PR does / why we need it**: This PR makes a clearer distinction between supported and deprecated index/chunk storage types. There are individual commits for the different factory functions: * Refactor table client factory * Refactor index client factory * Refactor chunk client factory * Split initStore function --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Kevin Burke	9dad6a1350	clients,pkg: use http.NewRequestWithContext (#10616) ... **What this PR does / why we need it**: It saves one line, and the new function (vs. http.NewRequest) performs additional validation of the request. **Which issue(s) this PR fixes**: None **Special notes for your reviewer**: None **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) Not necessary: - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Kevin Burke	23d8c7c63c	clients/pkg/promtail/client: better error on nil metrics (#10611) ... Fixes #10610. **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) Not necessary: - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Kevin Burke	1256692f95	clients/pkg/promtail/targets/gcplog: fix spelling error (#10615) ... **What this PR does / why we need it**: Fix spelling error **Which issue(s) this PR fixes**: none **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) Not necessary: - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Robert Jacob	835d0452d4	operator: Update example configurations to use demo size (#10606)	2 years ago
Periklis Tsirakidis	5415b11468	operator: Update Loki operand to v2.9.1 (#10600)	2 years ago
Grot (@grafanabot)	ca43a6ba77	[CI/CD] Update yaml file `./production/helm/loki/Chart.yaml` (+1 other) (#10592) ... **Here is a summary of the updates contained in this PR:** *** Update attribute `$.appVersion` in yaml file `./production/helm/loki/Chart.yaml` to the following value: `2.9.1` *** Bump version of Helm Chart Add changelog entry to `./production/helm/loki/CHANGELOG.md` Re-generate docs Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	f17e55030a	Reflect release 2.9.1 on main branch (#10597) ... **What this PR does / why we need it**: Reflect Loki 2.9.1 release on main branch to be up-to-date with latest released version. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Raul Gonzales	a7b83bf9bb	chore(docs): Fix structured_metadata usage (#10526) ... **What this PR does / why we need it**: According to the code the pipeline name uses `_` not `-` for the name https://github.com/grafana/loki/blob/main/clients/pkg/logentry/stages/stage.go#L46C38-L46C38 Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Christian Haudum	e858779e6b	Fix bug in cache of the index object client (#10587) ... The bug could manifest in missing results when querying logs older than what is kept on ingesters. This commit changes the behaviour of the caching in the storage client client used by the indexshipper to download indexes from object store. The object store client maintains an internal, in-memory cache of tables (= per-day index directories) available on object storage to reduce the amount of expensive List operations. This cache gets updated in a regular interval (1m) by the table manager that uses the client. However, when the per-tenant setting `query_ready_index_num_days` is set the 0, which is the default (to disable index pre-fetching), the sync is only performed once on startup, but not subsequently. This leads to a stale list of table names in the cache and any new index directory added to the object storage is not added to the client cache. This commit changes the object client to use a read-through caching strategy for listing tables. If a table is not found locally in cache, it performs a lookup against object storage. In this way, the cache is updated even without the regular sync described above. **Special notes for your reviewer**: This is a port of #10585 which was first applied on the 2.9 release branch. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Dylan Guedes	79fcdb4f4c	Update references to 2.8.5 (#10590) ... **What this PR does / why we need it**: We're releasing a new 2.8.5 patch. This adds references to it to our docs.	2 years ago
J Stickler	bdfa240647	[Docs] Fix 404s/broken links in main branch (#10556) ... Cleaning up broken links reported against the main branch. --------- Co-authored-by: Jack Baldry <jack.baldry@grafana.com>	2 years ago
Robert Jacob	8133b34b23	operator: Update gateway arguments to enable namespace extraction (#10545)	2 years ago