3889 Commits (c271f7923a31de403c6fb7c12912a7de5987bdbf)

 

All Branches   

Search

Author	SHA1	Message	Date
Michel Hollands	c271f7923a	Fix setting name in documentation (#7086) ... Signed-off-by: Michel Hollands <michel.hollands@grafana.com> **What this PR does / why we need it**: The delete_max_interval was previously named delete_max_query_range. This rename was not done in the documentation, **Checklist** - [X] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Arkadiusz Spiewak	d7b86ba940	Helm: Fix s3 region in ruler storage config (#7077) ... **What this PR does / why we need it**: Currently helm chart helper does not pass region or any other information if s3 storage is configured. This causes issues with connecting to S3 on AWS. This Pr fixes it, by passing all revelant configs to ruler storage config. **Which issue(s) this PR fixes**: Fixes #7060 **Special notes for your reviewer**: **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Michel Hollands	08cf2153d5	Add sequence num to log output for delete requests (#7087) ... Signed-off-by: Michel Hollands <michel.hollands@grafana.com> **What this PR does / why we need it**: Print the sequence number of the delete request so the messages are not all the same. **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Periklis Tsirakidis	1a41682bb0	Fix internal server initialization (#7085)	3 years ago
Ryan Garcia	3324427814	Helm: Adding template-able image value for tokengen job container (#7075) ... **What this PR does / why we need it**: Currently the image for the `create-secret` container within the tokengen job is not configurable, this adds a value to make the image template-able. **Which issue(s) this PR fixes**: Fixes #7074 **Special notes for your reviewer**: **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: garcia.ryan <garcia.ryan@solute.us>	3 years ago
Periklis Tsirakidis	c30fb9450b	Add support for custom internal server (#7069) ... Co-authored-by: Christian Simon <simon@swine.de>	3 years ago
Ludovic Cleroux	e2952f9ce1	Troubleshooting: instructions for loki + istio (#6205) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: Add troubleshooting instructions for running istio and loki. It takes a long time to figure out how to make it work, hope it will help others. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [x] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Trevor Whitney	9c47b3084e	Add prefix for helm-release tags (#7073) ... **What this PR does / why we need it**: The helm release job creates a tag for each new release. The tags was previously `<chart name>-<chart version>`, which in the case of the loki chart in this repo was `loki-3.0.0`. This is confusing, so we are adding the prefix `helm-` to these tags.	3 years ago
Adam	eaf7f34f49	bugfix:add cases.NoLower option (#7052) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: see #7048 if we don't apply cases.NoLower, it will upper first alphabet lower the others ```go package main import ( "fmt" "golang.org/x/text/cases" "golang.org/x/text/language" "strings" ) func main() { source := "testAPI" fmt.Println(cases.Title(language.Und).String(source)) // out: Testapi fmt.Println(cases.Title(language.Und, cases.NoLower).String(source)) // out: TestAPI fmt.Println(strings.Title(source)) // out: TestAPI } ``` **Which issue(s) this PR fixes**: **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: wujw39640 <wujw39640@hundsun.com> Signed-off-by: wujunwei <wjw3323@live.com> Co-authored-by: wujw39640 <wujw39640@hundsun.com>	3 years ago
Xavier Coulon	30548ccc9c	docs: fix typo in layout (#7071) ... **What this PR does / why we need it**: Fix a typo in the layout: `kuberentes` instead of `kubernetes`	3 years ago
Jonathan	6b9bf0b5f4	feat: add kms and ec2 permission (#7058) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: This PR will help to increase a necessary policy (kms and ec2) to resource log group and lambda. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Trevor Whitney	a369189752	Add nix configuration (#7020) ... **What this PR does / why we need it**: This PR adds nix configuration to the Loki repo to make it easier for people who use nix to integrate building Loki into their existing workflows. For example, I have a `direnv` hook that uses nix shell to make sure I have the correct version of all dependent binaries (ie. `go`, `golangci-lint`, etc) on my `$PATH` whenever I change into the Loki directory. I find this especially useful for building with `CGO` enabled as it requires a C compiler with systemd headers to be available on the path. Furthermore, by having a `flake.nix` in the root of this repo, it will make it easier for users of the nix package manager to grab a specific version of Loki for their own systems / projects. This PR does _not_ attempt to replace any existing build processes, this is merely supplemental. All these nix configurations wrap existing `Makefile` targets, and the addition of these files will have no effect on anyone not using nix. Please see the README.md included in this PR for more information. **Checklist** - [X] Documentation added Co-authored-by: Christian Simon <simon@swine.de>	3 years ago
Mohamed-Amine Bouqsimi	a068c1241d	operator: Configure gateway to honor the global TLS security profile (#6870)	3 years ago
Ashwanth	53e01a795a	tsdb: build tsdb from tenantheads (#7036) ... **What this PR does / why we need it**: refactors tsdb manager code to expose a new method `BuildFromHead` which comes in handy for building tsdb from tenant heads after rotation. Currently we build it by recovering head from WAL even when we have access to the in-mem tenant heads. **Special notes for your reviewer**: using `loki_tsdb_head_rotations_*` metric to track loop failures which could be caused either by rotation failures or if rotation is blocked by prev head failing to build **Checklist** - [ ] Documentation added - [x] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	3 years ago
李国忠	93a5a71e62	[doc] logql: logql engine support exec vector(0) grama (#7044) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: logql engine support exec vector(0) grama. new PR of :https://github.com/grafana/loki/pull/7023 **Which issue(s) this PR fixes**: Fixes #6946 **Special notes for your reviewer**: preview  <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Danny Kopping <dannykopping@gmail.com>	3 years ago
Michel Hollands	34c2c0c4a9	Update metrics descriptions (#6982) ... **What this PR does / why we need it**: The docs team reviewed and fixed the descriptions of the deletion metrics added to the documentation in https://github.com/grafana/loki/pull/6971. The fix is in a separate PR https://github.com/grafana/loki/pull/6976. This PR fixes that same description in the code. **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Adam	d69d3b664c	Deprecation Fix: Replace strings.Title with cases.Title().String() (#7048) ... Signed-off-by: wujunwei <wjw3323@live.com> <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: [Golang 1.18 deprecates the `strings.title function`](https://github.com/golang/go/issues/48367) use `cases.Title(language.Und, cases.NoLower).String()` instead. **Which issue(s) this PR fixes**: **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: wujw39640 <wujw39640@hundsun.com>	3 years ago
Trevor Whitney	e39ea21e3c	Do not run helm release on tags matching our version releases (#7038) ... **What this PR does / why we need it**: We don't want to run the helm release on tags matching our loki releases. We can add the helm chart to the release process later, but we're not there yet. Right now our flow is ship the updated loki binary, then circle back around and update the helm charts to use that new version is a separate PR/release.	3 years ago
Karsten Jeschkies	aa3716e84c	Helm: Publish Helm charts to grafana/helm-charts. (#6955) ... **What this PR does / why we need it**: This change pulls in the GitHub workflow for publishing Loki Helm charts to `grafana/helm-charts`. **Which issue(s) this PR fixes**: Fixes #6877	3 years ago
Trevor Whitney	c0eb73c34a	Enable build in env w/o internet or GCP creds (#7029) ... **What this PR does / why we need it**: The build is currently dependent upon you building in an environment with internet connectivity and GCP credentials. This PR fixes the few parts of our code and test that rely on these external dependencies so it can be built in an environment without internet or GCP credentials (such as when building a [nix flake](https://github.com/grafana/loki/pull/7020)) **Which issue(s) this PR fixes**: Fixes #7028	3 years ago
Ashwanth	5ea38fbd51	tsdb: build rotated heads asynchronously (#6956) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: Currently `Append` call to the head has to wait for block creation to complete when rotating head. And since we rotate head every 15m (default period), this would result in latency spikes at these times. This PR moves the TSDB build step to an async routine. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
李国忠	3296c6dbe8	[new feature] logql engine support exec `vector(0)` grama (#7023) ... **What this PR does / why we need it**: logql engine support exec `vector(0)` grama. new PR of :https://github.com/grafana/loki/pull/7007 **Which issue(s) this PR fixes**: Fixes #6946 **Special notes for your reviewer**: test passed. test1: vectorExpr only  test2: binExpr and vectorExpr  **Checklist** - [ ] Documentation added - [x] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Travis Patterson	bc16450349	Revert "Hide duplicates caused by stream sharding (#7005)" (#7031) ... @owen-d had a suggestion that was dramatically dimpler. Revert this whole thing and the simplified PR will be forthcoming	3 years ago
Gerard Vanloo	86f6c2eb8d	operator: Adding Lokistack Gateway Request Errors Alert (#6999)	3 years ago
Robert Jacob	766d4b7b5d	operator: Configure default node affinity for all pods (#7000)	3 years ago
Trevor Whitney	5ca7c29e2c	Helm: test migration from `grafana/loki-simple-scalable` (#6986) ... **What this PR does / why we need it**: Verify the upgrade path from `grafana/loki-simple-scalable` -> `grafana/loki` works. Fixes #6874	3 years ago
李国忠	25cc9402b7	fix test increase timeout (#7022) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: increase timeout, fix test fail. ``` --- FAIL: TestInstanceFor (0.01s) downstreamer_test.go:205: Error Trace: downstreamer_test.go:205 downstreamer_test.go:256 Error: lock couldn't be acquired Test: TestInstanceFor FAIL ``` **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
李国忠	6f4671a572	[new feature] syntax: support "vector" grama (#7007) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: Implement vector() function. The "vector function" feature is divided into multiple PRs to avoid excessively large PRs resulting in too long review time. **Which issue(s) this PR fixes**: Fixes #6946 **Special notes for your reviewer**: The test for syntax has passed  <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
sardismart	821301b95f	azblob: Make azblob support private cloud endpoint suffix (#6842) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: The endpoint suffix of private cloud(azure stack hub) azblob is different from Azure. The loki configuration has already support customize endpoint suffix but not use when select blob/container URL. This PR introduce following change: * Consider endpoint suffix from configuration when select blob or container URL format **Which issue(s) this PR fixes**: Fixes #6863 **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [x] Documentation added - [x] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Travis Patterson	a98b7ebaf4	Hide duplicates caused by stream sharding (#7005) ... As streams are sharded and resharded across different ingesters, the likelihood of duplicate logs being ingested increases dramatically. This removes duplicates at query time by dynamically removing the `__shard_stream__` label for both log and sample queries so the `stream hash` used by the `MergeEntryIterator` will match when doing comparisons for duplicates. The `stream hash` is created in either the `store` or `ingester instance` at query time so the labels are removed there. Additionally, the `MergeEntryIterator` has been modified to remove duplicates in a stream rather than just across streams. For completeness, `__shard_stream__` has also been removed from Label and Series Queries.	3 years ago
Karsten Jeschkies	c23e541fe7	Generate Helm chart README. (#7012) ... **What this PR does / why we need it**: The Helm CI would break because the Helm chart README was not generated. This change * updates the README * introduces a filter that will only run the Helm CI on changes to the Helm chart directory * removes trailing whitespace * corrects the Minio chart repository	3 years ago
Karsten Jeschkies	1c3f5d0985	Helm: Upgrade Minio chart dependency to 4.0.12. (#6974) ... **What this PR does / why we need it**: Changes to a supported Minio chart. Minio is also enabled by default. **Which issue(s) this PR fixes**: Fixes #6973 **Checklist** - [x] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
李国忠	562ccd6932	[bugfix] fix flush.go nil pointer panic (#6993) ... **What this PR does / why we need it**: Fix a panic when reporting chunk statistics. The chunk would already be garbage-collected as it is marked as flushed before reporting its statistics. This fix it by first reporting its statistics and only then marking it as flushed. Panic description: ```go panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x177d158] goroutine 14455 [running]: github.com/grafana/loki/pkg/chunkenc.(*MemChunk).Size(0xc00441c750?) /loki@v1.6.2-0.20220816085209-65ed7b60fbd4/pkg/chunkenc/memchunk.go:611 +0x18 github.com/grafana/loki/pkg/ingester.(*Ingester).reportFlushedChunkStatistics(0xc00087a000, 0xc00f8efe80, 0xc0519d0870, {0x36b7008, 0xc01a593860}, {0x36b7008, 0xc01a5938c0}, {0x307e2f9, 0x7}) /loki@v1.6.2-0.20220816085209-65ed7b60fbd4/pkg/ingester/flush.go:395 +0x2f2 ```	3 years ago
Dylan Guedes	bba6c14459	Loki: Modify ingesters to return rate-limited streams to distributors (#6977) ... **What this PR does / why we need it**: Modify ingesters to return a list of rate-limited streams on its reply to `(*distributor).Push`. This is done by using the gRPC details feature. I've updated the tests and apparently, the returned error message is exactly the same so this shouldn't be a breaking change. Also modifies `(*stream).Push` to return a list of all entries with errors instead of returning the final gRPC error. This will allow us in the future to easily return to distributors the indices of all reject entries, so that distributors won't have to always retry all entries. This work is necessary for the new stream sharding feature.	3 years ago
Owen Diehl	30db6c120e	[TSDB] 600MB max per shard in query planning (#7004) ... This PR gives a little better understanding into shard query planning and targets a range of `(300MB,600MB]` per shard. Doubling this also allows us to halve query parallelism configs and still schedule the same amount of work in parallel, which should make for easier schema upgrades as the parallelism differences between TSDB and prior period configs will be less. Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	3 years ago
Kaviraj Kanagaraj	5a6d09f83e	doc(community): Update community call timings. (#7002)	3 years ago
afayngelerindbx	36109eb09a	compactor: prefetch existing compacted user index files in parallel (#6803) ... **What this PR does / why we need it**: Before this change, these files are fetched while [holding a table global lock](a1e75da970/pkg/storage/stores/shipper/index/compactor/table_compactor.go (L195)), serializing fetches. I chose the approach of prefetching index files over attempting to change the locking pattern to allow parallel, on-demand fetches because the complexity of parallelizing on-demand fetches outweighed the benefits of avoiding extra fetches. More specifically, this patch trades off fetching idle, non-compactable index files in favor of avoiding blocking during compactions. The extra fetches are parallelized and so should incur a negligible wall clock penalty even for large numbers of tenants(e.g. 1k idle tenants at 50 threads and 500ms to fetch is ~10s). **Which issue(s) this PR fixes**: This should help with #6775, without completely resolving it. **Checklist** This change does not introduce any settings, metrics etc. It should be transparent from a user perspective. Therefore, I don't believe it merits documentation/changelog updates. - [x] Tests updated	3 years ago
Gerard Vanloo	1bb535a730	operator: Fixing logcli pod image value for operator addons (#6997)	3 years ago
Christian Haudum	177f37700a	Make TSDB store singleton resettable (#6950) ... This changes allows downstream projects to reset the TSDB store singleton. This must only be called in test cases where a new store instances cannot be explicitly created. Also add a `nil` check to the reset function for the BoltDB shipper singleton instance. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Callum Styan	da6fd01444	[docs] document logfmt pipeline stage (#6238) ... I noticed earlier while writing a design doc that the `logfmt` pipeline stage is not documented. It was originally added here: https://github.com/grafana/loki/pull/4346 I've also fixed a minor typo in the json pipeline stage docs. Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Ed Welch <ed@oqqer.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Trevor Whitney	c83b6c543d	Helm: Add helm ci (#6988) ... Tests Helm chart when changed as part of PR CI process.	3 years ago
Dylan Guedes	c6dc0d2649	Docs: Fix default max_chunk_age (#6996) ... **What this PR does / why we need it**: Fix a doc stating a wrong `max_chunk_age` value. **Which issue(s) this PR fixes**: Fixes #6919	3 years ago
Vladyslav Diachenko	615941234a	Logql Analyzer (#6171) ... * created logql-debugger page and API endpoint that returns information for each stage of the query * added docker-compose setup for local testing * refactored analyzer and added unit tests * removed docs page to release it in a separate PR when back-end API is published * changes structure of results object in response to return origin log line needed for case when only stream selector is sent in query * merged the latest main and resolved conflicts * added endpoint for readiness probe and configured CI * fixed drone.yml discrepancy * fixed method signature * fixed path to Dockerfile * added clean step	3 years ago
Owen Diehl	9e84648f3e	querier deployment rollingupdate works as a percentage to allow faste… (#6990) ... …r rollouts in large environments	3 years ago
Trevor Whitney	cffcb468f2	Remove deprecated helm chart source code (#6978) ... **What this PR does / why we need it**: All of these helm charts have been moved out of this repo to `grafana/helm-charts`. As we are now moving `grafana/loki` back into this repo, this deprecated charts are confusing. I have opted to include `README`s with the deprecation notice, as well as a link to the current sha of `main` so that people can easily find the old source code.	3 years ago
Ed Welch	bf5a25dd5b	Docs: update `split_queries_by_interval` documentation to have the correct explanation for how to disable. (#6715) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: I noticed when responding to #5123 the docs did not correctly explain how to disable splitting queries by time. I searched through the code and confirmed `0` is the correct value to disable this feature **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Ed Welch	6d54505485	Loki: add `__line__` and `__timestamp__` to label_format (#6983) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: We have the `__line__` and `__timestamp__` functions currently available in the `line_format` stage but not in the `label_format` stage. This PR fixes that. Specifically `__timestamp__` can be valuable in a label because you can do things like ``` label_format hour_of_day=`{{__timestamp__ | date "15"}}` ``` To be able to conditionaly select log lines based on the hour of the day they were created, this can be useful for conditional alerting as an example. **Which issue(s) this PR fixes**: Fixes #6402 **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [x] Documentation added - [ ] Tests updated - [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Callum Styan	4a21923913	update to alpine:3.6.2 (#6979) ... Updates the alpine image version we use to 3.16.2 and will also build/publish a new loki build image Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Juha Tiensyrjä	073c62051b	lambda-promtail: Add support for Kinesis data stream events (#5977) ... **What this PR does / why we need it**: This PR adds support for sending Kinesis data stream events to lambda-promtail. One use case would be e.g. to send CloudFront realtime logs to Loki. **Which issue(s) this PR fixes**: Fixes #5978 **Special notes for your reviewer**: n/a **Checklist** - [x] Documentation added - [x] Tests updated - [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
李国忠	e3cd312c37	[fix panic] s3: fix object nil point (#6970) ... **What this PR does / why we need it**: fix a s3 panic. ```go panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x14fec7b] goroutine 6271245 [running]: github.com/grafana/loki/pkg/storage/chunk/client.(*client).getChunk(_, {_, _}, _, {{0xb130cf997b272f99, {0xc05077e6a0, 0x1b}, 0x182d1aefb2d, 0x182d1b47984, 0x9fab6388}, ...}) /loki@v1.6.2-0.20220816085209-65ed7b60fbd4/pkg/storage/chunk/client/object_client.go:171 +0x2fb ```	3 years ago