apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Garrett Guillotte	7eadddbcfc	Docs: Populate empty index pages (#7200 ) Three docs index pages are dead ends - they are either blank or have no links. In lieu of more detailed content, this PR populates these pages with `{{< section >}}` shortcodes that automatically generate TOC-like links to subpages.	4 years ago
Eng Zer Jun	e2842c69c5	Replace deprecated `io/ioutil` with `io` and `os` packages (#7212 ) The `io/ioutil` package has been deprecated in Go 1.16 (See https://pkg.go.dev/io/ioutil). This PR replaces the existing `io/ioutil` functions with their new definitions in `io` and `os` packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	4 years ago
李国忠	557fdf2206	[bugfix] logql: vectorExpr run fail when query_range: parallelise_shardable_queries: true (#7045 ) …hardable_queries: true <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: A grafana error occurred when I deployed the vector(0) function to our online loki cluster. It was finally found because online loki has a configuration `unexpected expr type (syntax.VectorExpr) for ASTMapper type (logql.ShardMapper) ` ```yaml query_range: parallelise_shardable_queries: true ``` Which issue(s) this PR fixes: Fixes # Special notes for your reviewer: error snapssnapshot ![image](https://user-images.githubusercontent.com/9583245/188357957-e573e589-a789-4e5e-a1c3-b91c019ceb62.png) with this pr，success snapshot ![image](https://user-images.githubusercontent.com/9583245/188358036-5ca0d9be-c2b6-4516-807c-c027445ee62d.png) <!-- Note about CHANGELOG entries, if a change adds: an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Sandeep Sukhani	2af3d680cb	tsdb supports data deletion, this PR takes care of todo to enable delete request client for tsdb (#7223 ) What this PR does / why we need it: `tsdb` index store supports deletion of data, but we had missed enabling the creation of delete requests client, which is required for query time filtering of data requested for deletion. This PR takes care of it. I have also removed some unused code and adjusted the tests accordingly. Checklist - [x] Tests updated	4 years ago
Danny Kopping	16a7dd19a2	Jsonnet: fix overrides exporter to use correct CPU limits (#7221 ) Currently the overrides exporter uses 200 vCPUs due to a missing `m` unit indicating "milli-CPUs".	4 years ago
Alexandru Gologan	5e134f3b6d	Helm: add support for configuring ruler (#7196 ) What this PR does / why we need it: Enables configuring various `ruler` options like alertmanager url. `rulerConfig` is added second in the template so users can override the `storage` config if necessary. Have seen questions in community Slack asking how to get filesystem support and alertmanager back when migrating from the old loki helm which should now be possible.	4 years ago
Alexandru Gologan	f4967c46da	Helm: expose Prometheus-compatible endpoints in gateway (#7195 ) What this PR does / why we need it: The gateway in non-enterprise mode doesn't expose Prometheus endpoint for alerts/rules so Grafana fails to query and display them in alerting section. Note: When comparing nginxConfig enterprise vs non-enterprise there are a couple of other endpoints which are missing from the gateway config.	4 years ago
Trevor Whitney	afd63c5981	[helm] Add Loki Canary to Helm Chart (#7173 ) What this PR does / why we need it: This PR adds the Loki canary as well as the GEL provisioner to the Loki helm chart. The Loki canary expands the self-monitoring functionality of the Helm chart, giving operators insight beyond whether Loki is just running, but also if it's functioning properly. The GEL provisioner was necessary for the canary to work when the enterprise option (`enterprise.enabled = true`) is enabled. Since enabling enterprise in the helm chart also enables auth/multi-tenancy, the addition of the provisioner allows the helm chart to automatically create a tenant for the canary via the GEL admin API, and to create applicable access policies and tokens. Furthermore, additional tenants can also be specified, which will be provisioned, creating a read and write token for each that will be stored in k8s secrets. Which issue(s) this PR fixes: Fixes #7019 Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz> Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	4 years ago
Jarjar	9265b8eef5	Helm: Fix ExternalLicence (#7189 ) What this PR does / why we need it: Update references from: ``` .Values.useExternalLicense .Values.externalLicenseName ``` to ``` .Values.enterprise.useExternalLicense .Values.enterprise.externalLicenseName ``` Which issue(s) this PR fixes: Fixes #7171 Signed-off-by: CAJNA Jarod <contact@jarodcajna.fr>	4 years ago
gerhard-strauss-jambit	f29a03833e	Helm: Make automountServiceAccountToken configurable (#7114 ) What this PR does / why we need it: just a simple modifikation to set automountServiceAccountToken on Container Level. ServiceAccount already has this setting	4 years ago
Sandeep Sukhani	7f298ff72f	avoid using bloomfilters for chunks in stats calls by avoiding duplicates (#7209 ) What this PR does / why we need it: Avoid using bloomfilters for chunks deduplication in tsdb `Stats` calls by avoiding fetching duplicate entries. The idea is to split and align queries by [ObjectStorageIndexRequiredPeriod](`61794710a7/pkg/storage/config/schema_config.go (L47)`) and make each split process chunks with a start time >= start time of the table interval. In other terms, table interval that contains start time of the chunk, owns it. For e.g. if the table interval is 10s, and we have chunks 5-7, 8-12, 11-13. Query with range 6-15 would be split into 6-10, 10-15. query1 would process chunks 5-7, 8-12 and query2 would process chunks 11-13. This check is not applied for the first split so that we do not eliminate any chunks that overlaps the original query intervals but starts at the previous table. For e.g. if the table interval is 10s, and we have chunks 5-7, 8-13, 14-13. Query with range 11-12 should process chunk 8-13 even though its start time <= start time of table we will query for index. The caveat here is that we will overestimate the data we will be processing if the index is not compacted yet since it could have duplicate chunks when RF > 1. I think it is okay since the Stats call is just an estimation and need not be accurate. Removing all the extra processing saves us quite a bit of CPU and memory, as seen from the benchmark comparison between the two implementations: ``` name old time/op new time/op delta IndexClient_Stats-10 187µs ± 0% 34µs ± 1% -82.00% (p=0.008 n=5+5) name old alloc/op new alloc/op delta IndexClient_Stats-10 61.5kB ± 4% 12.5kB ± 2% -79.69% (p=0.008 n=5+5) name old allocs/op new allocs/op delta IndexClient_Stats-10 1.46k ± 0% 0.48k ± 0% -67.28% (p=0.008 n=5+5) ``` Checklist - [x] Tests updated	4 years ago
Owen Diehl	bac52165d3	highly sharded subqueries are ellided in expr.String (#7203 ) This is mostly a quality of life improvement. Highly sharded subqueries can be burdensome to view and they're logged on every request. This PR will only display the first four subqueries in a `Concat{Log,Sample}Expr`, which is easier on the eyes when viewing logs and is more efficient.	4 years ago
Sandeep Sukhani	4e7105d018	add index requests panel to reads/writes dashboard (#7163 ) What this PR does / why we need it: In PR #7154, I refactored the code also to measure the latency on index write requests. This PR adds the index requests panel to the Loki Reads/Writes dashboard.	4 years ago
Enzo Nocera	4f9c85df5d	Guidelines: Update CONTRIBUTING.md (#6757 ) For a new contributor (which I am 🎉), the `CONTRIBUTING.md` file might seem a bit vague in regard to why we need to use the `GOPATH`-scoped Loki repository to contribute. The rationale is now given (If I've missed any other reasons, please tell me). * Fixed wrong level title, * Added information about local workflow when dealing with locally modified dependencies, * Added references to Go 1.18 workspaces. Signed-off-by: Raskyld <enzo@nocera.eu>	4 years ago
Christian Haudum	67b12f2357	Update distributor push tests (#6641 ) Adds a test case for concurrent pushes (which would have spotted a bug in https://github.com/grafana/loki/pull/5248) and a test case for tolerance regarding replication factor. The PR also replaces the mock ring with an actual in-memory kv store ring. And variables and function names were changed to meet Loki/logs terminology. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Ryan Garcia	d8eb8b2a9f	Helm: Adding tolerations value support for tokengen job (#7122 ) What this PR does / why we need it: Adding toleration support for the enterprise tokengen job template. Which issue(s) this PR fixes: Fixes #7091 Signed-off-by: garcia.ryan <garcia.ryan@solute.us>	4 years ago
Roelof Blom	a1d77bf5ea	[helm] Storage: Read support for Azure Object Storage (#7162 ) What this PR does / why we need it: Using Azure Object Storage worked before Helm chart version 3.0. Which issue(s) this PR fixes: Fixes #7090	4 years ago
Danny Kopping	72aec6a38b	Document validation & rate-limit errors (#7145 ) This PR adds documentation for the various validation & rate-limit errors that can occur when sending logs to Loki.	4 years ago
Robert Jacob	f1fbb7bb3f	operator: Write configuration for per-tenant retention (#7201 )	4 years ago
nniehoff	a5c1557d03	docs: Change AWS EKS example eventrouter rbac resources from v1beta1 apis to v1 (#6483 ) RBAC Resource apiVersion `rbac.authorization.k8s.io/v1beta1` was deprecated with Kubernetes 1.22 (see the [deprecation notice](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#rbac-resources-v122)). Now that EKS supports 1.22 the docs need to be updated.	4 years ago
Jeremy Wu	161fcac646	support custom label selector in kubernetes_sd_configs (#5920 ) make promtail scrape logs of pod with specific field or label selector, so that it will watch less logs and make fewer calculations and get a better performance in some scenarios. Signed-off-by: Zhimin Wu <hoozecn@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Ashwanth	3245625c40	ingester_querier: early exit queries on reaching quorum (#7172 ) Currently `forGivenIngesters` and its derivatives don't allow passing down context to the work function as a result of which we are not making use of the tracking functionality provided by `replicationSet.Do` to exit early and relieve pressure on the ingesters. This change ensures that Unary rpc calls receive ctx populated by the replication set tracker to exit early on reaching quorum or max failures. The same cannot be applied to stream rpc calls given how the code is structured at the moment where we are processing the streams outside of the work func	4 years ago
Trevor Whitney	8d552878cd	Bring addition of server config into existing style (#7174 ) What this PR does / why we need it: A recent PR (https://github.com/grafana/loki/pull/7137) was merged to make the server settings more configurable in the helm chart. This PR fixes a few style nits with that PR, mainly: * Name the config block the same as the header in the actual loki config file (in this case, `server` not `serverConfig`) * Use `with`, and wrap around the header in the config file so the entire section is omitted if empty, rather than setting an empty object under the header.	4 years ago
Jennifer Villa	9467a341f6	Update README.md (#7190 ) Typo fix on helm chart readme	4 years ago
Jennifer Villa	4310ffe0bd	Update README.md (#7191 ) Fix typo on README	4 years ago
Dylan Guedes	bf1b5a5ced	Project: Change PR template and CONTRIBUTING guide (#7116 ) What this PR does / why we need it: - Move contributing instructions from the PR template to the CONTRIBUTING guide - Rewrite the contributing guide - Make the PR template slim by getting rid of sections that are optional but often left blank by most contributors/maintainers - We're making the PR template slim as we're using the merged PR body as the merge commit message. Because the comments are often left there by contributors, they're prone to be merged by maintainers, hence making our commit log history dirty with a lot of undesired comments.	4 years ago
Salva Corts	df4461d712	Evenly spread memcached across nodes (#7183 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: This PR enables the [TopologySpreadConstraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/) for Memcached chunks cache introduced at https://github.com/grafana/jsonnet-libs/pull/832. Now the chunks cache will be evenly spread across nodes but more than one cache can run on the same node. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Karsten Jeschkies	a1dce32d9a	Abide to Mixin linting rules. (#7010 ) Cloud integrations need to pass linting rules such as proper dashboard names. It should be `Data Source` not `datasource`. See also grafana/cloud-onboarding#2228	4 years ago
Owen Diehl	12ebd33f94	TSDB indexBuckets helper logs instead of errors (#7187 ) indexBuckets will return the list of overlapping TSDB buckets for a time range. Historically, this errored when there was no overlap, which is generally impossible. While mutating period configurations as a test (which should never be done in production), I realized this would preemptively error us when there were old TSDB WALs from a bucket that no longer belonged to a TSDB period (i.e. by replacing a `tsdb` index with a `boltdb-shipper` one). This PR changes this to a log line instead as we don't need to error and stop computation; it's better to return an empty overlap instead. Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	4 years ago
MiggL	1e22b14430	Helm: Bump helm chart patch version (#7184 ) What this PR does / why we need it: Bumps the helm chart version to trigger a release for #7182	4 years ago
Owen Diehl	f1114fd221	[Bugfix] TSDB index bucket calculation (#7178 ) When calculating the index buckets a chunk belongs to, we shouldn't error when a bucket falls outside the current TSDB store's periodConfig range. This can happen when a chunk is written across the bounds between a previous index type (i.e. boltdb-shipper) and a new tsdb type or vice versa. Instead of erroring when this happens, we'll now correctly return only the buckets matching the TSDB store (the other buckets are handled by their respective stores). I think we can also remove the error from this function signature, although I left it in as it's conceivably helpful in the case of a chunk range that doesn't overlap at all with the store (which shouldn't happen). Dependent on https://github.com/grafana/loki/pull/7177	4 years ago
Owen Diehl	85e8f00958	fixes azure client secret duplicate CLI registration (#7185 ) Fixes duplicate key registration from https://github.com/grafana/loki/pull/7179/files#diff-665c7ce7ef736bdf9f9da76a8e389d5b6fcdc90099db913ce36b4c0da2a97ef2R121-R122	4 years ago
MiggL	62f0596afe	Helm: Fix limits_config key in helm values file (#7182 )	4 years ago
Owen Diehl	ab24efe1bf	allow backup store metrics registration by including backend type in labels (#7177 ) The refactor [here](https://github.com/grafana/loki/blob/main/pkg/storage/store.go#L237) introduced in https://github.com/grafana/loki/pull/7154 will cause a panic due to duplicate registration via the same store prefix when using a backup index store (for use during tsdb migrations). This PR adds more granularity to the label value to differentiate these and avoid the problem.	4 years ago
Vladyslav Diachenko	7f629c8849	[loki] Azure auth using Service Principal (#7179 ) This PR adds the ability to use Service Principal credentials to authenticate through Azure OAuth. It might be needed when we want to access Azure Blob storage from another tenant. Previously we could use only StorageAccount key that can not be used cross-tenant.	4 years ago
Owen Diehl	2766e0a8c7	fixes nil interface value with non-nil type in cache construction (#7175 ) Fixes a bug introduced in https://github.com/grafana/loki/pull/7013 This will panic when the interface's _value_ is nil because it's been assigned a concrete type already and then passed as an interface. Instead, we treat it as a nil interface with no type bound to it: ``` panic: runtime error: invalid memory address or nil pointer dereference goroutine 40570 [running]: github.com/grafana/loki/pkg/util/server.onPanic({0x2209320, 0x3dfae20}) /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/util/server/recovery.go:41 +0x53 github.com/grafana/loki/pkg/util/server.glob..func1.1.1() /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/util/server/recovery.go:29 +0x3a panic({0x2209320, 0x3dfae20}) /usr/local/go/src/runtime/panic.go:838 +0x207 github.com/grafana/loki/pkg/storage/stores/indexshipper/compactor/generationnumber.(GenNumberLoader).getCacheGenNumber(0x0, {0xc0081e14d7, 0x2}) /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/storage/stores/indexshipper/compactor/generationnumber/gennumber_loader.go:120 +0x60 github.com/grafana/loki/pkg/storage/stores/indexshipper/compactor/generationnumber.(GenNumberLoader).getCacheGenNumbersPerTenants(0x2?, {0xc000e8c4d0?, 0x1, 0x0?}) /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/storage/stores/indexshipper/compactor/generationnumber/gennumber_loader.go:98 +0xe7 github.com/grafana/loki/pkg/storage/stores/indexshipper/compactor/generationnumber.(*GenNumberLoader).GetResultsCacheGenNumber(0x2b26298?, {0xc000e8c4d0?, 0xc0050ad080?, 0x2dcdeeb9bcc040e4?}) ```	4 years ago
Trevor Whitney	4b6df86333	Add local k3d environment for enterprise-logs (#7146 ) What this PR does / why we need it: This adds a local k3d based development cluster for testing out the enterprise logs side of the helm chart, which can be enabled by setting: ``` enterprise: enabled: true ``` It requires both the `gex-plugins` and `enterprise-logs` repos to be checked out locally (though the latter soon will not be needed once the provisioner is published). We need this environment when developing enterprise features for the helm chart.	4 years ago
Jarjar	41861bdacb	Helm: Allows loki s3 http_config block to be configured (#7041 ) What this PR does / why we need it: This will allow users of the chart to configure the s3 http_config block as documented here [https://grafana.com/docs/loki/latest/configuration/#s3_storage_config](https://grafana.com/docs/loki/latest/configuration/#s3_storage_config) through values. Which issue(s) this PR fixes: Fixes grafana/helm-charts#1762 Checklist - [x] Documentation added - [x] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: CAJNA Jarod <contact@jarodcajna.fr>	4 years ago
Danny Rehelis	180a93e430	Helm: Make server configuration configurable (#7137 ) What this PR does / why we need it: Expose server configuration to be configurable with Helm values Checklist - [x] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz>	4 years ago
Mohamed-Amine Bouqsimi	c9ea27532f	operator: Fix TLSProfile not working on vanilla k8s (#7167 )	4 years ago
Trevor Whitney	d972e77ae2	Expose limits and compactor config via helm values (#7108 ) What this PR does / why we need it: Exposes the compactor and limits config in the helm chart via helm values. Replaces https://github.com/grafana/loki/pull/7055 with some minor stylistic changes, and moving default limits into the new `limits_config` value.	4 years ago
Trevor Whitney	35003f3da0	Fix minio values, reduce replicas, and fix tokengen job (#7147 ) What this PR does / why we need it: This PR fixes a few small things with the helm chart: * After upgrading to the new minio chart, the subvalues for that chart were not correctly changed to the new values for `rootUser` and `rootPassword` * The default replica count for the new minio chart is 16, which seems a little much for our needs. * The `grafana-token` created by the tokengen job was meant to be used when configuring the GEL plugin, but it was not working, so I removed it.	4 years ago
Trevor Whitney	a6101f47ac	Update GEL helm deployment example (#7144 ) What this PR does / why we need it: This update the example for deploying a GEL cluster using the new helm chart. The previous example was a bit out of date.	4 years ago
Chris Hodges	d1f42fff50	promtail: Fix issue with dropping logs when a file based SD target's labels are updated (#7016 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: Allows for modifying labels for a target and continuing to scrape logs afterwards. Which issue(s) this PR fixes: Fixes #7001 Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [x] Tests updated - [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Travis Patterson	071118a090	revert stream push error response (#7160 ) We've changed approaches to how we want to do per-stream rate limiting. The new approach will rely on creating an api on the ingester so the distributors can decide how streams should be sharded. As a result, the work to return rate-limited streams in the error body is no longer needed.	4 years ago
Robert Jacob	a95e900a76	operator: Skip enforcing matcher for certain tenants on OpenShift (#7037 )	4 years ago
Robert Jacob	d21f0d3725	operator: Manage global stream-based retention (#7106 )	4 years ago
Sandeep Sukhani	c8c8303a6a	refactor index storage code to index reader/writer interfaces for better monitoring (#7154 ) What this PR does / why we need it: Currently, it is hard to monitor just the index latencies since the chunks PUT code is coupled with both object storage and index writer. This PR refactors the code to have separate interfaces defined for index readers/writers. I have also added a wrapper called `monitoredReaderWriter`, which measures the latency on the index operations. Special notes for your reviewer: Cortex had added `PutChunksAndIndex` for DynamoDB, which would write both chunks and index in a single request when configured for storing both. I have removed the use of it since it would be too complex to support it with this refactoring. Loki, anyways does not recommend using chunks in the index store, and using a hosted object storage would be much cheaper. Also, this is not a breaking change, it might somewhat impact the performance of flushing the chunks. Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Ashwanth	88503ae2e1	tsdb: clean-up metrics (#7150 ) What this PR does / why we need it: clean-up unused tsdb metrics and refactor others to use a `CounterVec` Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	4 years ago
李国忠	a290549a59	Promtail: Add support for max stream limit (#7101 ) What this PR does / why we need it: Add Promtail support to a new feature/configuration named `max_stream_limit` * This new feature limits the max number of streams Promtail can materialize. The new feature is also useful to avoid OOMing Promtail Which issue(s) this PR fixes: Fixes #6968	4 years ago

1 2 3 4 5 ...

3969 Commits (7eadddbcfcc23bd345ae130630e4e0554a032d0d) All Branches Search

3969 Commits (7eadddbcfcc23bd345ae130630e4e0554a032d0d)

All Branches