apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Sandeep Sukhani	1fcefe8efa	index-gateway: fix failure in initializing index gateway when boltdb-shipper is not being used (#8755 ) What this PR does / why we need it: When only `tsdb` index is being used in the schema without configuring `boltdb-shipper` index store, we fail to initialize `index-gateway` since it also tries to build `boltdb-shipper` index client. This reference is used for serving `QueryIndex` calls which are for serving index from boltdb files. This PR fixes the issue by adding a check for the existence of `boltdb-shipper` in the schema config before initializing an index client. On the index gateway side, it adds a `failingIndexClient`, which errors when someone tries to query it. Checklist - [ ] `CHANGELOG.md` updated --------- Co-authored-by: Travis Patterson <masslessparticle@gmail.com> Co-authored-by: Dylan Guedes <djmgguedes@gmail.com> Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	3 years ago
Ed Welch	7264e0ef53	Loki: Remove global singleton of the tsdb.Store and keep it scoped to the storage.Store (#8928 ) What this PR does / why we need it: I made this change to fix an issue with the cmd/migrate tool which I'm also sneaking a few fixes on in this PR too. We had a global singleton of the tsdb.Store which was leading to the migrate tool reading from and writing to the `source` storage.Store because it created the singleton with an object store client only pointing to the source store. This PR changes the scoping of that singleton to not be global but instead local to each instance of storage.Store Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Sandeep Sukhani	69d6ee293e	tsdb: use unique uploader name while building tsdb index filename, same as boltdb-shipper (#8975 ) What this PR does / why we need it: While building file names from `boltdb-shipper` and `tsdb`, we add the name of the uploader to it. To avoid collisions in filenames when running multiple ingesters with the same name, we also add a nanosecond precise timestamp to the ingester name, which is persistent as the uploader name in the active index directory. This is not being done for the tsdb index built from ingesters which can cause collisions in filenames when running multiple ingesters with the same name. I have moved the code to manage the uploader name to the index shipper config and used the same to share it with tsdb.	3 years ago
Dylan Guedes	9159c1dac3	Loki: Improve spans usage (#8927 ) What this PR does / why we need it: - At different places, inherit the span/spanlogger from the given context instead of instantiating a new one from scratch, which fix spans being orphaned on a read/write operation. - At different places, turn spans into events. Events are lighter than spans and by having fewer spans in the trace, trace visualization will be cleaner without losing any details. - Adds new spans/events to places that might be a bottleneck for our writes/reads.	3 years ago
Danny Kopping	de84737368	Fix fetched chunk from store size in metric (#8971 ) What this PR does / why we need it: The `Size()` function on `chunk.Chunk` calls out to `func (ChunkRef) Size() int` in the proto, which doesn't actually include the size of the chunk data! We need to call `chunk.Data.Size()` for that.	3 years ago
Suruthi-G-K	e7b0d4d26e	feat(cos): Support authentication with trusted profiles (#8939 ) What this PR does / why we need it: Add trusted profile authentication in COS client Which issue(s) this PR fixes: Fixes NA Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: shahulsonhal <shahulsonhal@gmail.com> Co-authored-by: tareqmamari <tariq.mamari@de.ibm.com>	3 years ago
Periklis Tsirakidis	1bcf683513	Expose optional label matcher for label values handler (#8824 )	3 years ago
Athira Varghese	ffb961c439	feat(storage): add support for IBM cloud object storage as storage client (#8826 ) What this PR does / why we need it: Add support for IBM cloud object storage as storage client Which issue(s) this PR fixes: Fixes NA Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated --------- Signed-off-by: Shahul <shahulsonhal@gmail.com> Co-authored-by: Aman Kumar Singh <amankrsingh2110@gmail.com> Co-authored-by: Suruthi-G-K <shruthi.suruthi@gmail.com> Co-authored-by: tareqmamari <tariq.mamari@de.ibm.com> Co-authored-by: shahulsonhal <shahulsonhal@gmail.com> Co-authored-by: Aditya C S <aditya.gnu@gmail.com> Co-authored-by: Tareq Al-Maamari <tariq.mamari@gmail.com>	3 years ago
André Águas	44f1d8d7f6	azure: respect retry config before cancelling the context (#8732 ) What this PR does / why we need it: On GET blob operations a context timeout is set to `RequestTimeout`, however this value is not the full timeout. This is only the first timeout and afterwards there are retries. As a result, the retry configuration is never used because the context is immediately cancelled. To fix we should set the context timeout to a value larger than the Azure timeout after retries. Since the backoff is exponential and randomized we need to take the value of `MaxRetryDelay` to set an upper bound.	3 years ago
Sandeep Sukhani	4e893a0a88	tsdb: sample chunk info from tsdb index to limit the amount of chunkrefs we read from index (#8742 ) What this PR does / why we need it: Previously we used to read the info of all the chunks from the index and then filter it out in a layer above within the tsdb code. This wastes a lot of resources when there are too many chunks in the index, but we just need a few of them based on the query range. Before jumping into how and why I went with chunk sampling, here are some points to consider: * Chunks in the index are sorted by the start time of the chunk. Since this does not tell us much about the end time of the chunks, we can only skip chunks that start after the end time of the query, which still would make us process lots of chunks when the query touches chunks that are near the end of the table boundary. * Data is written to tsdb with variable length encoding. This means we can't skip/jump chunks since each chunk info might vary in the number of bytes we write. Here is how I have implemented the sampling approach: * Chunks are sampled considering their end times from the index and stored in memory. * Here is how `chunkSample` is defined: ``` type chunkSample struct { largestMaxt int64 // holds largest chunk end time we have seen so far. In other words all the earlier chunks have maxt <= largestMaxt idx int // index of the chunk in the list which helps with determining position of sampled chunk offset int // offset is relative to beginning chunk info block i.e after series labels info and chunk count etc prevChunkMaxt int64 // chunk times are stored as deltas. This is used for calculating mint of sampled chunk } ``` * When a query comes in, we will find `chunkSample`, which has the largest "largestMaxt" that is less than the given query start time. In other words, find a chunk sample which skips all/most of the chunks that end before the query start time. * Once we have found a chunk sample which skips all/most of the chunks that end before the query start, we will sequentially go through chunks and consider only the once that overlap with the query range. We will stop processing chunks as soon as we see a chunk that starts after the end time of the query since the chunks are sorted by start time. * Sampling of chunks is done lazily for only the series that are queried, so we do not waste any resources on sampling series that are not queried. * To avoid sampling too many chunks, I am sampling chunks at `1h` steps i.e given a sampled chunk with chunk end time `t`, the next chunk would be sampled with end time >= `t + 1h`. This means typically, we should have ~28 chunks sampled for each series queried from each index file, considering 2h default chunk length and chunks overlapping multiple tables. Here are the benchmark results showing the difference it makes: ``` benchmark old ns/op new ns/op delta BenchmarkTSDBIndex_GetChunkRefs-10 12420741 4764309 -61.64% BenchmarkTSDBIndex_GetChunkRefs-10 12412014 4794156 -61.37% BenchmarkTSDBIndex_GetChunkRefs-10 12382716 4748571 -61.65% BenchmarkTSDBIndex_GetChunkRefs-10 12391397 4691054 -62.14% BenchmarkTSDBIndex_GetChunkRefs-10 12272200 5023567 -59.07% benchmark old allocs new allocs delta BenchmarkTSDBIndex_GetChunkRefs-10 345653 40 -99.99% BenchmarkTSDBIndex_GetChunkRefs-10 345653 40 -99.99% BenchmarkTSDBIndex_GetChunkRefs-10 345653 40 -99.99% BenchmarkTSDBIndex_GetChunkRefs-10 345653 40 -99.99% BenchmarkTSDBIndex_GetChunkRefs-10 345653 40 -99.99% benchmark old bytes new bytes delta BenchmarkTSDBIndex_GetChunkRefs-10 27286536 6398855 -76.55% BenchmarkTSDBIndex_GetChunkRefs-10 27286571 6399276 -76.55% BenchmarkTSDBIndex_GetChunkRefs-10 27286566 6400699 -76.54% BenchmarkTSDBIndex_GetChunkRefs-10 27286561 6399158 -76.55% BenchmarkTSDBIndex_GetChunkRefs-10 27286580 6399643 -76.55% ``` Checklist - [x] Tests updated	3 years ago
W.T. Chang	183fe85f1c	Loki: Add route_randomly to Redis options (#8852 ) What this PR does / why we need it: Current Redis client only reads from master node, which makes master node under heavy pressure. Usually, one or more read replicas are expected on production environments. This PR helps to increase the utilization of read replicas.	3 years ago
Danny Kopping	2d93952929	(chore): remove incorrect cache config preample (#8859 ) With the addition of the [config generation tool](https://github.com/grafana/loki/pull/7916), a comment prefix was added for all cache types: `Cache config for index entry writing.` https://grafana.com/docs/loki/next/configuration/#cache_config This appears in the description of each cache config option, and is misleading.	3 years ago
Travis Patterson	5a08a6bcb9	Label selector optimizations (#8763 ) This PR introduces the following label selector optimizations: - any selector that is `=~<literal>` is converted to an `Equals/NotEquals` matcher - Use the greedy->non-greedy regex improvements that exist in line and label filters - Never run `.*` label matchers. Prometheus regex matchers anchor the match to the beginning and end of the line. That means matchers of the form `=~<literal_value>` are just testing equality. Benchmarks -- this is just a quick bench to demonstrate equality matchers are faster than their equivalent regex matchers: ``` BenchmarkMatcherTypes/regex-8 27694834 53.74 ns/op 0 B/op 0 allocs/op BenchmarkMatcherTypes/equals-8 627045850 2.153 ns/op 0 B/op 0 allocs/op ```	3 years ago
Karsten Jeschkies	f5f1753851	Print duration in error messages with more readable. (#8816 ) What this PR does / why we need it: The old error messages would print only up to hours. E.g. `169h30s`. This change will print it as `7d1h30s`. See [model.Duration](`66b493f42b/model/time.go (L259-L290)`) for details. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
René Scheibe	76c2e910cb	chore: Fix spelling of "syncing" (#8829 ) This makes it consistent in the repository.	3 years ago
Trevor Whitney	b46e392adb	gofmt is needed for the build (#8678 ) What this PR does / why we need it: `gofmt` is needed for the build, `make format` fails without it.	3 years ago
Ed Welch	eadb9d2c00	Loki: Change the behavior of the 0 value for retention_period to disable retention. (#8753 ) What this PR does / why we need it: Issue #8359 exposed a very unfortunate inconsistency in a Loki config where the zero value did not disable a feature. By convention in Loki the zero value for a config typically "disables" it. This PR changes the retention_period to honor the zero value of `0` or `0s` to disable retention. Which issue(s) this PR fixes: Fixes #8359 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Sandeep Sukhani	162a2d0057	deletion: fix issue in processing delete requests with tsdb index (#8665 ) What this PR does / why we need it: When a delete request creates a new chunk from an existing chunk, we index it to the index store. In TSDB, we [drop](`944b488bd2/pkg/storage/stores/tsdb/head_manager.go (L235-L237)`) `__name__` label, which was being added to chunkrefs for historical reasons. However, this was not done while indexing chunks from the deletion code, causing deletion to fail with the error [chunk insertion is only allowed on existing streams](`619c2212f5/pkg/storage/stores/tsdb/builder.go (L67)`). This PR handles the issue by dropping the `__name__` label and adding relevant tests. Special notes for your reviewer: I will add an integration test for deletion with TSDB once we have PR https://github.com/grafana/loki/pull/7754 merged since it does some groundwork to run integration tests with both `boltdb-shipper` and `tsdb`. Checklist - [x] Tests updated - [x] `CHANGELOG.md` updated	3 years ago
Dylan Guedes	f9a1f2d94e	Loki: Stop running tickers on deletes (#8697 ) What this PR does / why we need it: Calls `ticker.Stop()` to avoid leaking ticker resources for `GenNumberLoader` and `deleteRequestsClient`.	3 years ago
Callum Styan	9a2a038f43	Allow passing of context to query related limits functions (#8689 ) In this PR we're allowing for passing of a `context.Context` via the Limits interfaces (some of which are new, to clean up hardcoding/embedding of `validation.Overrides`) This is based on work/ideas by @jeschkies . Fixes #8694 --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>	3 years ago
Danny Kopping	ab7a970b94	Querier/Ruler: add histogram to track fetched chunk size distribution (#8682 ) What this PR does / why we need it: We are looking at making some changes to the chunk caching strategy, and we need this data to know what size chunks we're typically requesting. For example, if the overwhelming majority of chunks are small we may decide to cache only small chunks to decrease the number of requests to the object store; we could fit more chunks in cache this way, which may have a positive performance impact.	3 years ago
Jan Jansen	84c5e20af1	S3: add support for s3 session token (#8474 ) We want to automate the access account rotation. This is that easy at moment with service accounts in MinIO. Our currently life time of tokens is 3 months with MinIO STS works this fine. Signed-off-by: Jan Jansen <jan.jansen@gdata.de>	3 years ago
justcompile	ea55f285b6	S3 Storage Class Configuration (#8532 )	3 years ago
brettlounsbury	4c227bd268	DynamoDB: Accept a KMS key for DynamoDB tables (#8469 ) Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	3 years ago
李国忠	1fc00fa02d	[new] chunk backend: Integrate Alibaba Cloud oss (#8410 )	3 years ago
janhorstmann	0e81d78914	Swift: Allow to configure swift with the internal endpoint (#8438 )	3 years ago
Etran-H	5447541e2e	Fix open "/tmp/*" the system cannot find the path specified on windows (#8170 ) Instead of hard-coding `/tmp` directory, use the OS default directory for temporary files	3 years ago
Owen Diehl	c638d924af	Update index metrics (#8316 )	3 years ago
Travis Patterson	66f7407f15	Ensure a delete request interval of at least a second (#8340 )	3 years ago
Sandeep Sukhani	b2d0481cbe	deletion: fix log deletion with line filters (#8151 )	3 years ago
Owen Diehl	944b488bd2	logs more metadata when failing to recover head from WAL in TSDB (#8250 )	3 years ago
Sandeep Sukhani	96a4fc1409	index-store: fix indexing of chunks overlapping multiple schemas (#8251 )	3 years ago
Bryan Boreham	d2281a7b6e	Benchmark_store_SelectSample: stop accumulating results (#8242 )	3 years ago
Bryan Boreham	321ee92fd2	stores/series: always return entries to pool (#8245 )	3 years ago
Bryan Boreham	15b382294e	Remove dependency on thanos-io/thanos (#8227 ) What this PR does / why we need it: This avoids complications trying to match up versions between Prometheus and Thanos. Special notes for your reviewer: I don't know why the dependency on `github.com/efficientgo/e2e` has appeared in go.mod, but it doesn't seem to change what is in vendor. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - NA Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Ed Welch	c8760f31e2	Loki: Remove validation of matchers when performing label values lookup. (#8224 ) the curren validation fails on != style matchers when they are the only matchers present, requiring at least one `=` matcher. However for looking up label values in TSDB, by design it uses a `desired_label != ""` style query, as such it should be allowed to also include these as matchers. Additional matchers on this Label Value queries typically only exist with LBAC headers adding additional labels. Signed-off-by: Edward Welch <edward.welch@grafana.com> What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Dimitar Dimitrov	6ea5a8e497	Update vendored dskit (#8155 ) What this PR does / why we need it: The Changelog diff between the two commits is `3e308a4944...7242706251` ``` * [CHANGE] Cache: Switch Memcached backend to use `github.com/grafana/gomemcache` repository instead of `github.com/bradfitz/gomemcache`. #248 * [ENHANCEMENT] Add cache, gate and stringsutil packages. #239 * [ENHANCEMENT] Cache: add Redis support. #245 * [ENHANCEMENT] Cache: Add the ability to use a custom memory allocator for cache results. #249 * [BUGFIX] Memberlist: fix crash when methods from `memberlist.Delegate` interface are called on `memberlist.KV` before the service is fully started. #244 ``` Checklist* - [x] Reviewed the [`CONTRIBUTING.md`](../CONTRIBUTING.md) guide (required) - N/A Documentation added - N/A Tests updated - N/A `CHANGELOG.md` updated - N/A Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Owen Diehl	07487cd89d	fixes bug with queryIngesterWithin logic in asyncStore ingester stats… (#8145 ) Fixes a previous mistake in the logic calculating when to skip querying ingesters in the async store Statistics method. Notably `through.After` should be `through.Before` when skipping querying ingesters as that's when there's no overlap with the `query-ingesters-within` period: ```go // OLD CODE BELOW if a.queryIngestersWithin != 0 { // don't query ingesters if the query does not overlap with queryIngestersWithin. if !through.After(model.Now().Add(-a.queryIngestersWithin)) { // <----- should be through.Before return a.Store.Stats(ctx, userID, from, through, matchers...) } } ``` I discovered the problem while debugging querier OOMs during a boltdb-shipper -> tsdb migration and ultimately found this happened under the following circumstances: * Queries over high volumes of _recent_ data wouldn't query ingesters for index metadata * Without index metadata, we only checked storage metadata * There is a delay before we ship the index to storage, meaning we don't see it if ingesters are skipped * Calculating ideal shard factors without recent data for queries that only touch recent data _dramatically_ underestimates the desired shard factor * Queries aren't split enough and get scheduled onto too few querier replicas * They oom. We had some fun examples like a single replica trying to download 419,000 chunks/query To be clear, this is still a hypothesis, but a plausible one, especially after finding the boolean logic error this PR fixes. Instead of changing this one line, I took the opportunity to refactor this into a shared utility used by our other `GetChunkRefs` method which is already tested, ensuring the logic works as expected. I also added some more logging visibility into this code so we can understand what the difference is when querying statistics from storage vs ingesters. Finally, I added a helper to prepare our `Stats` objects to be logged which is now used in a few places.	3 years ago
Nils	e4fbd3c949	Loki: Add support for Azure Workload Identity authentication (#7540 )	3 years ago
Sandeep Sukhani	dfff86f1cd	skip compacting tables that do not fit in start and end date of desired schema config (#8127 ) Before compacting tables, we look up the schema config for them. If we do not find a schema config for a table, we skip them. However, the schema lookup code was considering the start time of the schema and not the end time. This can cause the compactor to try compacting tables with the same prefix as one of the previous schemas, but the table interval falls into one of the subsequent schemas with a different config. This PR fixes the issue and simplifies the code by reusing existing schema lookup code. I have also updated test cases to verify the fix works.	3 years ago
Ashwanth	b3e038c7a8	config_wrapper: Add support for named stores (#7946 ) Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> What this PR does / why we need it: This PR adds support for named stores which allows users to define multiple object store configurations for a storage type. Named stores can be referred to from other sections of the config similar to how provider type is being used `i.e aws, gcs...`. Named store reference should be of the format: `<store_type>.<key_name>` Below is an example configuration with two named stores defined for aws `store-1, store-2`. Schema config is referring to one of them as `aws.store-1` ``` storage_config: aws: endpoint: s3://common-bucket region: us-east1 access_key_id: abc123 secret_access_key: def789 named_stores: aws: store-1: endpoint: s3://foo-bucket region: us-west1 access_key_id: 123abc secret_access_key: 789def store-2: endpoint: s3://bar-bucket region: us-west2 access_key_id: 456def secret_access_key: 789abc schema_config: configs: - from: "2020-07-31" index: period: 24h prefix: loki_prod_index_ object_store: aws.store-1 schema: v11 store: boltdb-shipper ``` Which issue(s) this PR fixes: Fixes #7276 Special notes for your reviewer: Configuration defined in common storage config has no effect on the named stores. Common storage config only overrides storageConfig if it's not explicitly set - same behavior as before. This should be a non-intrusive change. Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	3 years ago
Ashwanth	dbf09246d5	Avoid failing tests on gRPC server shutdown errors (#8060 )	3 years ago
Ashwanth	dc5cc4c494	store: write overlapping chunks to multiple stores (#7988 ) Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> What this PR does / why we need it: When a chunk falls on the period boundary it should be written to both the stores as it gets referred to by the indexes on both sides. But we currently skip the writes for subsequent periods if the chunk is found in the cache. This PR ensures that overlapping chunks get written to both the stores Which issue(s) this PR fixes: Fixes #7847 Special notes for your reviewer: Downside of this approach is that we end up writing the overlapping chunk multiple times, once from each instance in the replica set. But these duplicate writes would be infrequent given this occurs only on schema changes Checklist - [X] Reviewed the `CONTRIBUTING.md` guide - [X] Tests updated - [x] `CHANGELOG.md` updated Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	3 years ago
Susana Ferreira	d086c237df	Add missing period_config root block to doc generator (#7981 ) What this PR does / why we need it: Update doc generator tool to validate if slice element is a root block. Add missing `period_config` root block to doc generator tool. Special notes for your reviewer: Checklist - [ ] Documentation updated	3 years ago
Danny Kopping	a4f306399a	Add store & cache download statistics (#7982 )	3 years ago
Irina	10b869c82e	Move to fork github.com/grafana/gomemcache (#7936 )	3 years ago
Susana Ferreira	f93b91bfb5	Add configuration documentation generation tool (#7916 ) What this PR does / why we need it: Add a tool to generate configuration flags documentation based on the flags properties defined on registration on the code. This tool is based on the [Mimir doc generation tool](https://github.com/grafana/mimir/tree/main/tools/doc-generator) and adapted according to Loki configuration specifications. Prior to this PR, the configuration flags documentation was dispersed across two sources: * [_index.md](`5550cd65ec/docs/sources/configuration/_index.md`) * configuration flags registration in the code This meant that there was no single source of truth. In this PR, the previous `_index.md` file is replaced with the new file generated by the tool. The next step includes adding a CI step that validates if the _index.md file was generated according to the flags settings. This will be done in a follow-up PR. NOTE: this is not a documentation update PR. Apart from some minor typo fixes, the documentation changes on the code, were copied from the `_index.md` file. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki-private/issues/83 Special notes for your reviewer: Files: * [docs/sources/configuration/index.template](`5550cd65ec/docs/sources/configuration/index.template`): template used to generate the final configuration file * [/docs/sources/configuration/_index.md](`c32e5d0acb/docs/sources/configuration/_index.md`): file generated by tool * `loki/pkg` directory files updated with up-to-date documentation from `_index.md` file * [tools/doc-generator](`5550cd65ec/tools/doc-generator`) directory with documentation generation tool. Checklist - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Robert Fratto	85392a9728	Update Prometheus dependency to latest release (v2.40.4) (#7826 ) Closes #7811, which is needed for Grafana Agent to update to v2.40 and add support for native histograms. I did not add support for native histograms to Loki, sorry :)	3 years ago
Kaviraj Kanagaraj	25f4dda811	fix(memcached): use `default` branch avoid writing to closed chan (#7833 ) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> What this PR does / why we need it: Follow up to https://github.com/grafana/loki/pull/7817 This PR uses `default` branch instead of `inputCh <- work` to make sure we are writing to closed `inputCh`. Gist is, `default` run [only when none of the branch is ready](https://go.dev/tour/concurrency/6). which makes more sense rather than to have `inputCh <- work` (writing to closed channel on the branch condition) These can be explained by these two tiny snippets. * [with `default`](https://go.dev/play/p/-FspbTZd20I) * [without `default`](https://go.dev/play/p/Ag4WznOaEq0) Which issue(s) this PR fixes: Fixes #NA Special notes for your reviewer: We already have test `TestMemcached_fetchKeysBatched` to catch this behaviour. In fact this test caught this probabilistic behaviour in some CI failures. [here ](https://drone.grafana.net/grafana/loki/17853/3/4) and[ here](https://drone.grafana.net/grafana/loki/17854/3/4) Thanks @DylanGuedes for catching this CI failures. Checklist Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Kaviraj Kanagaraj	4efdc131be	fix(memcached): panic on `send on closed channel`. (#7817 )	3 years ago

1 2 3 4 5 ...

554 Commits (Alex3k-patch-2)