apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Salva Corts	cd64d6ddbd	refactor(blooms): Implement retry mechanisms in planner (#13064 )	2 years ago
Karsten Jeschkies	6e4555010e	fix: Introduce feature flag for [last\|first]_over_time sharding. (#13067 )	2 years ago
Salva Corts	ab7af0575d	refactor(blooms): Builder retrieves tasks from planner (#13046 )	2 years ago
Vladyslav Diachenko	1111595179	feat: new stream count limiter (#13006 ) Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: JordanRushing <rushing.jordan@gmail.com>	2 years ago
Salva Corts	efd8f5dc1b	refactor(blooms): Add queue to bloom planner and enqueue tasks (#13005 )	2 years ago
Salva Corts	3195036435	refactor(bloom planner): Compute gaps and build tasks from metas and TSDBs (#12994 )	2 years ago
Salva Corts	8978ecf0c8	feat: Boilerplate for new bloom build planner and worker components. (#12989 )	2 years ago
Dylan Guedes	18a558c7fd	docs: Don't allow running new and old querier worker grpc clients (#12916 )	2 years ago
Dylan Guedes	afd9e36306	chore: Call `shardstreams.Config` by value instead of by reference (#12915 ) Use `shardstreams.Config` by value instead of by reference to fix docs generation. Our `docs-generator` tool relies on the struct address/references to assume that flags are present. Using this config by value fixes it.	2 years ago
Dylan Guedes	7b6f0577c3	feat: Querier: Split gRPC client into two. (#12726 ) What this PR does / why we need it: Split the gRPC client used by the querier into two, one for the communication with the scheduler, the other for communicating with the query-frontend. - This change is retrocompatible: you don't have to change anything to keep existing behavior. - To configure the custom scheduler grpc client, you can use the new `query_scheduler_grpc_client` config or the new CLI flag `querier.scheduler-grpc-client` - If you'd like to configure your frontend grpc client using a better named section, you can use the new `query_frontend_grpc_client` instead of the old `grpc_client_config`. Just make sure you don't use both at the same time, it will result in an error. This work is necessary for configuring custom behavior between `querier<->scheduler` vs `querier<->frontend`. A use case is configuring mTLS when a different certificate is used by queriers, schedulers and frontends. You can only configure a single `server_name` with our current setup, making it impossible.	2 years ago
Shantanu Alshi	6904a6520d	fix: Fix log level detection (#12651 )	2 years ago
Christian Haudum	fcd544c2d9	feat(blooms): Add in-memory LRU cache for meta files (#12862 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Owen Diehl	bc78d13d9b	feat(blooms): compute chunks once (#12664 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Owen Diehl	eac56224b8	feat(blooms): limit bloom size during creation (#12796 )	2 years ago
Trevor Whitney	4404f7d2f7	docs: order configuration template fileds alphabetically (#12771 )	2 years ago
Trevor Whitney	af09f53a33	ci: release workflows get build image from makefile (#12778 )	2 years ago
Travis Patterson	1161846e19	fix(docs): Move promtail configuration to the correct doc (#12737 )	2 years ago
Ed Welch	4c563f7823	docs: unhide compactor GRPC config (#12587 ) Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: DylanGuedes <djmgguedes@gmail.com>	2 years ago
Cyril Tovena	397aa56e15	feat: Enable log volume endpoint by default (#12628 )	2 years ago
Jack Baldry	50fc13c422	docs: Refactor Loki configuration to be included via docs/shared (#12556 ) Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	2 years ago
Christian Haudum	a9345d0fb6	chore(blooms): Clean up settings and make reasonable defaults (#12483 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Ed Welch	705379cc81	chore: remove experimental flags from features and configs which are no longer so. (#12481 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Christian Haudum	ff08d2fcc2	chore(blooms): Remove bloom gateway ring (#12484 ) Remove ring in favour of a client side, DNS discovery of the bloom gateway servers combined with consistent hashing using the jumphash algorithm (#12470). Instead of configuring the ring, the bloom gateway client needs to be configured with the `-bloom-gateway-client.addresses` flag, which takes a list of comma separated, DNS-discovery-style strings. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	58c8c61905	perf(blooms): Use jumpmap for client side sharding of the filter requests (#12470 ) What this PR does / why we need it: Use the same client-side sharding mechanism for bloom gateway filter requests that we use for client side sharding in the memcached client. This uses DNS discovery to resolve a list of addresses uses as buckets for the jumphash algorithm to shard GroupedChunkRefs across bloom gateway servers. The ring component is still part of the bloom gateway, but it is not used any more. Special notes for your reviewer: https://arxiv.org/pdf/1406.2294.pdf Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Ed Welch	3ece2ea6c4	refactor!: remove async cache writeback on chunk fetcher (#12456 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Cyril Tovena	9e6e6d22e0	feat: Detect level as structured metadata by default (#12479 )	2 years ago
Cyril Tovena	491d2513e7	feat: add log patterns ingestion and query API. (#12403 ) Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Anton Kolesnikov <anton.e.kolesnikov@gmail.com> Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Ed Welch	bc26198c00	feat: enable caching of index stats results, volume results, series results and label results by default (#12452 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Ed Welch	7607f04484	feat!: enable structure metadata by default. (#12453 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Ed Welch	d39dc09ca6	refactor!: refactor how we do defaults for runtime overrides (#12448 ) Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Dylan Guedes <djmgguedes@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Sandeep Sukhani	b66c343383	feat: add support for discovering and adding log levels as structured metadata (#12428 )	2 years ago
Christian Haudum	71602eb10f	chore(blooms): Clean up bloom component configuration (#12387 ) * Remove unused setting and move download parallelism configuration one level up. * Update description of bloom_shipper config block Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Sandeep Sukhani	0b7ff48175	chore: delete request processing improvements (#12259 )	2 years ago
Ed Welch	a509871c55	chore: remove experimental flags for l2 cache and memcached "addresses" config (#12410 )	2 years ago
Trevor Whitney	246623f7d0	fix(detected_fields): fix issues with frontend integration (#12406 ) This PRs fixes issues we found when integrating with the frontend * the `/experimental` api made it difficult to interact with using the existing datasource, so move to `v1/detected_fields` * the config flag was considered cumbersome as the only potential negative impact of the endpoint is when it is used, and nothing is currently using it * the use of an enum in the protobuf produced unexpected results in the json, so type was converted to string	2 years ago
Sandeep Sukhani	288c0061bf	feat: discovery of name of services emitting the logs (#12392 )	2 years ago
Salva Corts	86c768c3cc	feat: Blooms retention (#12258 )	2 years ago
Trevor Whitney	5b4bfa54b6	feat(detected_fields): initial plumbing of new endpoint (#12385 )	2 years ago
Christian Haudum	15dc2bac04	feat(bloomstore): Support for sharding blocks across multiple different directories (#12375 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Owen Diehl	120b76d63e	chore(blooms): misc fixes (#12369 ) unbounds bloom-gw-client concurrency removes unused log_gateway_requests param adds note wrt bloom-gw results cache keygen issue add msg field to stats log sets bloomshipper download workers count to 16 default	2 years ago
Christian Haudum	19c046f14b	chore(blooms): Make max bloom page size for querying configurable (#12337 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Owen Diehl	a36483b731	feat(blooms): bloom integration in query planning (#12208 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Dylan Guedes	51c54adc4f	feat: Memcached: Add TLS support (#12318 )	2 years ago
Christian Haudum	863acc0152	chore(blooms): Make block query concurrency configurable (#12292 ) A concurrency of 10 per worker seems to be too high for a reasonable 8GiB memory limit on bloom gateways. So this PR makes the concurrency configurable and uses a default of 4. Memory usage of loading block pages into memory therefore being: ``` worker_concurrency x block_query_concurrency x max_block_page_size x 2 ``` With current defaults: ``` 4 x 4 x 32MiB x 2 = 1GiB ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	62b3f358a6	feat(bloomstore): Wire up new blocks cache (#12275 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	eb03974086	feat(blooms): Make bloom block compression configurable (#12293 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Salva Corts	63c88489d0	refactor: Add RF and Tokens to Loki ring Cfg and allow overwriting docs. (#12142 ) Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Sandeep Sukhani	8b9d8b0226	feat: Make list of otel blessed attributes configurable (#12180 )	2 years ago
Owen Diehl	8a8756aa7a	chore(blooms): honors bloom_compactor_enabled for tenants (#12190 )	2 years ago
Christian Haudum	a0fce39169	feat(bloom-gw): Make num multiplexing tasks configurable, and other adjustments (#12169 ) * We want to be able to change the amount of dequeued items per iterations to test different configurations. * Ignore block pages that are magnitues bigger than the target page size. * Increase download queue capacity to avoid blocking when enqueueing Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago

20 Commits (cd64d6ddbd58981c4d3328a50a59179b0b33c696)