76 Commits (867ce3d711f3b9e823e8eb9b61c076acca339096)

Author	SHA1	Message	Date
Christian Haudum	f06eabbf0e	feat: Add settings for cpu/mutex/block profiling options (#13278) ... The HTTP endpoints for scraping CPU, mutex and block profiles are already exposed by the server, however, block and mutex profiling must be explicitly enabled by setting a non-zero value for `SetBlockProfileRate` and `SetMutexProfileFraction` respectively. For more information about the available options consult the Go documentation: * https://pkg.go.dev/runtime#SetBlockProfileRate * https://pkg.go.dev/runtime#SetMutexProfileFraction * https://pkg.go.dev/runtime#SetCPUProfileRate Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	11 months ago
Trevor Whitney	467eb1bb1b	feat: collect and serve pre-aggregated bytes and counts (#13020) ... Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	11 months ago
George Robinson	384f451193	chore: Change default version to Alertmanager APIv2 (#13263)	11 months ago
JordanRushing	e7689b248d	feat: add recalculateOwnedStreams to check stream ownership if the ring is changed (#13103) ... Signed-off-by: JordanRushing <rushing.jordan@gmail.com> Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	11 months ago
George Robinson	9767807680	feat: Add backoff to flush op (#13140)	12 months ago
Salva Corts	f7887269f8	refactor(blooms): Limit task retries in bloom planner (#13139)	12 months ago
Salva Corts	cd64d6ddbd	refactor(blooms): Implement retry mechanisms in planner (#13064)	12 months ago
Karsten Jeschkies	6e4555010e	fix: Introduce feature flag for [last|first]_over_time sharding. (#13067)	12 months ago
Salva Corts	ab7af0575d	refactor(blooms): Builder retrieves tasks from planner (#13046)	12 months ago
Vladyslav Diachenko	1111595179	feat: new stream count limiter (#13006) ... Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: JordanRushing <rushing.jordan@gmail.com>	1 year ago
Salva Corts	efd8f5dc1b	refactor(blooms): Add queue to bloom planner and enqueue tasks (#13005)	1 year ago
Salva Corts	3195036435	refactor(bloom planner): Compute gaps and build tasks from metas and TSDBs (#12994)	1 year ago
Salva Corts	8978ecf0c8	feat: Boilerplate for new bloom build planner and worker components. (#12989)	1 year ago
Dylan Guedes	18a558c7fd	docs: Don't allow running new and old querier worker grpc clients (#12916)	1 year ago
Dylan Guedes	afd9e36306	chore: Call `shardstreams.Config` by value instead of by reference (#12915) ... Use `shardstreams.Config` by value instead of by reference to fix docs generation. Our `docs-generator` tool relies on the struct address/references to assume that flags are present. Using this config by value fixes it.	1 year ago
Dylan Guedes	7b6f0577c3	feat: Querier: Split gRPC client into two. (#12726) ... **What this PR does / why we need it**: Split the gRPC client used by the querier into two, one for the communication with the scheduler, the other for communicating with the query-frontend. - This change is retrocompatible: you don't have to change anything to keep existing behavior. - To configure the custom scheduler grpc client, you can use the new `query_scheduler_grpc_client` config or the new CLI flag `querier.scheduler-grpc-client` - If you'd like to configure your frontend grpc client using a better named section, you can use the new `query_frontend_grpc_client` instead of the old `grpc_client_config`. Just make sure you don't use both at the same time, it will result in an error. This work is necessary for configuring custom behavior between `querier<->scheduler` vs `querier<->frontend`. A use case is configuring mTLS when a different certificate is used by queriers, schedulers and frontends. You can only configure a single `server_name` with our current setup, making it impossible.	1 year ago
Shantanu Alshi	6904a6520d	fix: Fix log level detection (#12651)	1 year ago
Christian Haudum	fcd544c2d9	feat(blooms): Add in-memory LRU cache for meta files (#12862) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	bc78d13d9b	feat(blooms): compute chunks once (#12664) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Owen Diehl	eac56224b8	feat(blooms): limit bloom size during creation (#12796)	1 year ago
Trevor Whitney	4404f7d2f7	docs: order configuration template fileds alphabetically (#12771)	1 year ago
Trevor Whitney	af09f53a33	ci: release workflows get build image from makefile (#12778)	1 year ago
Travis Patterson	1161846e19	fix(docs): Move promtail configuration to the correct doc (#12737)	1 year ago
Ed Welch	4c563f7823	docs: unhide compactor GRPC config (#12587) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: DylanGuedes <djmgguedes@gmail.com>	1 year ago
Cyril Tovena	397aa56e15	feat: Enable log volume endpoint by default (#12628)	1 year ago
Jack Baldry	50fc13c422	docs: Refactor Loki configuration to be included via docs/shared (#12556) ... Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	1 year ago
Christian Haudum	a9345d0fb6	chore(blooms): Clean up settings and make reasonable defaults (#12483) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Ed Welch	705379cc81	chore: remove experimental flags from features and configs which are no longer so. (#12481) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Christian Haudum	ff08d2fcc2	chore(blooms): Remove bloom gateway ring (#12484) ... Remove ring in favour of a client side, DNS discovery of the bloom gateway servers combined with consistent hashing using the jumphash algorithm (#12470). Instead of configuring the ring, the bloom gateway client needs to be configured with the `-bloom-gateway-client.addresses` flag, which takes a list of comma separated, DNS-discovery-style strings. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	58c8c61905	perf(blooms): Use jumpmap for client side sharding of the filter requests (#12470) ... **What this PR does / why we need it**: Use the same client-side sharding mechanism for bloom gateway filter requests that we use for client side sharding in the memcached client. This uses DNS discovery to resolve a list of addresses uses as buckets for the jumphash algorithm to shard GroupedChunkRefs across bloom gateway servers. The ring component is still part of the bloom gateway, but it is not used any more. **Special notes for your reviewer**: https://arxiv.org/pdf/1406.2294.pdf Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Ed Welch	3ece2ea6c4	refactor!: remove async cache writeback on chunk fetcher (#12456) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Cyril Tovena	9e6e6d22e0	feat: Detect level as structured metadata by default (#12479)	1 year ago
Cyril Tovena	491d2513e7	feat: add log patterns ingestion and query API. (#12403) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Anton Kolesnikov <anton.e.kolesnikov@gmail.com> Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Ed Welch	bc26198c00	feat: enable caching of index stats results, volume results, series results and label results by default (#12452) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Ed Welch	7607f04484	feat!: enable structure metadata by default. (#12453) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Ed Welch	d39dc09ca6	refactor!: refactor how we do defaults for runtime overrides (#12448) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Dylan Guedes <djmgguedes@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	1 year ago
Sandeep Sukhani	b66c343383	feat: add support for discovering and adding log levels as structured metadata (#12428)	1 year ago
Christian Haudum	71602eb10f	chore(blooms): Clean up bloom component configuration (#12387) ... * Remove unused setting and move download parallelism configuration one level up. * Update description of bloom_shipper config block Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Sandeep Sukhani	0b7ff48175	chore: delete request processing improvements (#12259)	1 year ago
Ed Welch	a509871c55	chore: remove experimental flags for l2 cache and memcached "addresses" config (#12410)	1 year ago
Trevor Whitney	246623f7d0	fix(detected_fields): fix issues with frontend integration (#12406) ... This PRs fixes issues we found when integrating with the frontend * the `/experimental` api made it difficult to interact with using the existing datasource, so move to `v1/detected_fields` * the config flag was considered cumbersome as the only potential negative impact of the endpoint is when it is used, and nothing is currently using it * the use of an enum in the protobuf produced unexpected results in the json, so type was converted to string	1 year ago
Sandeep Sukhani	288c0061bf	feat: discovery of name of services emitting the logs (#12392)	1 year ago
Salva Corts	86c768c3cc	feat: Blooms retention (#12258)	1 year ago
Trevor Whitney	5b4bfa54b6	feat(detected_fields): initial plumbing of new endpoint (#12385)	1 year ago
Christian Haudum	15dc2bac04	feat(bloomstore): Support for sharding blocks across multiple different directories (#12375) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	120b76d63e	chore(blooms): misc fixes (#12369) ... unbounds bloom-gw-client concurrency removes unused log_gateway_requests param adds note wrt bloom-gw results cache keygen issue add msg field to stats log sets bloomshipper download workers count to 16 default	1 year ago
Christian Haudum	19c046f14b	chore(blooms): Make max bloom page size for querying configurable (#12337) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	a36483b731	feat(blooms): bloom integration in query planning (#12208) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Dylan Guedes	51c54adc4f	feat: Memcached: Add TLS support (#12318)	1 year ago
Christian Haudum	863acc0152	chore(blooms): Make block query concurrency configurable (#12292) ... A concurrency of 10 per worker seems to be too high for a reasonable 8GiB memory limit on bloom gateways. So this PR makes the concurrency configurable and uses a default of 4. Memory usage of loading block pages into memory therefore being: ``` worker_concurrency x block_query_concurrency x max_block_page_size x 2 ``` With current defaults: ``` 4 x 4 x 32MiB x 2 = 1GiB ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago