138 Commits (15fe69cec9e705f38a2d7af3bc7b3f3ba8e28fe9)

Author	SHA1	Message	Date
Robby Milo	e4c6888ba8	chore: Remove relref shortcodes (#16624)	3 months ago
Ashwanth	b63c191a5a	docs(thanos): add migration doc for thanos storage clients (#16133) ... Signed-off-by: Ashwanth <iamashwanth@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 months ago
J Stickler	709a3a22c5	docs: add info about compression (#15699) ... Signed-off-by: J Stickler <julie.stickler@grafana.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	4 months ago
J Stickler	1b03a08bad	docs: fixing some Vale linter errors (#15212)	6 months ago
Sean P.	39c47e5b5a	docs: added additional GCS authentication information (#15112) ... Co-authored-by: J Stickler <julie.stickler@grafana.com>	6 months ago
J Stickler	30afd21ce7	docs: Replace shortcodes on Storage topic (#13748)	8 months ago
J Stickler	7b9ef19f0b	docs: Late review comments and linting (#13714)	10 months ago
J Stickler	3cbdd95e76	docs: Update storage topics (v3.0) (#12768)	1 year ago
Paul Rogers	afb4b604cd	docs: Add Azure example configuration snippets (#12725) ... Co-authored-by: J Stickler <julie.stickler@grafana.com>	1 year ago
J Stickler	41246a05ac	docs: fix doc-validate errors due to config move (#12662)	1 year ago
J Stickler	5e21ffafa9	docs: add missing aliases for moved pages (#12580) ... Co-authored-by: Jack Baldry <jack.baldry@grafana.com>	1 year ago
Jack Baldry	50fc13c422	docs: Refactor Loki configuration to be included via docs/shared (#12556) ... Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	1 year ago
Jack Baldry	cafc7b6f87	docs: Fix up storage page front matter (#12532) ... Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	1 year ago
Ed Welch	23b2e1d252	docs: remove whitespace in links (#12525) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
J Stickler	1c5a736641	docs: proper fix for #12510 (#12516)	1 year ago
Ed Welch	670041368a	chore: add more validation at startup to make sure we have a proper schema, and necessary directories are created (#12496) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	1 year ago
J Stickler	3634ff4d64	docs: move Storage page under Configuration (#12478)	1 year ago
Christian Haudum	a9345d0fb6	chore(blooms): Clean up settings and make reasonable defaults (#12483) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Ed Welch	705379cc81	chore: remove experimental flags from features and configs which are no longer so. (#12481) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Christian Haudum	ff08d2fcc2	chore(blooms): Remove bloom gateway ring (#12484) ... Remove ring in favour of a client side, DNS discovery of the bloom gateway servers combined with consistent hashing using the jumphash algorithm (#12470). Instead of configuring the ring, the bloom gateway client needs to be configured with the `-bloom-gateway-client.addresses` flag, which takes a list of comma separated, DNS-discovery-style strings. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	58c8c61905	perf(blooms): Use jumpmap for client side sharding of the filter requests (#12470) ... **What this PR does / why we need it**: Use the same client-side sharding mechanism for bloom gateway filter requests that we use for client side sharding in the memcached client. This uses DNS discovery to resolve a list of addresses uses as buckets for the jumphash algorithm to shard GroupedChunkRefs across bloom gateway servers. The ring component is still part of the bloom gateway, but it is not used any more. **Special notes for your reviewer**: https://arxiv.org/pdf/1406.2294.pdf Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Ed Welch	3ece2ea6c4	refactor!: remove async cache writeback on chunk fetcher (#12456) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Cyril Tovena	9e6e6d22e0	feat: Detect level as structured metadata by default (#12479)	1 year ago
Cyril Tovena	491d2513e7	feat: add log patterns ingestion and query API. (#12403) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Anton Kolesnikov <anton.e.kolesnikov@gmail.com> Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Ed Welch	bc26198c00	feat: enable caching of index stats results, volume results, series results and label results by default (#12452) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Ed Welch	7607f04484	feat!: enable structure metadata by default. (#12453) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Ed Welch	d39dc09ca6	refactor!: refactor how we do defaults for runtime overrides (#12448) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Dylan Guedes <djmgguedes@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	1 year ago
Sandeep Sukhani	b66c343383	feat: add support for discovering and adding log levels as structured metadata (#12428)	1 year ago
Christian Haudum	71602eb10f	chore(blooms): Clean up bloom component configuration (#12387) ... * Remove unused setting and move download parallelism configuration one level up. * Update description of bloom_shipper config block Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Sandeep Sukhani	0b7ff48175	chore: delete request processing improvements (#12259)	1 year ago
Ed Welch	a509871c55	chore: remove experimental flags for l2 cache and memcached "addresses" config (#12410)	1 year ago
Trevor Whitney	246623f7d0	fix(detected_fields): fix issues with frontend integration (#12406) ... This PRs fixes issues we found when integrating with the frontend * the `/experimental` api made it difficult to interact with using the existing datasource, so move to `v1/detected_fields` * the config flag was considered cumbersome as the only potential negative impact of the endpoint is when it is used, and nothing is currently using it * the use of an enum in the protobuf produced unexpected results in the json, so type was converted to string	1 year ago
Paul Rogers	b286075428	chore: Update examples to use v13 schema, along with related defaults (#12402) ... Co-authored-by: J Stickler <julie.stickler@grafana.com>	1 year ago
Sandeep Sukhani	288c0061bf	feat: discovery of name of services emitting the logs (#12392)	1 year ago
Salva Corts	86c768c3cc	feat: Blooms retention (#12258)	1 year ago
Trevor Whitney	5b4bfa54b6	feat(detected_fields): initial plumbing of new endpoint (#12385)	1 year ago
Christian Haudum	15dc2bac04	feat(bloomstore): Support for sharding blocks across multiple different directories (#12375) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	120b76d63e	chore(blooms): misc fixes (#12369) ... unbounds bloom-gw-client concurrency removes unused log_gateway_requests param adds note wrt bloom-gw results cache keygen issue add msg field to stats log sets bloomshipper download workers count to 16 default	1 year ago
Christian Haudum	19c046f14b	chore(blooms): Make max bloom page size for querying configurable (#12337) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	a36483b731	feat(blooms): bloom integration in query planning (#12208) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Dylan Guedes	51c54adc4f	feat: Memcached: Add TLS support (#12318)	1 year ago
Christian Haudum	863acc0152	chore(blooms): Make block query concurrency configurable (#12292) ... A concurrency of 10 per worker seems to be too high for a reasonable 8GiB memory limit on bloom gateways. So this PR makes the concurrency configurable and uses a default of 4. Memory usage of loading block pages into memory therefore being: ``` worker_concurrency x block_query_concurrency x max_block_page_size x 2 ``` With current defaults: ``` 4 x 4 x 32MiB x 2 = 1GiB ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	62b3f358a6	feat(bloomstore): Wire up new blocks cache (#12275) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	eb03974086	feat(blooms): Make bloom block compression configurable (#12293) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Salva Corts	63c88489d0	refactor: Add RF and Tokens to Loki ring Cfg and allow overwriting docs. (#12142) ... Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Sandeep Sukhani	8b9d8b0226	feat: Make list of otel blessed attributes configurable (#12180)	1 year ago
Owen Diehl	8a8756aa7a	chore(blooms): honors bloom_compactor_enabled for tenants (#12190)	1 year ago
Christian Haudum	a0fce39169	feat(bloom-gw): Make num multiplexing tasks configurable, and other adjustments (#12169) ... * We want to be able to change the amount of dequeued items per iterations to test different configurations. * Ignore block pages that are magnitues bigger than the target page size. * Increase download queue capacity to avoid blocking when enqueueing Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	f6cfff5356	refactor(bloomstore): Introduce fetch option for blocks fetcher (#12160) ... Requests to the bloom gateway should not wait for blocks to be downloaded into cache first, but should operate on readily available blocks. This PR introduces fetch options for the bloom store that specify how the store should behave when requesting blocks from block refs. `WithIgnoreMissing`: Ignore errors from blocks that could not be found in object storage, and instead return a `nil` value in the response. `WithFetchAsync`: Return only the blocks that are available locally in cache and dispatch downloading of blocks to the blocks downloader, that operates in the background, and make them available for subsequent requests. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Salva Corts	f7710ad7aa	feat: (blooms) Configurable number of tokens (#12125)	1 year ago