Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Robert Fratto f6091a67d1 chore(engine): move to toggleable section prefetching (#19142) ... dataset.readerDownloader was originally introduced in #16429, an attempt to balance peak memory usage of reading a section with read times by downloading a configurable size of pages in advance. In practice, each roundtrip to object storage adds too much of a latency hit, and we've started to set the cache limit high enough to ensure that each reader only needs a single prefetch. Given what we've found, it no longer makes sense to control peak memory usage via the prefetch size. Other options, such as downloading directly to disk, may be explored in the future. In the meantime, this PR removes the ability to specify a cache size. All non-pruned pages will be bulk requested using the range reader (#19067) on the first read call. Pages which have left the potential read window will continue to be eagerly removed for garbage collection. However, we don't want to prefetch when the dataset is entirely in memory, which is the case when the logs section builder is performing k-way merge over in-memory sections. To lower the memory usage of builders, prefetching is configurable. For this initial PR, prefetching is only disabled for the logs section builder; all other reads force prefetching. Signed-off-by: Robert Fratto <robertfratto@gmail.com>		4 months ago
..
analytics	ci: skip flaky analytics test (#17009)	9 months ago
blockbuilder	refactor(stringlabels): Support stringlabels in util, engine and logql. (#17998)	7 months ago
bloombuild	chore: consistently use new labels.Labels API (#18490)	6 months ago
bloomgateway	chore: consistently use new labels.Labels API (#18490)	6 months ago
canary	chore: Autofix to make nosec comments also readable by semgrep (#18664)	5 months ago
chunkenc	chore: upgrade Prometheus to 208187eaa19b (#18756)	5 months ago
compactor	chore: make index updates idempotent from horizontally scalable compactor (#19093)	4 months ago
compression	…
dataobj	chore(engine): move to toggleable section prefetching (#19142)	4 months ago
distributor	feat: remove tenant topic tee (#19082)	4 months ago
engine	chore(engine): move to toggleable section prefetching (#19142)	4 months ago
framedstdcopy	…
goldfish	fix: Goldfish context cancellation and child spans (#18837)	5 months ago
indexgateway	chore: consistently use new labels.Labels API (#18490)	6 months ago
ingester	feat: per policy overridable stream limits (#18994)	4 months ago
iter	fix: Revert "perf: Fix memory leak in cachedIterator (#17628)" (#18687)	5 months ago
kafka	fix: update kfake to fix tests and revert go-redis (#18958)	5 months ago
limits	chore: fix inconsistent function name in comment (#19081)	4 months ago
logcli	feat: add delete commands to logcli (#18375)	6 months ago
loghttp	chore(otlp): Add support for zstd and lz4 content-encodings for OTLP logs (#18842)	4 months ago
logproto	chore: Autofix to make nosec comments also readable by semgrep (#18664)	5 months ago
logql	chore(engine): move to toggleable section prefetching (#19142)	4 months ago
logqlanalyzer	chore: consistently use new labels.Labels API (#18490)	6 months ago
logqlmodel	chore(logqlmodel/stats): print new engine summary in metrics.go (#19052)	4 months ago
loki	chore: Remove dataobj querier component (#19132)	4 months ago
lokifrontend	chore: Autofix to make nosec comments also readable by semgrep (#18664)	5 months ago
pattern	test: use static time to avoid flaky test (#18718)	5 months ago
push	fix(deps): update module github.com/stretchr/testify to v1.11.1 (main) (#19042)	4 months ago
querier	chore: Implement building multi-tenant indexes from multi-tenant objects (#18959)	4 months ago
queue	chore: Autofix to make nosec comments also readable by semgrep (#18664)	5 months ago
ruler	chore: consistently use new labels.Labels API (#18490)	6 months ago
runtime	chore: improve logging of push request streams (#17872)	7 months ago
scheduler	refactor: Use OTel tracing library (#17859)	7 months ago
scratch	chore(dataobj): add ability to buffer pending sections to disk (#18780)	5 months ago
server	…
storage	chore: make index updates idempotent from horizontally scalable compactor (#19093)	4 months ago
tool	chore: Autofix to make nosec comments also readable by semgrep (#18664)	5 months ago
tracing	refactor: Use OTel tracing library (#17859)	7 months ago
ui	chore(deps): update dependency vite to v7.1.4 (main) (#19116)	4 months ago
util	chore(rangeio): add tracing to rangeio.ReadRanges (#19067)	4 months ago
validation	feat: per policy overridable stream limits (#18994)	4 months ago