115 Commits (main)

Author	SHA1	Message	Date
George Robinson	c405d54f72	fix: Don't drain in Kafka mode (#21716)	2 days ago
George Robinson	61b377db50	fix: Flush reason counter not initialized to 0, breaks PromQL functions (#21772)	3 days ago
Christian Haudum	aaa2d1137a	chore: Remove unused/dead code (#21724) ... * chore(dataobj): Remove dead code * chore(engine): Remove unused code * chore(ci): Fix PR/commit title for Renovate (#21721) * fix: Incorrect impl of ProduceSync (#21702) * (origin/k304) docs: New usage statistics topic (#21668) * feat(lokitool): Rename `rules lint` to `rules format` (#21671) * chore: single binary without Kafka (#21318)	5 days ago
Shantanu Alshi	368583e778	chore: single binary without Kafka (#21318) ... Signed-off-by: Shantanu Alshi <shantanu.alshi@grafana.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 weeks ago
George Robinson	47e1e81f4e	feat: add time sharded estimate (#21463) ... This commit adds a new metric that tracks the number of data objects produced per flush if we build data objects with 12 hour UTC windows.	4 weeks ago
Dylan Guedes	bd7530f322	feat: Use `AppendOrdered` strategy to avoid sorting and merging stripes (#21395) ... Signed-off-by: Dylan Guedes <djmgguedes@gmail.com> Co-authored-by: George Robinson <george.robinson@grafana.com>	4 weeks ago
Shantanu Alshi	30549ac587	feat(dataobj): Add sort_schema config and feature flag (#21460)	1 month ago
George Robinson	6cf495f646	chore: rename flushManager to flushCommitter to prepare for future changes (#21135)	2 months ago
George Robinson	8a68101b3b	feat: add builder pool to logsobj package (#21134)	2 months ago
George Robinson	a6c83aa2e0	chore: remove deprecated metrics (#20844)	3 months ago
Robert Fratto	d85fb0317f	chore(dataobj): Add Open semantics to section readers (#20790) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	3 months ago
George Robinson	93f4bdf796	feat: allow the initial offset to be set in the starting phase (#20770)	3 months ago
Robert Fratto	7e03400309	chore(dataobj): rename dataset.ReaderOptions to dataset.RowReaderOptions (#20756)	3 months ago
Trevor Whitney	83e275775a	chore: bring comparison metrics and goldfish metrics more in line (#20706) ... Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	3 months ago
George Robinson	c169ce3ae9	feat: add flush manager to control data object flush lifecycle (#20680)	3 months ago
George Robinson	4d1c4032cc	chore: small doc fix, missing period end of help (#20681)	3 months ago
George Robinson	72aaea20a6	chore: no lazy struct embed (#20682)	3 months ago
George Robinson	fcdd2adf2a	fix: service failure on stopping kafkav2 consumer (#20672)	3 months ago
George Robinson	7aee83d6d2	chore: use temporary builder to sort data objects (#20664)	3 months ago
George Robinson	18f2c5049e	chore: move metrics into observeRecordErr (#20625)	3 months ago
Robert Fratto	736ccdbf93	chore(dataobj): rename dataset.Reader to RowReader (#20629)	3 months ago
George Robinson	14d2285b76	fix: set consumption lag to zero after idle flush timeout (#20620)	3 months ago
George Robinson	0d9df6b055	chore: clean up consumer metrics (#20619)	3 months ago
George Robinson	cb938d0a9d	feat: separate metastore events and offsets from flushes (#20605)	3 months ago
George Robinson	45a7f55d39	fix: revert "feat: reset the consumption lag gauge after 1 minute of idle period" (#20617)	3 months ago
George Robinson	f1d0cdf145	feat: reset the consumption lag gauge after 1 minute of idle period (#20616)	3 months ago
George Robinson	8b86d42507	chore: no need for defer func (#20615)	3 months ago
George Robinson	4b2f46fc37	chore: move builder initialization out of processor.go (#20594)	3 months ago
George Robinson	7010fa8a87	chore: clean up partition processor code (#20593)	3 months ago
George Robinson	a85a9a200c	chore: replace quartz with synctest (#20452)	4 months ago
George Robinson	d5f6026e30	fix: service failed to shutdown errors when context canceled (#20450)	4 months ago
George Robinson	0b0a98de44	chore: use services.BasicService in partition_processor.go (#20448)	4 months ago
Ivan Kalita	7da73aaa7e	chore(dataobj): optimize bitmap decoder (#20467) ... - Use pkg/memory.Bitmap instead of []Value for pageReader presence bitmap - More efficient decoding for bitmapDecoder - only support 0/1 values (this encoding is used only for presence bitmap).	4 months ago
George Robinson	27df4ab481	feat: create separate flusher to support parallel flushes (#20446)	4 months ago
George Robinson	6a8b879d8f	feat: add support for cancelation to copy and sort (#20370)	4 months ago
George Robinson	ed4f27ed60	feat: add metric to track flush failures (#20399)	4 months ago
Dylan Guedes	498656bbe7	feat: Add new dataobj builder flush criteria (#20323)	4 months ago
George Robinson	533b1f0ae8	chore: remove unregister metrics (#20385)	4 months ago
George Robinson	3acb3106bb	feat: remove final flush (#20360)	4 months ago
George Robinson	c3e909d538	feat: decouple dataobj consumers from the reader service (#20315)	4 months ago
George Robinson	5a5e90e9f7	feat: add histogram loki_dataobj_consumer_flush_duration_seconds (#20304)	4 months ago
George Robinson	fba0c5d8a6	feat: dataobj-consumer add processed bytes metric (#20303)	4 months ago
George Robinson	bd77a47df9	chore: remove use of namespace/subsystem in builder metrics (#20301)	4 months ago
George Robinson	e0e5d78e69	chore: move appends total metric into builder (#20300)	4 months ago
George Robinson	a03e00a3bc	chore: better comments when end offset is zero (#20196)	5 months ago
George Robinson	333da739c7	feat: add processed records metric (#20191)	5 months ago
George Robinson	c0c27b3596	feat: add downscalePermittedFunc to check downscale is permitted (#20171)	5 months ago
benclive	225a0595ac	chore: Define an E2E ingestion processing time metric (#20054)	5 months ago
Ivan Kalita	3559d4bd42	chore(metastore): add column reader for pointers section (#19992) ... Adding columnar reader for (log)pointers section. Using it to read the stream section pointers. New reader is slower than existing RowReader due to double-translation of the data (columns->rows->rows vs columns->rows->columns->rows). The idea though is to get rid of the double-translation in the future.	5 months ago
Christian Haudum	8b59b944f2	chore(dataobj): Use common base config for indexobj and logsobj builders (#20037) ... Use the same config struct for both the indexobj builder and the logsobj builder, since they are configured identically, only with different default values.	5 months ago