106 Commits (080a9300a1daccac562ae05c0d3be45e09a6ca91)

Author	SHA1	Message	Date
George Robinson	a6c83aa2e0	chore: remove deprecated metrics (#20844)	4 days ago
Robert Fratto	d85fb0317f	chore(dataobj): Add Open semantics to section readers (#20790) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	1 week ago
George Robinson	93f4bdf796	feat: allow the initial offset to be set in the starting phase (#20770)	2 weeks ago
Robert Fratto	7e03400309	chore(dataobj): rename dataset.ReaderOptions to dataset.RowReaderOptions (#20756)	2 weeks ago
Trevor Whitney	83e275775a	chore: bring comparison metrics and goldfish metrics more in line (#20706) ... Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2 weeks ago
George Robinson	c169ce3ae9	feat: add flush manager to control data object flush lifecycle (#20680)	2 weeks ago
George Robinson	4d1c4032cc	chore: small doc fix, missing period end of help (#20681)	2 weeks ago
George Robinson	72aaea20a6	chore: no lazy struct embed (#20682)	3 weeks ago
George Robinson	fcdd2adf2a	fix: service failure on stopping kafkav2 consumer (#20672)	3 weeks ago
George Robinson	7aee83d6d2	chore: use temporary builder to sort data objects (#20664)	3 weeks ago
George Robinson	18f2c5049e	chore: move metrics into observeRecordErr (#20625)	3 weeks ago
Robert Fratto	736ccdbf93	chore(dataobj): rename dataset.Reader to RowReader (#20629)	3 weeks ago
George Robinson	14d2285b76	fix: set consumption lag to zero after idle flush timeout (#20620)	3 weeks ago
George Robinson	0d9df6b055	chore: clean up consumer metrics (#20619)	3 weeks ago
George Robinson	cb938d0a9d	feat: separate metastore events and offsets from flushes (#20605)	3 weeks ago
George Robinson	45a7f55d39	fix: revert "feat: reset the consumption lag gauge after 1 minute of idle period" (#20617)	3 weeks ago
George Robinson	f1d0cdf145	feat: reset the consumption lag gauge after 1 minute of idle period (#20616)	3 weeks ago
George Robinson	8b86d42507	chore: no need for defer func (#20615)	3 weeks ago
George Robinson	4b2f46fc37	chore: move builder initialization out of processor.go (#20594)	3 weeks ago
George Robinson	7010fa8a87	chore: clean up partition processor code (#20593)	3 weeks ago
George Robinson	a85a9a200c	chore: replace quartz with synctest (#20452)	1 month ago
George Robinson	d5f6026e30	fix: service failed to shutdown errors when context canceled (#20450)	1 month ago
George Robinson	0b0a98de44	chore: use services.BasicService in partition_processor.go (#20448)	1 month ago
Ivan Kalita	7da73aaa7e	chore(dataobj): optimize bitmap decoder (#20467) ... - Use pkg/memory.Bitmap instead of []Value for pageReader presence bitmap - More efficient decoding for bitmapDecoder - only support 0/1 values (this encoding is used only for presence bitmap).	1 month ago
George Robinson	27df4ab481	feat: create separate flusher to support parallel flushes (#20446)	1 month ago
George Robinson	6a8b879d8f	feat: add support for cancelation to copy and sort (#20370)	1 month ago
George Robinson	ed4f27ed60	feat: add metric to track flush failures (#20399)	1 month ago
Dylan Guedes	498656bbe7	feat: Add new dataobj builder flush criteria (#20323)	1 month ago
George Robinson	533b1f0ae8	chore: remove unregister metrics (#20385)	1 month ago
George Robinson	3acb3106bb	feat: remove final flush (#20360)	1 month ago
George Robinson	c3e909d538	feat: decouple dataobj consumers from the reader service (#20315)	1 month ago
George Robinson	5a5e90e9f7	feat: add histogram loki_dataobj_consumer_flush_duration_seconds (#20304)	2 months ago
George Robinson	fba0c5d8a6	feat: dataobj-consumer add processed bytes metric (#20303)	2 months ago
George Robinson	bd77a47df9	chore: remove use of namespace/subsystem in builder metrics (#20301)	2 months ago
George Robinson	e0e5d78e69	chore: move appends total metric into builder (#20300)	2 months ago
George Robinson	a03e00a3bc	chore: better comments when end offset is zero (#20196)	2 months ago
George Robinson	333da739c7	feat: add processed records metric (#20191)	2 months ago
George Robinson	c0c27b3596	feat: add downscalePermittedFunc to check downscale is permitted (#20171)	2 months ago
benclive	225a0595ac	chore: Define an E2E ingestion processing time metric (#20054)	3 months ago
Ivan Kalita	3559d4bd42	chore(metastore): add column reader for pointers section (#19992) ... Adding columnar reader for (log)pointers section. Using it to read the stream section pointers. New reader is slower than existing RowReader due to double-translation of the data (columns->rows->rows vs columns->rows->columns->rows). The idea though is to get rid of the double-translation in the future.	3 months ago
Christian Haudum	8b59b944f2	chore(dataobj): Use common base config for indexobj and logsobj builders (#20037) ... Use the same config struct for both the indexobj builder and the logsobj builder, since they are configured identically, only with different default values.	3 months ago
George Robinson	677b2ecda1	feat: add prepare downscale handler (#20007)	3 months ago
George Robinson	9fb90309c0	chore: clean up prepare downscale handler (#20006)	3 months ago
Paul Rogers	91d4eb6ce1	feat: Add partition state to consumption lag metric (#19912)	3 months ago
George Robinson	a28e755e79	chore: rename the prepare delayed downscale handler to better communicate what it does (#19727)	4 months ago
George Robinson	0af297cd4c	chore: remove loki_dataobj_consumer_processing_delay histogram (#19718)	4 months ago
Christian Haudum	0b4a7f5a2c	chore(dataobj-consumer): Re-use builder for CopyAndSort() (#19696) ... Re-use logsobj.Builder instance in partition processor when sorting logs object wide after flushing. The metrics for the builder in CopyAndSort are never registered, which means the loki_dataobj_sort_duration_seconds histogram is never exported/scraped. We cannot register them with RegisterMetrics() because the metrics are already registered here, and registering them twice would cause a panic.	4 months ago
benclive	6ee50064ea	chore: Ensure partition is set on partition-processor (#19693)	4 months ago
George Robinson	da37290a40	fix: loki_dataobj_sort_duration_seconds never registered (#19679)	4 months ago
George Robinson	977fb2ba6f	feat: consume single topics in dataobj-consumer using partition ring (#19186)	4 months ago