Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Ivan Kalita 3f62bb7207 chore(compactor): Allow overflowed logsobj builders (#22148) ... I'm changing the behavior of logsobj builders to allow appends to overflowed builders. Before this PR: (1) Processor tries to append a record to a builder (2) If a builder is full it doesn't append and returns an error ErrBuilderFull (3) Processor checks the error and flushes the builder (4) Processor tries to append the same record as in the step (1) to the now empty builder This approach works fine but becomes problematic once we introduce multiple TOC windows builders. The issue is that with a single builder we have atomic "try append - flush if error" behavior guarantee. With multiple builders this becomes a problem as a single kafka record might contain entries that belong to different builders. Example: - we have a record R that has a log entry L0 for TOC window W0 AND a log entry L1 for TOC window W1 - we attempt to append L0 to the first builder B0 (for TOC W0) -> success - we attempt to append L1 to the second builder B1 (for TOC W1) -> not appended, err builder full - we flush all the builders (have to flush all of them because otherwise we can lose data as Kafka offset is committed after flushing) - we attempt to append R again -> builder B0 will receive a duplicate of log L0 To simplify the processing we can allow appends to full builders ("one last time"). For the example above we'd allow the append of L1 to B1 even though B1 is already full. This way the dataobjects might become slightly bigger than the target size but this could already be the case given that the existing size estimates are estimates.		6 days ago
..
dag	feat: Resolve cached tasks in parent task (#21558)	2 months ago
ewma	chore(engine): track scheduler load/pressure metrics (#19924)	7 months ago
objtest	chore(compactor): Allow overflowed logsobj builders (#22148)	6 days ago
queue/fair	chore(scheduler): assign tasks in per-worker loop (#20826)	4 months ago
tree	chore(engine): add workflow package (#19511)	8 months ago
array_list.go	fix: errors in parse pipeline (#19667)	7 months ago
time.go	chore(linter): Fix issues flagged by updated golangci-lint version (#19206)	9 months ago