Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

aarogoss b80f18636f feat: update loki canary to support batched log push (#17558) ... What this PR does / why we need it: This commit introduces a new EntryWriter implementation in the Loki canary codebase, supporting batched log pushes. A new runtime CLI opt, -logs-batch-size receives an int, specifying the # of log lines to batch before they are sent to Loki. This opt may be set to '0' or '1' to disable batching entirely, in which case the canary functionality remains as-is. If an int value > 1 is passed to this CLI opt, a new BatchedPush instance is created. This instance collects log lines from its channel and stores them in local array; once the array size is at least the value of -logs-batch-size, the logs are then marshaled and pushed to Loki using the standard push API. A second CLI opt is added, -logs-batch-size-max. This arg restricts the -logs-batch-size opt passed by the user so the # of batched log lines don't blow up the memory of the loki-canary pod. The default value is 20, ensuring the array stays w/in ~20MBs (1MB log payloads). If 20MB is too restrictive, the -logs-batch-size-max can be used to exceed this limit. This implementation also maintains a 30s countdown timer; every 30s, the timer triggers and whatever logs exist in the "batch" array are pushed to Loki. This ensures batched logs are pushed to Loki even if the canary fails to receive more logs. Moreover, if/when the canary org is shutdown (i.e., the termination channel receives a signal to quit), the remaining logs are likewise pushed to Loki and the shutdown process will continue.		1 year ago
..
comparator	chore: Preparation for incoming static code analysis CI check (#15164)	2 years ago
reader	fix(canary): Reconnect immediately upon tail max duration (#14287)	2 years ago
writer	feat: update loki canary to support batched log push (#17558)	1 year ago