apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
wphan	d627cda848	Support microsecond timestamp format (#1426 ) Adds microsecond timestamp support. https://github.com/grafana/loki/issues/1407	5 years ago
Peter Štibraný	a2c2a90907	Lint: comments Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Peter Štibraný	06007c0ad1	Renamed LZ4 -> Lz4 to make lint happy. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Peter Štibraný	fb143fec02	Undo small change. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Peter Štibraný	9979321c2a	Don't pool LZ4 readers that read big blocks. Such readers must have allocated twice as many bytes as the block size, and we don't want to pool them and keep that memory allocated. This can happen when reading chunks that were compressed with high buffer size. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Peter Štibraný	b0a7cf8f9d	Change default LZ4 buffer size to 64k. Added more LZ4 buffer size options for testing. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Robert Fratto	874d016bd7	promtail: add support for passing through journal entries as JSON This commit adds an extra configuration field (`json`) to the journal target configuration schema. When set to true, messages from the journal target will be passed through as JSON, containing all of the fields from the entry as keys and their values as the value. For backwards compatibility, the value for `json` defaults to false. Closes #1288.	5 years ago
Peter Štibraný	dca244f7aa	pkg/ingester: added sync period flags SyncPeriod can be used to tell ingesters to cut chunks early (before they are full) to synchronize ingesters so that they put the same data into the new chunks. If they do, they generate fully equal chunks and then they don't need to store them. To avoid very small chunks, additional parameter is added: -ingester.sync-min-utilization. When set to eg. 0.5, chunk must be at least 50% full (utilization) in order to be cut early for sync purposes. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Xiang Dai	bb0a64a64b	fix all misspell (#1437 ) Signed-off-by: Xiang Dai <764524258@qq.com>	6 years ago
Joe Elliott	4f488e7dea	Removed unsupported encodings from docs (#1432 ) * Removed unsupported encodings from docs Signed-off-by: Joe Elliott <number101010@gmail.com> * Also removed none Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Peter Štibraný	93b85f92fa	pkg/ingester: Added possibility to disable transfers. (#1425 ) Previously, with -ingester.max-transfer-retries set to zero, transfers would never stop. Now zero means don't transfer at all. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Cyril Tovena	949126e75b	Add a line limit in case a chunk is corrupted to avoid OOM. Also when running out of memory we should just return an error and not keep going. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Owen Diehl	ccf9630c58	asserts encodings are nominally the same in chunks during roundtrips	6 years ago
Owen Diehl	245fe8e022	roundtrip v2 encoding tests	6 years ago
Cyril Tovena	549250c30d	Sets the chunk encoding correctly when creating chunk from bytes. Previously only the reader and writer for the encoding was set, on subsequent serialization the format would be missing, this only happens if the chunk is created from a chunk transfer. Because it uses NewBytesChunk to load and also write. I've also added a regression tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	7654c27c12	Adds configurable compression algorithms for chunks (#1411 ) * Adds L4Z encoding. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds encoding benchmarks Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds snappy encoding. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds chunk size test Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds snappy v2 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve benchmarks Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove chunkenc Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update lz4 to latest master version. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use temporary buffer in serialise method to avoid allocations when doing string -> byte conversion. It also makes code little more readable. We pool those buffers for reuse. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Added gzip -1 for comparison. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Initialize reader and buffered reader lazily. This helps with reader/buffered reader reuse. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Don't keep entries, extracted generateData function (mostly to get more understandable profile) Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Improve test and benchmark to cover all encodings. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds support for a new chunk format with encoding info. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Ingesters now support encoding config. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add support for no compression. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add docs Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove default Gzip for ByteChunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Removes none, snappyv2 and gzip-1 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Move log test lines to testdata and add supported encoding stringer Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	041b612bdf	Add Joe as member of the team. (#1415 ) Welcome !	6 years ago
Cyril Tovena	b1e232c007	Adds tenant-id for docker driver. (#1414 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Peter Štibraný	4fea713b1e	pkg/promtail/client: missing URL in client returns error (#1416 ) Previously it would just panic, because URL.Host is used to construct logger. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Peter Štibraný	f6be636e13	RangeQuery benchmark optimizations (#1413 ) * Added some duplicate streams. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Instead of using sort.Sort(), we keep tuples sorted already. Also just return first element if there is only one tuple. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Don't do time comparison twice, compare nanoseconds instead. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Sebastian Widmer	f0f6f24926	pkg/promtail: IETF Syslog (RFC5424) Support (#1275 ) * IETF syslog (rfc5254) support * Upgrade go-syslog to correctly released v2.0.1 * Incorporate feedback from new linter * Update documentation with relevant RFC sections. * Incorporate feedback from review * Use context to shutdown server. * Use util.Backoff from cortex * Do not embed mutex into TestLabeledClient * Use strings.HasPrefix * Better naming of connectionsClosed -> openConnections * Incorporate further feedback from review * Callback instead of channel in ParseStream. Removes one running Goroutine per connection. * Improve parse error log level. * Move mutex above field it protects. * Use backoff.Ongoing() * Switch to "nontransparent" parser * Further improvements to the documentation	6 years ago
Terje Sannum	9420fb10fd	helm: support adding init containers to the loki pod (#1336 )	6 years ago
Chance Zibolski	904bf2fcc9	Fix example AWS storage configuration (#1394 ) `store` must be `aws`, not `dynamo`. Without this change, the previous example configuration causes Loki to error at startup: ``` level=error ts=2019-12-10T00:27:54.489032373Z caller=main.go:66 msg="error initialising loki" err="error initialising module: table-manager: Unrecognized storage client dynamo, choose one of: aws, cassandra, inmemory, gcp, bigtable, bigtable-hashed" ```	6 years ago
Peter Štibraný	3aff4dfe2f	pkg/cfg: print help only when requested, and print it on stdout (#1396 ) When invalid parameter or value is provided, error is printed to stderr, together with message indicating how to get list of all parameters. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
vishesh92	c4e669d928	Docs: Add configuration docs for redis (#1399 )	6 years ago
Ed Welch	f9b25968f7	change the chunk size histogram to allow for bigger buckets (#1409 ) * change the chunk size histogram to allow for bigger buckets Signed-off-by: Edward Welch <edward.welch@grafana.com> * changing buckets again Signed-off-by: Edward Welch <edward.welch@grafana.com>	6 years ago
Edward Welch	591905ed64	forgot to register the new metric Signed-off-by: Edward Welch <edward.welch@grafana.com>	6 years ago
Ed Welch	5aa82ba199	allow configuring a target chunk size in compressed bytes (#1406 ) * allow configuring a target chunk size in compressed bytes Signed-off-by: Edward Welch <edward.welch@grafana.com> * addressing feedback Signed-off-by: Edward Welch <edward.welch@grafana.com>	6 years ago
Peter Štibraný	7bb77ecdd8	Convert string to bytes once only when doing string filtering. (#1405 ) This makes it slightly faster: name old time/op new time/op delta ContainsFilter-4 18.2ns ± 2% 9.8ns ± 1% -46.20% (p=0.000 n=10+8) name old alloc/op new alloc/op delta ContainsFilter-4 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta ContainsFilter-4 0.00 0.00 ~ (all equal) Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Robert Fratto	7b834c5ce1	Update release instructions and release script (#1391 ) * fix tools/release_prepare.sh so sed works on Linux * docs: expand release instructions * docs: add line describing releasing instructions	6 years ago
Robert Fratto	bc3da94f1a	v1.2.0: update helm and ksonnet (#1389 )	6 years ago
Robert Fratto	ccef3da2b6	Update changelog for 1.2.0 release (#1388 ) * Update changelog for 1.2.0 release * Update CHANGELOG.md Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: update reference to release tag to v1.2.0	6 years ago
Cyril Tovena	1fc0ffde8a	Logql benchmark and performance improvement. (#1371 ) * Add logql benchmark test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes store benchmark limits. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add LogQL improvements. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add back logging in the storage benchmark Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Edward Welch	ea5eec2abf	respond to label queries at `loki/api/v1/labels` instead of `loki/api/v1/label`	6 years ago
Owen Diehl	5227f5bee0	application/x-www-form-urlencoded support (#1381 ) * application/x-www-form-urlencoded support * handler func instead of unnecessary struct	6 years ago
Peter Štibraný	2da8d6f4b8	docs: Fix grpc_listen_host and http_listen_host. (#1353 ) They should be grpc_listen_address and http_listen_address respectively. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Peter Štibraný	dd96fa146c	pkg/ingester: check that ingester is in LEAVING state when transferring chunks and claiming tokens. Required when using memberlist client. (#1300 ) * On chunks transfer, verify that source ingester is LEAVING This is important when using gossiping ring. If source ingester is not leaving yet, than claiming its tokens will fail (merge function will not move ownership to the target ingester), and target ingester will end up with no tokens in the ring. * Register ring to prometheus. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Update check using latest version from Cortex. There is no direct usage of logging, which is consistent with the rest of this method. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Move import to group of imports with third-party packages. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Labesse Kévin	17b2ce0700	documentation: iam requirement for autoscaling (#1350 )	6 years ago
Johannes Staffans	7b2fde3e58	Fluentd: guard against nil values when sanitizing labels (#1376 ) * Guard against nil values when sanitizing labels * Gem version bump	6 years ago
Cyril Tovena	3ffa385b1e	Let Loki start when using the debug image. (#1372 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Peter Štibraný	e0ec61f986	pkg/ingester: handle labels mapping to the same fast fingerprint. (#1247 ) * pkg/ingester: handle labels mapping to the same fast fingerprint. Uses slightly adapted fpMapper code from Cortex. Fixes issue #898 Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * empty commit to force new build Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed empty lines in imports. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Added test that pushes entries concurrently. To be run with -race. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Stream now keeps both original and mapped fingerprint Mapped fingerprint is used in streams map. Original ("rawFP") is preserved when saving chunks. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Vendoring Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Reverted previous commit (keep the test, with updated checks). Preserving raw fingerprints could lead to data loss when doing chunk deduplication while storing chunks. We don't want that. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Cyril Tovena	2b7f375a21	Fixes fluentd new push path API. (#1363 ) * Update fluentd to support the new v1 push API. - also add tests for ordering. - improve nanoseconds precision. - fixes all lint/style issue. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Bump the gem and add myself as author. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Edward Welch	a8a143b578	Updating version numbers for 1.1.0 release	6 years ago
Ed Welch	191f87a376	Adding `zip` to the build image (#1364 ) Adding `drone` to the build image and updating the make target to run the drone command in the build image Re-adding `GO111MODULE=on` to the dependency import as it is still required and was removed in error. (Also updated the comment a little to reflect why it is still needed)	6 years ago
Ed Welch	28009050b4	Update changelog for 1.1.0 release (#1360 ) * Update changelog for 1.1.0 release	6 years ago
Cyril Tovena	f7ee1c753c	Supports same duration format in LogQL as Prometheus Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Daniel D'Abate	df89859369	HTTP API: Support duration and float formats for step parameter (#1355 ) * Support duration and float formats for step parameter in http api * Support Prometheus duration format and add missing tests * Fixes linter Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Xiang Dai	9afda2c81e	doc(promtail): fix prometheus link (#1333 ) Signed-off-by: Xiang Dai <764524258@qq.com>	6 years ago
Xiang Dai	1dcf3d371b	chore(/release): distribute binaries as .zip (#1160 ) Switches from `.gz` to `.zip` for the compressed binaries, as this format is more common across operating systems (works out of the box on probably all of them, including tooling such as Puppet, Ansible, etc.)	6 years ago
Adam Johnson	8c27cca496	Update loki helm chart to support service account annotations (#1341 ) * update loki helm chart to support service account annotations for use with EKS IAM roles Signed-off-by: Adam Johnson <a.johnson@winton.com> * Bump minor versions and fix serviceAccount reference for annotations Signed-off-by: Adam Johnson <a.johnson@winton.com>	6 years ago

... 2 3 4 5 6 ...

1252 Commits (f9ff2a4b02832c7495f890abd2d73f4c8d44f1af) All Branches Search

1252 Commits (f9ff2a4b02832c7495f890abd2d73f4c8d44f1af)

All Branches