1252 Commits (f9ff2a4b02832c7495f890abd2d73f4c8d44f1af)

 

All Branches   

Search

Author	SHA1	Message	Date
Cyril Tovena	4d853b36e5	Adds support for the old query string regexp in the frontend. (#1522) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Robert Fratto	bcbb1f294e	docs: Document timestamp ordering rules (#1521) ... Co-authored-by: Ed Welch <ed@oqqer.com>	5 years ago
Robert Fratto	65013218b7	pkg/chunkenc: ignore duplicate lines pushed to a stream (#1519) ... * pkg/chunkenc: ignore duplicate lines pushed to a stream This commit changes the behavior of appending to a stream to ignore an incoming line if its timestamp and contents match the previous line received. This should reduce the chances of ingesters storing duplicate log lines when clients retry push requests whenever a 500 is returned. Fixes #1517	5 years ago
Benjamin Huo	9f9d05efbf	Fix error 'Field does not exist: jaeger_mixin' in tk show (#1518) ... Signed-off-by: Benjamin <benjamin@yunify.com>	5 years ago
Edward Welch	b43039ed5a	update the position and size metrics at the same time to try to end the race between these two metrics and the two timers that care about position and size ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Robert Fratto	5511de88b5	Link to release docs in README.md, not master docs (#1516) ... Co-authored-by: Ed Welch <ed@oqqer.com>	5 years ago
Chance Zibolski	3a2e5d4bbc	pkg/promtail: Initialize counters to 0 when creating client (#1453) ... This ensures metrics are not missing before they're incremented the first time, as recommended by the upstream Prometheus instrumentation guidelines: https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics. Currently, in the case of dropped_entries_total and dropped_bytes_total, the metric is simply missing until the first occurrence of an entry being dropped.	5 years ago
Sandeep Sukhani	fbcfad13dc	fix panic in tailer when max tail duration exceeds (#1511)	5 years ago
Cyril Tovena	5739150136	Fixes bad json in Loki API documentation. (#1508) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Sandeep Sukhani	f1d431778b	fix sample yaml in docs for installing promtail to k8s (#1505)	5 years ago
petruha	4f314f3df4	Fileglob parsing fixes (#1485) ... * Update doublestar module to version 1.2.2. * Add absolute path for each element in list. * Remove doublestar test and update vendore module version.	5 years ago
myoung-su,shin	073258c098	jsonnet: fix common/defaultPorts parameters (#1501)	5 years ago
Robert Fratto	9227eed9f4	pkg/ingester: limit total number of errors a stream can return on push (#1071) ... * pkg/ingester: limit total number of errors a stream can return on push * pkg/ingester: s/max_ignored_stream_errors/max_returned_stream_errors	5 years ago
OlivierB	0bd46200a0	Allow to define custom parsers to use with fluentbit.io/parser annotation (#1430) ... Signed-off-by: Olivier Boudet <oboudet@gmail.com>	5 years ago
Cyril Tovena	c09a01d7c3	Bump fluentd plugin to 1.2.6. (#1500) ... This was ommitted in the previous PR https://github.com/grafana/loki/pull/1475. Unfortunatelly I need a new version to publish the new gem. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Beorn Facchini	88681a9974	Read websocket close in tail handler (#1383) ... * Read websocket close in tail handler * Send close message to websocket on quit signals	5 years ago
Yuvraj	8369f7467c	Helm chart url added in helm.md (#1418)	5 years ago
Cyril Tovena	f3332272ed	Update Loki mixin to include frontend QPS and latency. (#1497) ... Also rework frontend route registration to name metrics and spans correctly. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Ed Welch	35dd07c4e0	fixes a panic where prefetch could try to operate on a nil object (#1499) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Marco Pracucci	ec40515d31	Added max streams per user global limit (#1493) ... * Added max streams per user global limit Signed-off-by: Marco Pracucci <marco@pracucci.com> * Updated changelog Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Marco Pracucci	ce407d3276	Add ingestion rate global limit support (#1486) ... * Added ingestion rate global limit support Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed code comments Signed-off-by: Marco Pracucci <marco@pracucci.com> * Updated config doc Signed-off-by: Marco Pracucci <marco@pracucci.com> * Updated changelog Signed-off-by: Marco Pracucci <marco@pracucci.com> * Updated doc Signed-off-by: Marco Pracucci <marco@pracucci.com>	5 years ago
Cyril Tovena	33f70b70ce	Add duplicates info and remove timing informations. (#1496) ... * Add duplicates info and remove timing informations. Timing information in the chunkenc iterator are too eating too much resources. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Yevhen Stolietnii	616843b7e0	Helm: fix issue with config.clients (#1431) ... * Add statement for config.clients * Bump up chart version * Update production/helm/promtail/templates/daemonset.yaml Yeap, make sense. Co-Authored-By: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	5 years ago
Matej	aa09aa5f34	fluentd-plugin: call gsub for strings only (#1475) ... * Use gsub only for string variables * Delete version change	5 years ago
Ed Welch	7c4906786d	pre-fetch the next chunk in parallel to processing the current chunk such that we aren't waiting for fetches from GCS and spend all our CPU time decompressing and processing (#1495) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Marco Pracucci	fa63c09418	Fixed TOUCH_PROTOS in all DroneCI pipelines (#1494) ... * Fixed TOUCH_PROTOS in all DroneCI pipelines * Fixed drone.yaml generation Signed-off-by: Marco Pracucci <marco@pracucci.com>	5 years ago
Owen Diehl	a1d090e161	Fix/deadlock frontend queue (#1490) ... * buffers splitby interval channels to prevent deadlocks * improves codec merge test requiring merging from multiple intervals * bound concurrency by min(parallelism,jobs) * approximates goroutine test	5 years ago
Owen Diehl	7cc93b2694	unifies reverse iterators (#1489)	5 years ago
Cyril Tovena	589e53d844	Fixes response json encoding and add regression test. (#1488) ... Also align stream labels alphabetical order with querier. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	6c64e2ca6b	Fixes the typo in the result cache config of the Loki ksonnet lib. (#1478) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	be010f049d	Close iterator properly and check nil before releasing buffers. (#1480) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Terje Sannum	8de2bc6b3d	docs: fix broken promtail link	5 years ago
Labesse Kévin	18faf99fcb	<doc>: information about max_look_back_period (#1474) ... * <doc>: information about max_look_back_period * update example * added a note to retention doc and fixed a spell error Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	5 years ago
Owen Diehl	29e4ad8cf8	TOUCH_PROTOS build arg for dockerfile (#1479) ... * TOUCH_PROTOS build arg for dockerfile * echos touch protos in dockerfile * touch protos correct var * make touch-protos options in all dockerfiles, run for amd builds	5 years ago
Owen Diehl	9e9652a3b8	initiates docker daemon for circle windows builds (#1476) ... * initiates docker daemon for circle windows builds * touch-protos amke target, circle windows build uses it * touch-protos doenst explicitly depend on $(PROTO_GOS)	5 years ago
Cyril Tovena	4153740a12	Loki Query Frontend (#1442) ... * Adds frontend to Loki. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improves tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes sneaky bug in entries sorting. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes the split by interval. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Tweak jsonnet deployments and add a way to lint/fmt. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fix timezone issue. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve tests and rollback change in loghttp package. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a flaky test that might run one more goroutine. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes windows build. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve tracing in the split by interval. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add test stream to proto conversion. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes flappy retry test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove err shadowing in stopQueryFrontend as it was confusing. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor grpc message size in the libsonnet config file. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Don't check auth header for GRPC TransferChunks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Query frontend (#3) * frontend codec merging optimizations * codec benchmarks * removes unused bounds code in queryrange ordering * [wip] splitby uses channels instead of sub batching intervals * splitBy channel limit test * single allocation for merging entries from a single stream * skip merging loki responses when limit is already hit * removes checks for unlimited queries in queryrange * removes splitByInterval{,.interval} spans * removes interval_batch_size from jsonnet lib * moves benchmark utils to own file * renames markers -> entries * priority queue comments * Removes unused logRequest. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Sets the cache interval to the same split interval. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Missing import libsonnet for the frontend. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Frontend should not be a cluster IP. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	5 years ago
Owen Diehl	0b72ddb6a6	positions.ignore-corruptions (#1472) ... * positions.ignore-corruptions * semantic change: s/ignore-corruptions/ignore-invalid-yaml/ * Update pkg/promtail/positions/positions.go Co-Authored-By: Robert Fratto <robertfratto@gmail.com>	5 years ago
Robert Fratto	52931a1bc5	pkg/querier: don't query all ingesters (#1473) ... This commit uses ReplicationSet.Do for communicating with the ingesters from the queriers. This mimics the behavior of Cortex. An extra_query_delay flag has been added to match the flag present in Cortex's querier config. Fixes #1447.	5 years ago
Nathan Zender	282d77b690	Using strict parsing for yaml configs (#1433)	5 years ago
Labesse Kévin	4f2316ff74	<documentation>: Update IAM requirement (#1466) ... * update iam requirement * move ListTables in correct section	5 years ago
Jens Erat	557f9ccd7b	fluent-bit: multi-instance support (#1294) ... To run multiple plugin instances at the same time, the plugin instance must be registered and later retrieved. Additionally, a list of registered plugin instances must be stored for proper disposal during fluent-bit shutdown. Based on the [out_multiinstance example] in the upstream repository. [out_multiinstance example]: fc386d2638/examples/out_multiinstance/out.go Signed-off-by: Jens Erat <email@jenserat.de> Co-authored-by: Hiroshi Hatake <cosmo0920.oucc@gmail.com>	5 years ago
Jens Erat	c9016f4b2a	fluent-bit-plugin: re-enable JSON marshal tests (#1455) ... My upstream bugfix to `json-iterator/go` was accepted. This commit updates to the newest upstream release, adds checks for the returned error message and reenables the commented-out JSON marshaller tests. Re-enabling the JSON marshaller tests that should return an error message, I realized `sendMessage` logged and ignored marshalling errors instead of passing them to the caller. Signed-off-by: Jens Erat <email@jenserat.de>	5 years ago
Cyril Tovena	d9d5c989ee	Update README.md (#1471)	5 years ago
Cyril Tovena	10b5bcab72	Validates limit parameter. (#1470) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Robert Fratto	9b264e5b2e	Makefile: re-enable journal scraping on ARM (#1469) ... Drone should be able to handle this since it has native ARM machines. Fixes #1459.	5 years ago
Owen Diehl	dafb9d8425	Enables Series API in loki (#1419) ... * helpers for parsing /series requests * logproto.SeriesResponse impl * passes unparsed matchers in protobuf for simplicity * ingester support for series queries * labels map coercion * fixes label match loghttp test * removes labelmatcher protobuf adapters * series api impl * tests matchers for dispatching series queries * series looked up serially for /series api * splits series api async code and remove unnecessary type assertion * adds series handle to /api/prom/series and includes docs * compatibility changes w/ upstream merge	5 years ago
Cyril Tovena	fb76416b27	Add decompression tracing instrumentation. (#1445) ... * Add decompression tracing instrumentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix time conversion. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Use span logger so we also get a log line. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve reusability and add fetched chunks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	768bf914e0	Changes the encoding used for the benchmark in the store. (#1448) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
vtereso	c1fe0fd85b	Update README.md (#1441) ... Spelling fix	5 years ago
Peter Štibraný	4381b28c80	pkg/chunkenc: BenchmarkRead focuses on reading chunks (#1423) ... Previously BenchmarkRead asked for every single line from chunks to be returned. This causes a lot of unnnecessary allocations, which dominate the benchmark. Instead of counting bytes when reading, we now count size when generating data for logging speed. Another test was added to show that these two approaches are comparable. This change makes BenchmarkRead to report real time needed to decode chunks: name old time/op new time/op delta Read/none-4 86.2ms ± 0% 33.2ms ± 0% ~ (p=1.000 n=1+1) Read/gzip-4 255ms ± 0% 194ms ± 0% ~ (p=1.000 n=1+1) Read/lz4-4 121ms ± 0% 64ms ± 0% ~ (p=1.000 n=1+1) Read/snappy-4 119ms ± 0% 67ms ± 0% ~ (p=1.000 n=1+1) name old alloc/op new alloc/op delta Read/none-4 134MB ± 0% 0MB ± 0% ~ (p=1.000 n=1+1) Read/gzip-4 135MB ± 0% 0MB ± 0% ~ (p=1.000 n=1+1) Read/lz4-4 136MB ± 0% 1MB ± 0% ~ (p=1.000 n=1+1) Read/snappy-4 135MB ± 0% 0MB ± 0% ~ (p=1.000 n=1+1) name old allocs/op new allocs/op delta Read/none-4 491k ± 0% 3k ± 0% ~ (p=1.000 n=1+1) Read/gzip-4 491k ± 0% 3k ± 0% ~ (p=1.000 n=1+1) Read/lz4-4 491k ± 0% 3k ± 0% ~ (p=1.000 n=1+1) Read/snappy-4 491k ± 0% 3k ± 0% ~ (p=1.000 n=1+1) Decompression speed is now also more correct (esp. None is much higher than LZ4/Snappy) Before (/s) Now (/s) None 1.1 GB (n=1) 4.0 GB (n=1) None 1.5 GB (n=9) 3.8 GB (n=38) None 1.6 GB (n=13) Gzip 516 MB (n=1) 640 MB (n=1) Gzip 509 MB (n=3) 664 MB (n=4) Gzip 514 MB (n=4) 649 MB (n=6) LZ4 1.1 GB (n=1) 1.7 GB (n=1) LZ4 1.1 GB (n=9) 1.9 GB (n=15) Snappy 1.1 GB (n=1) 2.0 GB (n=1) Snappy 1.1 GB (n=9) 1.8 GB (n=16) Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago