1623 Commits (096976511f134f027efbb2ca19b4fe419de60526)

 

All Branches   

Search

Author	SHA1	Message	Date
Or Sela	8f9e456535	helm/loki-stack: render loki datasource only if grafana is enabled (#2086)	5 years ago
James Hollowell	10a1f28a85	Support Prometheus on a sub-path in Grafana config (#2099) ... Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Aditya C S	5e0fe09f69	Fix configuration docs (#2139)	5 years ago
beorn7	a5aa34137a	Adjust Loki dashboards to the new label order ... - container_name and pod_name have disappeared in K8s 1.16. - We are changing the instance name in Prometheus (and removing it in Loki altogether). The instance name used to be the pod name, so switch usage of the instance label to the pod label. The pod label will generally only appear as a target label with rolling out recent changes to target labels. This will unfortunately break dashboards if looking at older data.	5 years ago
beorn7	123e6f51fe	Keep scrape config in line with the new Prometheus scrape config ... This is triggered by https://github.com/grafana/jsonnet-libs/pull/261 . The above PR removes the `instance` label. As it has turned out (see PR linked above), a sane `instance` label in Prometheus has to be unique, and that includes the case where a single container exposes metrics on two different endpoints. However, that scenario would still only result in one log stream for Loki to scrape. Therefore, Loki and Prometheus need to sync via target labels uniquely identifying a container (rather than a metrics endpoint). Those labels are namespace, pod, container, also added here. This commit removes the `container_name` label. It is the same as the `container` label and was already added to Loki previously. However, the `container_name` label is deprecated and has disappeared in K8s 1.16, so that it will soon become useless for direct joining.	5 years ago
Owen Diehl	89d80a6fc4	Feature/querysharding ii (#1927) ... * [wip] sharding evaluator/ast * [wip] continues experimenting with ast mapping * refactoring in preparation for binops * evaluators can pass state to other evaluators * compiler alignment * Evaluator method renamed to StepEvaluator * chained evaluator impl * tidying up sharding code * handling for ConcatSampleExpr * downstream iterator * structure for downstreaming asts * outlines sharding optimizations * work on sharding mapper * ast sharding optimizations * test for different logrange positions * shard mapper tests * stronger ast sharding & tests * shardmapper tests for string->string * removes sharding evaluator code * removes unused ctx arg * Revert "removes sharding evaluator code" This reverts commit 55d41b9519. * interfaces for downstreaming, type conversions * sharding plumbing on frontend * type alignment in queryrange to downstream sharded queriers * downstreaming support for sharding incl storage code * removes chainedevaluator * comment alignment * storage shard injection * speccing out testware for sharding equivalence * [wip] shared engine refactor * sorting streams, sharding eval fixes * downstream evaluator embeds defaultevaluator * other pkgs adopt logql changes * metrics & logs use same middleware instantiation process * wires up shardingware * middleware per metrics/logfilter * empty step populating StepEvaluator promql.Matrix adapter * sharding metrics * log/span injection into sharded engine * sharding metrics avoids multiple instantiation * downstreamhandler tracing * sharding parameterized libsonnet * removes querier replicas * default 32 concurrency for workers * jsonnet correct level override * unquote true in yaml * lowercase error + downstreamEvaluator defaults to embedded defaultEvaluator * makes shardRecorder private * logs query on failed parse * refactors engine to be multi-use, minimizes logger injection, generalizes Query methods, removes Engine interface * basic tests for querysharding mware * [wip] concurrent evaluator * integrates stat propagation into sharding evaluator * splitby histogram * extends le bounds for bytes processed * byte throughput histogram buckets to 40gb * chunk duration mixin * fixes merge w/ field rename * derives logger in sharded engine via ctx & logs some downstream evaluators * moves sharded engine to top, adds comments * logs failed merge results in stats ctx * snapshotting stats merge logic is done more effectively * per query concurrency controlled via downstreamer * unexports decodereq * queryrange testware * downstreamer tests * pr requests	5 years ago
Frederik Nordahl Jul Sabroe	156023ae9a	Allow configuring more options for output configuration (#2089) ... * Allow configuring more options for output configuration The `TenantID`, `BatchWait`, `BatchSize` and `AutoKubernetesLabels` options in the fluent-bit loki output plugin were not configurable. This change allows configuring those options via the helm chart. * typo in production/helm/fluent-bit/README.md Co-authored-by: Joseph Petersen <josephp90@gmail.com> Co-authored-by: Joseph Petersen <josephp90@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Jérôme Loyet	665712ff18	Add Entry variable to template (#2105) ... The new Entry variable is the log message that could be needed in template. For instance, if I want to convert all my log line to lowercase: - template: source: message template: '{{ ToLower .Entry }}'	5 years ago
Hassan Farid	b9f276cfed	updated documentation links to relative. (#2135) ... * updated documentation links to relative. The documentation is pointing to mast always, and is difficult and in-accurate to navigate for specific tag / version. * fixed relative path	5 years ago
Sebastian Widmer	d175fa58c8	Update go-syslog dependency to accept any encoding in message (#2127)	5 years ago
Aditya C S	ff0a4c35f4	Set JournalTarget Priority value to keyword (#2087)	5 years ago
i-takizawa	8cbeb53cf1	docs: make <placeholders> visible (#2092) ... * Fix on docs: make <placeholders> visible Added backslashes to make<placeholders> visible in parsed text. * docs: make it look a bit better	5 years ago
nileshcs	a0546a3bc3	Documentation: Outdated fluentd image name, UID details, link update (#2112) ... * documentation update: Fluentd docker image name, UID for docker image, and link to installing as container * Reverted to fix the name of the Ruby gem while keeping name change for Docker image	5 years ago
Aditya C S	f984e71192	Fix Promtail journal seeking known position (#2111)	5 years ago
Aditya C S	360d9a7f97	Support querying labels on time range in logcli (#2083)	5 years ago
Ed Welch	26a4c53cbd	updating all the images to 1.5.0 release (#2100) ... * updating all the images to 1.5.0 release Signed-off-by: Ed Welch <edward.welch@grafana.com> * duration must have a unit Signed-off-by: Ed Welch <edward.welch@grafana.com>	5 years ago
Ed Welch	12c7eab8bb	Prep 1.5.0 release (#2098) ... * Updating the Changelog and Upgrade guide for 1.5.0 release. Signed-off-by: Ed Welch <edward.welch@grafana.com> * Changing release number in all the docs Signed-off-by: Ed Welch <edward.welch@grafana.com>	5 years ago
Owen Diehl	13424a20ce	updates loki config examples (#2097)	5 years ago
Cyril Tovena	f9a2c436ac	Adds backtick for the quoted string token lexer. (#2095) ... * Fixes unit in stats request log. (#2093) * Fix unit conversion in stats log. 🤦 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix the tests because you know ! you know ? Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add backtick for the quoted string token lexer. This allows to pass unescaped string to Loki. For example we now support this : `` {name="cassandra"} |~ `error=\w+` `` `` {name!~`mysql-\d+`} `` As you can see no escape is required when using ` quoted strings. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes newlines. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add back missing test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	c81be6603e	Fixes unit in stats request log. (#2093) ... * Fix unit conversion in stats log. 🤦 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix the tests because you know ! you know ? Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Ed Welch	0520a0ac1c	allow no encoding/compression on chunks (#2088) ... Signed-off-by: Ed Welch <edward.welch@grafana.com>	5 years ago
Björn Rabenstein	d343c09aa9	Accept suffixes to pod name in instance labels (#2080) ... Currently, the instance label is just the pod name, but that's insufficient, cf. https://github.com/grafana/deployment_tools/issues/3441 . In the near future, we'll suffix the pod name in the instance label with container name and/or port name. To make this mixin still work, adjust the `instance` label matcher. Signed-off-by: beorn7 <beorn@grafana.com>	5 years ago
Owen Diehl	e0dacd2795	removes yolostring (#2078)	5 years ago
Or Sela	835dd29316	fluentd plugin: support placeholders in tenant field (#2074)	5 years ago
Cyril Tovena	0de1a42406	Fixes a panic in dry-run when using external labels. (#2075) ... Fixes #1972. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	c2f100c334	Allows to pass inlined pipeline stages to the docker driver. (#2076) ... * Allows to pass inlined pipeline stages to the docker driver. Fixes #1296 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * doc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Ed Welch	9803eab947	Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper (#2073) ... * Allow configuring query_store_max_look_back_period when running a filesystem store, which allows for boltdb-shipper to horizontally scale filesystem store. Signed-off-by: Ed Welch <edward.welch@grafana.com> * Update docs/operations/storage/boltdb-shipper.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * add some more precision to the timestamp used to make the name unique just to make sure there are not collisions if for some reason the ID is empty (which can happen if the hostname doesn't resolve for some reason) Signed-off-by: Ed Welch <edward.welch@grafana.com> * Allow the ingester to query the store for labels if query store is enabled. Signed-off-by: Ed Welch <edward.welch@grafana.com> * make queriers aware of the fact that ingesters may be querying the store so they don't duplicate query efforts Signed-off-by: Ed Welch <edward.welch@grafana.com> * improve the docs Signed-off-by: Ed Welch <edward.welch@grafana.com> * add tests to the calculateMaxLookBack Signed-off-by: Ed Welch <edward.welch@grafana.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	5 years ago
Ed Welch	7dd109763c	expand the match_all to also optionally allow counting bytes per stream (#2066) ... Signed-off-by: Ed Welch <edward.welch@grafana.com>	5 years ago
Sandeep Sukhani	3a28a17946	some metrics for measuring performance and failures in boltdb shipper (#2034) ... * some metrics for measuring performance and failures in boltdb shipper * counting both failures and success in upload and download operation in boltdb shipper * changes suggested from PR review	5 years ago
Ed Welch	f206165c4f	output a debug log if the requested path in the scrape config doesn't match any files (#2069) ... Signed-off-by: Ed Welch <edward.welch@grafana.com>	5 years ago
Aditya C S	11937d3c3a	Targets not required in promtail config (#2026) ... * Targets not required in promtail config * changing the defaulted value from `dummy` to `localhost`, I think this will be more intuitive for people, also re-worded the docs a bit. Signed-off-by: Ed Welch <edward.welch@grafana.com> Co-authored-by: Ed Welch <edward.welch@grafana.com>	5 years ago
Aditya C S	199e334126	Update loki to cortex master (#2030)	5 years ago
Cyril Tovena	a8b94aec3d	Reverse entry iterator pool (#2064) ... * wi Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	9988ce4b9f	Improve protobuf serialization (#2031) ... * Generate custom proto type for Streams. This is the first step to improve protobuf serialization. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve protobuf serialization allocations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Saves allocations by stringifying labels only once. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	bce4470a5b	Recover from panic in http and grpc handlers. (#2059) ... * Recover from panic in http and grpc handlers. I don't see any good reason to crash any component during a bad request. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add alerts to the mixin for panics. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * 😡 gomod Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Bohdan Kmit	f5b9cffc2e	Docker driver: Allow to provision external pipeline files to plugin (#2054) ... * Use latest Alpine version * Allow to provision external pipeline files to plugin * Add custom pipeline documentation * Untabify on reviewer request Co-authored-by: Mike Futerko <mike@maytech.net>	5 years ago
Cyril Tovena	fcb4c763dc	Fix a bug in range vector skipping data. (#2058) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Naoki Oketani	918ba41982	Docs: update Loki Helm Chart document to support Helm 3 (#2038) ... * update Loki Helm Chart document to support Helm 3 * update Loki Helm Chart document to support Helm 2 & 3	5 years ago
Aditya C S	83873d1c6f	Fix promtail client default values (#2049)	5 years ago
Arve Knudsen	d975b1fdd8	Fix spelling of per second (#2055) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	5 years ago
Cyril Tovena	4d5c2787b9	Update api.md (#2056) ... Vector timestamp are in seconds.	5 years ago
Arve Knudsen	cc98d10d74	Chore: Correct typo "per seconds" (#2050) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	5 years ago
Goutham Veeramachaneni	06b50590a7	Fix bug in logql parsing that leads to crash. (#2046) ... * Fix bug in logql parsing that leads to crash. ``` panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x20ae356] goroutine 39530021 [running]: github.com/grafana/loki/pkg/logql.andFilter.Filter(0x0, 0x0, 0x3beffc0, 0xc01f065bc0, 0xc01aaaec00, 0x13d, 0x200, 0xffffffffffffffff) /src/loki/pkg/logql/filter.go:66 +0x26 github.com/grafana/loki/pkg/chunkenc.(*bufferedIterator).Next(0xc00a48fba0, 0x0) /src/loki/pkg/chunkenc/memchunk.go:616 +0x1cb github.com/grafana/loki/pkg/iter.(*nonOverlappingIterator).Next(0xc00130ae40, 0x0) /src/loki/pkg/iter/iterator.go:438 +0x135 github.com/grafana/loki/pkg/iter.(*timeRangedIterator).Next(0xc00130ae80, 0x0) /src/loki/pkg/iter/iterator.go:497 +0x48 github.com/grafana/loki/pkg/iter.(*reverseIterator).load(0xc00072ac40) /src/loki/pkg/iter/iterator.go:555 +0x70 github.com/grafana/loki/pkg/iter.(*reverseIterator).Next(0xc00072ac40, 0x0) /src/loki/pkg/iter/iterator.go:563 +0x2f github.com/grafana/loki/pkg/iter.(*nonOverlappingIterator).Next(0xc00130af40, 0x10) /src/loki/pkg/iter/iterator.go:438 +0x135 ``` This is caused by people doing `|= ""` which would result in parsing of TrueFilter but would be returned as nil. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Add tests for the panic Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	5 years ago
Ed Welch	2629e9a3d6	fixes the cpu usage graphs (#2044) ... Signed-off-by: Ed Welch <edward.welch@grafana.com>	5 years ago
avii-ridge	69c1d1c0e0	Add extraOutputs variable to support multiple outputs for fluent-bit (#2040) ... * Adding support for extra outputs * added documentation	5 years ago
Joe Elliott	236d3728d5	Swapped to container restarts over terminated reasons (#2043) ... Signed-off-by: Joe Elliott <number101010@gmail.com>	5 years ago
Robert Fratto	f4c884f2f6	Remove generated comment at top of .drone/drone.yaml (#2042)	5 years ago
Ed Welch	fd99dbc926	Dashboard: Loki Operational improvements (#2041) ... * fixing indentation so I can see diffs better Signed-off-by: Ed Welch <edward.welch@grafana.com> * add rows for chunks, streams, dropped samples. success rate graphs now only look at 5xx errors (4xx are considered "successful") Limited the mouseover to only show the active line for a bunch of graphs. Signed-off-by: Ed Welch <edward.welch@grafana.com>	5 years ago
Cyril Tovena	4fd670dc7a	Support Post request in the frontend queryrange handler. (#2023) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Emil 'Skeen' Madsen	33a81528ec	fluent-plugin-loki: Restructuring and CI (#2006) ... * Move fluentd/fluent-plugin-grafana-loki to cmd/fluentd * Integrate fluentd with Makefile * Added drone-ci job for fluentd * Fix typo fluent-bit --> fluentd in .drone/drone.jsonnet Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> * Change default LOKI_URL in FluentD Dockerfile Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> * Utilize upstream configuration file * Test setup, adapt default configuration * Regen drone.yml * Change default label to job=fluentd, rather than env=dev Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago