516 Commits (e8cce09de29dc58c38b333e72dfe11613d49fbb3)

Author	SHA1	Message	Date
Owen Diehl	c6110bd93f	updates tanka installation docs (#3239)	4 years ago
Owen Diehl	224e68a50b	WAL backpressure (#3218) ... * adds WAL replay backpressure * backpressure fn doesnt lock during passed fn for concurrency * replayController * replayController supports a RemoveFlushedChunks fn * custom wal flusher * encoding test determinism * standardizes recoveredBytesTotal to replayControllers ownership * linting * default wal memory threshold to 4gb, adds docs * fix test after v3 chunk schema change * more lenient expectCheckpoint function * replayController protects against flush race condition * adds replay_memory_ceiling to wal jsonnet * replay ceiling help msg	4 years ago
Andreas Plesner	4c6f8e8149	docs: point to latest release for docker installation (#3235) ... Signed-off-by: Andreas Plesner <apj@mutt.dk>	4 years ago
Cyril Tovena	33abab5e42	Cortex update pre 1.7 (#3215) ... * Starting to update cortex. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes ruler overrides. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * go.mod Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Michel Hollands	2afebbc064	Add config endpoint (#3228) ... * Add config route Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add documentation Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	4 years ago
李国忠	0208071bbd	[doc] fix err. "loki_frontend" is invalid (#3220)	4 years ago
Kaviraj	3054d64377	doc(gcplog): Fix titles for Cloud provisioning for GCP logs (#3190)	4 years ago
Kaviraj	6cc41f92f8	Gcplog targetmanager (#3083) ... * Add new target manager(PubsubTarget) for promtail Lets you to scrape log entries from pubsub topic and send it to lok - Basic `Target` and `TargetManager` - Minimum config to make it work * make the receive loop on pubsub manager work * Add mmock pubsub server for testing * Remove debugging logs * Minor changes to messages pushing into loki * Basic formatter to convert GCP log entry to loki log entry * Reverting go.mod, go.sum and vendors to master branch * only pubsub dependencies * rebuild proto * Fix protobuf diffs * Remove debug printfs * Add basic formatter for GCP log entry - Create uniq label to create stream without out-of-order timestamp - extract instance id and timestamp from log entry * - Small fixes to formatter - Config changes on pubsub manager for testing * - Formatting changes - Rewrite timestamp to avoid out-of-order errors on loki - Minor config changes to Pubsub Target manager * Fix target formatter tests * PR remarks * Minor refactoring and following changes - Rename Pubsub* entities to Gcplog* - Rename pubsub package into gcplog - Add flag `KeepIncomingTimestamp` to toggle between either keeping incoming timestamps or rewriting with current timestamp - Tests for gcplog target - Docs and basic metrics to gcplog target * Add docs for cloud configure for gcplog. * Add docs for cloud configure for gcplog. * Review feedback * Checkin generated files * Making linters happy Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Hui Kang	1ffd4f641e	Doc: Add missing wal in configuration (#3162) ... * Doc: Add missing wal in configuration * Update docs/sources/configuration/_index.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update docs/sources/configuration/_index.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update docs/sources/configuration/_index.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	4 years ago
李国忠	54d1d3bea7	[doc] fix:querier do not handle "/flush" api (#3165)	4 years ago
Owen Diehl	322e4bc6fc	updates alerting docs post 2.0 (#3164) ... * updates alerting docs post 2.0 * 1s -> 10s	4 years ago
Hui Kang	1f6fd5e45e	Doc: add missing type supported by table manager (#3148)	4 years ago
Owen Diehl	6dc8bee9b0	ingester/shutdown -> ingester/flush_shutdown (#3146)	4 years ago
Cyril Tovena	5070605335	Fixes absent_over_time to work with all log selector. (#3143) ... * Fixes absent_over_time to work with all log selector. I made a mistake forcing absent_over_time to work only for unwrap, which is not the most useful usecase. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * actually add a test that cover the previous mistake :) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Karsten Jeschkies	776c9df9e3	Give another example for multiline. (#3138) ... * Give another example for multiline. Summary: This is a new example I've tested. It uses the zero-width space character `U+200B` to identify the first line of a multline block. * Update docs/sources/clients/promtail/stages/multiline.md Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> * Apply suggestions from code review Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	ed649eec37	Don't fail writes due to full WAL disk (#3136) ... * dont fail writes on full wal disk * wal full failure will cause flush on shutdown * logs the first full WAL failure	4 years ago
Danny Kopping	fcabfec470	Implementing line comments (#3126) ... * Implementing line comments Adding tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Further appeasement Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding tests to ensure that a # within a string is not incorrectly lexed as a comment Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding documentation Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Sam Coren	ca13e79f04	Update getting-in-touch.md (#3123) ... This page wasn't updated yet to reflect the community support channel shift from Grafana Community Slack -> Grafana Community Forums. Also added clarity re: support for Grafana Cloud-hosted Loki users and included community call information.	4 years ago
Cyril Tovena	fea5db12ed	Fixes LogQL documentation links. (#3128) ... * Fixes LogQL documentation ref links. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix old link. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
WuJie	ca344ca31a	fix time duration unit (#3124) ... time.ParseDuration does not support unit `d`	4 years ago
Val	5572366d93	Include instruction to enable variable expansion (#3115)	4 years ago
Kaviraj	3edcda744e	mention the use of `config.expand-env` flag in the doc. (#3078)	4 years ago
Owen Diehl	ee9c629f51	wal docs (#3026)	4 years ago
Andrea Scarpino	3640d2d4a8	Update reference to old helm chart repo (#3084)	4 years ago
nileshcs	92c7bb5513	Update docs for downstream_url (#3109)	4 years ago
Cyril Tovena	e83f809dd7	Small fixes for the documentation. (#3088) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Benjamin Huo	52afadc027	Fix storage guide links (#3094) ... Signed-off-by: Benjamin <benjamin@yunify.com>	4 years ago
Edward Welch	ff224c3f72	update changelog and upgrade guide for 2.1.0	4 years ago
Heds Simons	d08871d4c7	fluent-bit: Rename Fluent Bit plugin output name. (#2974) ... This prevents a clash with the Loki plugin now included with the Fluent Bit agent by default. The output plugin is now named `grafana-loki`. Signed-off-by: Heds Simons <heds@whaleway.net> (cherry picked from commit b195e94f39)	4 years ago
Torsten Walter	6c8ad5258b	Deprecate Charts as they have been moved (#2720) ... * Deprecate Charts as they have been moved Signed-off-by: Torsten Walter <torsten.walter@syncier.com> * update docs and links to new chart location Co-authored-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Cyril Tovena	6d85c7c212	Add absent_over_time (#3053) ... * Add absent_over_time This adds absent_over_time which is super useful when alerting, it can be combine with any logql selector including parsers and filter to returns a timeseries with 1 when logs go missing. Fixes #2893 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds comment. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	3f0800dc44	Add function label_replace. (#3047) ... You might be wondering why do we need label_replace when we already have label_format. Well for two reasons: 1. It's nice to have something similar to Prometheus, so users building metric queries will feel right at home. 2. Interestingly, they are quite different in how they execute. label_format executes for every log line, however label_replace execute for every sample. Since you aggregate log lines into sample then label_replace is way more efficient at scale. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	4e7a123c77	Support rate for unwrap expressions. (#3048) ... This allows to bytes throughput if there were printed in a log line. Examples : ```logql sum by (device) (rate({app="network-dev"} |= "192.16.8.1" | json | unwrap recv_bytes [1m])) ``` Closes #3044 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	2406ea72bc	Allows by/without to be empty and available for max/min_over_time (#3030) ... * Allows by/without to be empty and available for max/min_over_time operations. This essentially allows to aggregate over all dimensions when using by () while without() is a noop. This also makes it possible for max and min range vector aggregation to use grouping, which is simpler than doing max by (foo) max_over_time(...) Examples: - `max_over_time(...) by ()` gives your the max over time accross of all series. - `min_over_time(...) by (namespace)` gives you the min over time per namespace. - `max_over_time(...) without (namespace)` gives you the max over time removing the namespace dimension. PS: I've also refactored a bit how we optimized grouping to make it more clear. Fixes #2884 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * nit doc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Vitaliy Potapov	d05529f0e7	[Docs] Clarify docker-driver configuration options (#3049) ... Clarify docker-driver configuration about `max-file` and `max-size` default values. Fixes #3046	5 years ago
Heds Simons	b195e94f39	fluent-bit: Rename Fluent Bit plugin output name. (#2974) ... This prevents a clash with the Loki plugin now included with the Fluent Bit agent by default. The output plugin is now named `grafana-loki`. Signed-off-by: Heds Simons <heds@whaleway.net>	5 years ago
Jack Baldry	2d8cf154ba	doc: logql formatting fixes (#3039) ... * doc: Remove extraneous backtick Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * doc: Fix backtick label quoting example Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	5 years ago
Diana Payton	72b4c0181a	Update _index.md (#3032)	5 years ago
Reinhard Nägele	1bb4301f2e	Fix multiline docs (#3035) ... Signed-off-by: Reinhard Nägele <unguiculus@gmail.com>	5 years ago
Karsten Jeschkies	d70332c3aa	Correct `multiline` documentation. (#3029)	5 years ago
Ilias Trichopoulos	bc29c6109e	Fix docs header inconsistency (#3027)	5 years ago
Karsten Jeschkies	afc1e53df9	Collapse multiline logs based on a start line. (#3024) ... * Collapse multiline logs based on a start line. Summary: This is a very simple approach based on #1380 to provide multiline or block log entries in promtail. A `multiline` stage is added to pipelines. This stages matches a start line. Once a start line is matched all following lines are appended to an entry and not passed on to downstream stages. Once a new start line is matched the former block of multilines is sent. If now new line arrives withing `max_wait_time` the block is flushed to the next stage and a new block is started. * Test multiline stage process. * Format code. * Flush multiline block after `max_lines`. * Capture internal state of the stage. * Process different multiline streams in parallel. * Start documenting multiline stage. * Give an example configuration for `multiline` stage. * Make linter happy. * Pass through entries until first start line. * Update pkg/logentry/stages/multiline.go	5 years ago
Jack Baldry	794c324e64	doc: Cleanup formatting (#3017) ... * doc: Add missing backticks Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * doc: Use absolute path to positions.yaml file Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * doc: Remove .. from URL paths I'm not sure what the `..` means exactly. Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	5 years ago
Jón Tómas Grétarsson	f6fd6ae7da	Add a critical missing field to the extended config s3 example (#2991)	5 years ago
Jack Baldry	46637db9cc	doc: Fix query-frontend typo (#3009) ... Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	5 years ago
Owen Diehl	4d9865acd4	Adds WAL support (experimental) (#2981) ... * marshalable chunks * wal record types custom serialization * proto types for wal checkpoints * byteswith output unaffected by buffer * wal & record pool ifcs * wal record can hold entries from multiple series * entry pool * ingester uses noopWal * removes duplicate argument passing in ingester code. adds ingester config validation & derives chunk encoding. * segment writing * [WIP] wal recovery from segments * replay uses sync.Maps & preserves WAL fingerprints * in memory wal recovery * wal segment recovery * ingester metrics struct * wal replay locks streamsMtx in instances, adds checkpoint codec * ingester metrics * checkpointer * WAL checkpoint writer * checkpointwriter can write multiple checkpoints * reorgs checkpointing * wires up checkpointwriter to wal * ingester SeriesIter impl * wires up ingesterRecoverer to consume checkpoints * generic recovery fn * generic recovery fn * recover from both wal types * cleans up old tmp checkpoints & allows aborting in flight checkpoints * wires up wal checkpointing * more granular wal logging * fixes off by 1 wal truncation & removes double logging * adds userID to wal records correctly * wire chunk encoding tests * more granular wal metrics * checkpoint encoding test * ignores debug bins * segment replay ignores out of orders * fixes bug between WAL reading []byte validity and proto unmarshalling refs * conf validations, removes comments * flush on shutdown config * POST /ingester/shutdown * renames flush on shutdown * wal & checkpoint use same segment size * writes entries to wal regardless of tailers * makes wal checkpoing duration default to 5m * recovery metrics * encodes headchunks separately for wal purposes * merge upstream * linting * addresses pr feedback uses entry pool in stream push/tailer removes unnecessary pool interaction checkpointbytes comment fillchunk helper, record resetting in tests via pool redundant comment defers wg done in recovery s/num/count/ checkpoint wal uses a logger encodeWithTypeHeader now creates its own []byte removes pool from decodeEntries wal stop can error * prevent shared access bug with tailers and entry pool * removes stream push entry pool optimization	5 years ago
Anton Nehaev	3d27b1f468	docs: added loki4j to the list of unofficial clients (#2869)	5 years ago
LTek-online	56830bc78b	Changed incorrect headers of sections in the promtail config docu (#2636) ... The configuration documentation of promtails contains some errors where the headers of the sections don't match the actual configs. Notably configuration keys like 'server' and 'clients' have been mislabeled as 'server_config' and 'client_config'. This merge request is aimed to correct that so that it matches the configuration as in the written config files to avoid confusion.	5 years ago
Owen Diehl	1b6465d940	bumps rules-action ref to logqlv2+ version (#2951)	5 years ago
Cyril Tovena	61ba02e2c4	Limit series for metric queries. (#2903) ... * Work in progress limit series. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * First draft of the series limiter. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add limiter to query-frontend. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix build Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fmted Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint and build fix Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs/sources/configuration/_index.md Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com> * Review feedback. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Reduce // for to fix flaky test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>	5 years ago