949 Commits (8a95972f730c0133e19dccffdcba2e31a3e0117a)

 

All Branches   

Search

Author	SHA1	Message	Date
Ed Welch	8a95972f73	Add some additional logging to the canary on queries (#1137) ... * log what was missing from the websocket connection and what we received from a query to Loki to troubleshoot missing logs * removing unnecessary unused return variables from print statements * update test to handle the new debug output	7 years ago
kailwallin	4f780d3624	Update README.md (#1132) ... Fixed typo	7 years ago
Robert Fratto	f1c1097bbe	pkg/canary: use default HTTP client when reading from Loki ... The documentation[1] for net/http.Client says the following: A Client is an HTTP client. Its zero value (DefaultClient) is a usable client that uses DefaultTransport. The Client's Transport typically has internal state (cached TCP connections), so Clients should be reused instead of created as needed. Clients are safe for concurrent use by multiple goroutines. A Client is higher-level than a RoundTripper (such as Transport) and additionally handles HTTP details such as cookies and redirects. Canary was creating a new http.Client every time a query request was made to Loki, causing a new TCP connection to be established each time. If Loki has an outage, it's possible to get into a situation where all websocket requests are failing and for a high volume of requests to go through the normal query route. This can possibly lead to networking issues like socket starvation. [1]: https://golang.org/pkg/net/http/#Client	7 years ago
Robert Fratto	2cb3c82cc8	pkg/ingester: prevent shutdowns from processing during joining handoff ... This commit fixes a race condition where an ingester that is shut down during the joining handoff (i.e., receiving chunks from a leaving ingester before claiming its tokens) hangs and can never shut down cleanly. A shutdown mutex is implemented which is obtained at the start of the handoff process and released after the handoff process completes. This race condition also prevented the leaving ingester from completing its shutdown, since it waits for the joining ingester to claim the tokens. This means that if a brand new ingester is shut down, it will always have finished receiving chunks from the previous leaving ingester and have finished obtaining the tokens from the ingester attempting to leave.	7 years ago
Sandeep Sukhani	b75965e915	list backups just before starting deletion of wanted backups	7 years ago
Joe Elliott	b835e2082a	Fix Loki Chunks Dashboard (#1126) ... This commit cleans up the Loki Chunks dashboard either by renaming incorrectly named series or adding needed metrics. Added new metrics: - loki_ingester_memory_chunks - loki_ingester_memory_streams - loki_ingester_chunk_utilization - loki_ingester_chunk_compression_ratio The following metrics were already being published with a different name. Fixed in the dashboard: - cortex_chunk_store_index_entries_per_chunk - cortex_ingester_flush_queue_length - loki_ingester_chunk_entries	7 years ago
Marco Pracucci	2477f70431	docs: clarified the relation between retention period and table period (#1129)	7 years ago
Marco Pracucci	f3370cf120	docs: fix Promtail / Loki capitalization (#1130) ... * docs: fixed Promtail capitalization * docs: fixed Loki capitalization	7 years ago
Marco Pracucci	67f8ff8c64	Increased promtail's backoff settings in prod and improved doc (#1083)	7 years ago
Marcos Nils	52a9aa7674	Bump grafana image (#1110)	7 years ago
Marco Pracucci	bf5eabe2de	Do not retry chunks transferring on shutdown in the local dev env (#1085)	7 years ago
Sandeep Sukhani	569748ff18	logging: removed some noise in logs from live-tailing (#1100) ... * Removed some noise in logs by detecting cases where connection is erroring due to stopped or closed tail request * fixed log formats	7 years ago
Robert Fratto	05f8e564db	Clarify journal warning (#1109)	7 years ago
Robert Fratto	cdc0dc47e1	build: enable journal in promtail linux release build (#1072) ... * build: override promtail release build with cgo-enabled binary The published binary of promtail for linux/amd64 was not built with CGO support, so journal tailing could not be used. This is kind of a dirty hack: we just re-build promtail at the end of building everything else with CGO enabled. It's not clear how it could be done more cleanly. Fixes #1057. * build: break out gox builds	7 years ago
Joe Elliott	80313471e5	Updated push path to current prod (#1108) ... Signed-off-by: Joe Elliott <number101010@gmail.com>	7 years ago
Marco Pracucci	96237b5550	logcli: added --step support to query command (#1103) ... * logcli: added --step support to query command * Changed query range API default step value from 1s to a dynamic value based on start/end input params	7 years ago
Marco Pracucci	7ac9e4ac98	docs: improved table manager documentation (#1099)	7 years ago
Cyril Tovena	80a4261b46	Add fluent-bit missing link in the main documentation page. (#1106)	7 years ago
Marcos Nils	cc45d70477	Deploy prometheys from helm chart (#1104) ... Add chart values to deploy prometheus locally. This disables persistentVolumes as it'd fail if deployed locally	7 years ago
Jens Erat	c4913bfc88	fluent-bit: edge case tests (#1096)	7 years ago
Ángel Barrera	e4d6a8cb7c	Feature: Add extra containers to loki helm chart (#909) ... * Add extra containers feature to loki chart * minor bump version to loki helm charts	7 years ago
Joe Elliott	a48230d343	Reverted cloud.google.com/go to 0.44.1 (#1097) ... Signed-off-by: Joe Elliott <number101010@gmail.com>	7 years ago
Joe Elliott	9faa8bcf1d	Go 1.13 (#1089) ... Updated everything to 1.13 Signed-off-by: Joe Elliott <number101010@gmail.com>	7 years ago
Robert Fratto	57e7a1c386	docs: update stages README with the docker and cri stages (#1094)	7 years ago
Xiang Dai	ca6fce19cc	docs(stage): add docker and cri (#1091) ... Signed-off-by: Xiang Dai <764524258@qq.com>	7 years ago
Joe Elliott	fc6c5c063f	dep => go mod (#1062) ... * go mod files added. dep removed Signed-off-by: Joe Elliott <number101010@gmail.com> * Magically got prometheus version to stick Signed-off-by: Joe Elliott <number101010@gmail.com> * Cortex updated and prometheus updated Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated prometheus dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added additional deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Added two replaces from Gopkg.toml. Tests passing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated build image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golangci-lint to use vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod makefile step and referenced in drone and circle ci configs Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated loki-build-image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed linting error. Force go111module on for linting Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod vendor Signed-off-by: Joe Elliott <number101010@gmail.com> * Forced the inclusion of modtimevfs Signed-off-by: Joe Elliott <number101010@gmail.com> * Pin client-go due to issue with v12 go.mod Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod to drone Signed-off-by: Joe Elliott <number101010@gmail.com> * Re-readded correct golang client Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Pinned golang/x/net to avoid proxy errors Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed check-mod from all. Not necessary for tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go.sum to match new pin Signed-off-by: Joe Elliott <number101010@gmail.com> * Upgraded proto to match build image Signed-off-by: Joe Elliott <number101010@gmail.com> * Force check-mod to wait til after test and lint are successful Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to go builds to force usage of vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Turn on gomodules on all builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Added mod vendor to go builds to force usage of vendored dependencies" This reverts commit 65865a24c9a23133e0fa52942f2828ead7c22147. * Moved builds out of the gopath to enforce vendor usage Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Turn on gomodules on all builds" This reverts commit b5847f0158e928e935e0b3c1b1d4eaba840ca3dc. * Explicitly choose build image for docker driver Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golang.org/x/sys to match prometheus's version to fix promtail windows compilation Signed-off-by: Joe Elliott <number101010@gmail.com> * Added fluentbit dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added dependency management clause Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go version in contributing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added phony makefile targets Signed-off-by: Joe Elliott <number101010@gmail.com> * Circle CI is increasingly failing linting Signed-off-by: Joe Elliott <number101010@gmail.com> * Force vendored deps on builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Added logcli to gitignore Signed-off-by: Joe Elliott <number101010@gmail.com> * Reduced concurrency to help CircleCI Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved drone builds out of GOPATH to force use of go module Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to prevent go clean from redownloading all packages Signed-off-by: Joe Elliott <number101010@gmail.com> * Added to test as well Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod-vendor to final go commands Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved fluent-bit out of GO_PATH to force go modules Signed-off-by: Joe Elliott <number101010@gmail.com> * Pass mod vendor to the last holdout: go generate Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated ugorji/go to 1.17 to avoid version regression Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy and go mod vendor for ugorji/go Signed-off-by: Joe Elliott <number101010@gmail.com>	7 years ago
Marco Pracucci	42040f2d5b	Skip ingester tailer filtering if no filter is set	7 years ago
Sandeep Sukhani	3e0621ebf6	verify backups to work on latest list of backups ... when tables rotate, it creates a final backup of last active table. verify backups was not having that last backup in the list so this change makes sure it has latest list of backups	7 years ago
Hiroshi Hatake	0cfa33d90d	fluent-bit shared object go plugin (#847) ... * Import fluent-bit-go-loki plugin sources * Add fluent-bit-go package constraint * Ensure dependencies with dep ``` $ dep ensure ``` * Update .gitignore * Move fluent-bit-go-loki to fluent-bit/fluent-bit-go-loki directory * Add documentation for fluent-bit-go-loki * Fix for lint * Fix for goimports errors * Display fluent-bit-go-loki plugin version * Use more descriptive description for fluent-bit Loki plugin * Remove needless newlines * Rectify fluent-bit-go-loki makefile Remove needless parts and rename meaningless target * Use more descriptive description in README * refactor(fluent-bit): move make targets into root Makefile * Organize imports * Reorder imports with goimports $ goimports -w out_loki.go * Remove needless LICENSE file Because it is the same for Loki repository. * Rely on Loki client retry mechanism * Make descriptions more clearly * Change default unit * BatchWait: msec -> sec * BatchSize: KiB -> Byte * Use logql to parse Loki native labels format * Use logger instead of fmt.Printf * Update documentation to reflect recent changes * Use prometheus version module to print version * Dump error message with logger * Use DYN_GO_FLAGS instead of custom FLAGS for fluent-bit-plugin * Support RemoveKeys parameter * use logql.ParseMatchers() * refactor fluent-bit go plugin * add ci step for docker image * fix lint issues * fix a failing url test * fix a failing url test * add build images for drone ci * Adding labelmap file to complex record to labels Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add helm chart and fix a bug with labelmap * New push path * dep sync * Move fluent-bit folder and use jsonnet for drone * fix build * Adds more documentation and include a basic config in the container Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * remove binaries * Some readme grammar fixes * docs: really nitpicky grammar change in clients README * Add helm documentation * fixes bad config * bump chart * fixes .gitignore	7 years ago
flouthoc	1f0b7cc5da	Fix Docs: Change Imagepull policy to IfNotpresent / Add loki-canary base manifests (#1073)	7 years ago
sh0rez	00245403d9	feat: configuration source precedence (#980) ... This implements more sophisticated handling of our configuration: - Correct precedence: Flags set on the command line now precede values from YAML, which in turn precede the defaults from the `flag` package - Sticks to the `flagext.Registerer` pattern, as it is partly vendored from Cortex / Weaveworks code we cannot change easily	7 years ago
Xiang Dai	d59b6481b2	unify references to GOPATH (#1069) ... Signed-off-by: Xiang Dai <764524258@qq.com>	7 years ago
Robert Fratto	791262dc25	Add Drone CI badge to README.md (#1060)	7 years ago
Xiang Dai	8c8fc33d7f	doc: skip jb init when using Tanka (#1068) ... Signed-off-by: Xiang Dai <764524258@qq.com>	7 years ago
polar3130	a2060efd63	Helm: Remove default value of storageClassName in loki/loki helm chart (#1058) ... * delete default name of storageClassName "default" * bump loki stack	7 years ago
Robert Fratto	cc0a4c2188	Fix broken links to docs in README.md (#1067)	7 years ago
Jonas Kello	f34b006538	Fix spelling of HTTP header (#1064)	7 years ago
Robert Fratto	a3fde23651	docs: fix deprecated warning in api.md (#1063)	7 years ago
Robert Fratto	65ba42a6e7	Documentation Rewrite (#982) ... * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)	7 years ago
flouthoc	f755c59011	Fix Docs: Change Imagepull policy to IfNotpresent / Add loki-canary base manifests (#1053)	7 years ago
Joe Elliott	778c0b6857	Update loki push path (#1049) ... This commit adds a /loki/api/v1/push endpoint to mirror the new v1 API, and updates all references to the old push endpoint to new the one.	7 years ago
Peter Štibraný	7cb9538c85	dep ensure (#1054)	7 years ago
makocchi	54a27dc389	helm chart: Add missing operator to promtail (#967) ... * Add missing operator * Bump chart versions * Bump chart versions	7 years ago
Tatsuya Egawa	b089254f09	Fix README link (#1048)	7 years ago
Xiang Dai	20048f5474	doc(ksonnet): include ksonnet-lib (#1042) ... As Tanka does not include the ksonnet-lib by default, we need to add it manually using jsonnet-bundler	7 years ago
Joe Elliott	7794d15b6a	Extended test query to expose broken query. Parsed logql correctly in validate (#1044) ... Signed-off-by: Joe Elliott <number101010@gmail.com>	7 years ago
Joe Elliott	9614b021d1	Loki HTTP/JSON Model Layer (#1022) ... * First pass data model Signed-off-by: Joe Elliott <number101010@gmail.com> * Use prom model b/c we're serializing promql objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy query support and tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy label test Signed-off-by: Joe Elliott <number101010@gmail.com> * Added tail response marshalling and tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed marshallers and test Signed-off-by: Joe Elliott <number101010@gmail.com> * Expanded legacy test cases Signed-off-by: Joe Elliott <number101010@gmail.com> * Dropped streams nano test Signed-off-by: Joe Elliott <number101010@gmail.com> * First pass v1 new objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Added failing tail response tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added failing tailresponse test Signed-off-by: Joe Elliott <number101010@gmail.com> * Partial v1 tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved legacy labels test Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved legacy query tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved all legacy tests to new method Signed-off-by: Joe Elliott <number101010@gmail.com> * Added v1 tests and fixed stream marshalling bug Signed-off-by: Joe Elliott <number101010@gmail.com> * First pass new Model Signed-off-by: Joe Elliott <number101010@gmail.com> * Added vector test Signed-off-by: Joe Elliott <number101010@gmail.com> * Added matrix tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added conversions for all things except tailed responses Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed mixed case issues Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tail marshalling Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed unused testStream Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved TailResponse to loghttp Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed legacy tailresponse objects in favor of actual legacy tailresponse objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated v1 methods to take legacy tail objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Cleaned up tests. Added some comments Signed-off-by: Joe Elliott <number101010@gmail.com> * Versioned tail endpoint Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved readability on loghttp packages in http.go Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed new as a var name Signed-off-by: Joe Elliott <number101010@gmail.com> * Started all error messages with lowercase alerts Signed-off-by: Joe Elliott <number101010@gmail.com> * new => ret Signed-off-by: Joe Elliott <number101010@gmail.com> * Added comments on exported methods Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed two personal notes Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed legacy package name to loghttp Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved and renamed loghttp v1 package Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved marshalling code out of model Signed-off-by: Joe Elliott <number101010@gmail.com> * Added package comments Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy testing Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed DroppedStream slice to value type for consistency Signed-off-by: Joe Elliott <number101010@gmail.com> * gofmt'ed test files Signed-off-by: Joe Elliott <number101010@gmail.com> * Cleaned up linting issues Signed-off-by: Joe Elliott <number101010@gmail.com> * Minor comment cleanup Signed-off-by: Joe Elliott <number101010@gmail.com> * Adjusted GOGC to make CircleCI happy Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed legacy => loghttp for consistency Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed matrix error message to be correct Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved label query over to loghttp response Signed-off-by: Joe Elliott <number101010@gmail.com> * Added marshal loop tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added response type test Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tail response marshal/unmarshal Signed-off-by: Joe Elliott <number101010@gmail.com> * Passing unmarshal/marshal queryresponse tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed vector and matrix Signed-off-by: Joe Elliott <number101010@gmail.com> * Added output support for streams minus ordering Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tailing Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed output tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed query tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Order log output Signed-off-by: Joe Elliott <number101010@gmail.com> * Use labels instead of stream Signed-off-by: Joe Elliott <number101010@gmail.com> * Lowered parallelization for CircleCI Signed-off-by: Joe Elliott <number101010@gmail.com>	7 years ago
Sandeep Sukhani	158ca9d3dc	use latest bigtable backup docker image with fix for list backups (#1043)	7 years ago
Sandeep Sukhani	ec5bc7054d	limits: limits implementation for loki (#948) ... * limits implementation for loki Added following new limits: Length of query Number of active streams Number of streams matcher per query Ingestion rate * periodically reload ingestion rate limits to apply latest rates from overrides	7 years ago
sh0rez	b30c0d31bf	doc(production): replace ksonnet with Tanka (#1039) ... As `ksonnet` is long deprecated and [Tanka](https://github.com/grafana/tanka) being mature enough to replace it, we should show users how to do so.	7 years ago