756 Commits (6f8b6e1d330ffeba67d3c0a060ca9d1d87b602cd)

Author	SHA1	Message	Date
Cyril Tovena	f1b8d4d8ad	Improve logql query statistics collection. (#1573) ... * Improve logql query statistics collection. This also add information about ingester queries. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes bad copy/past in the result log. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes ingester tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve headchunk efficiency. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix bad commit on master. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes new interface of ingester grpc server. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve documentations of fields. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Peter Štibraný	d4cae667ae	loki: use new runtimeconfig package from Cortex (#1484) ... * loki: use new runtimeconfig package from Cortex runtimeconfig is a generalization of previous "overrides" mechanism, and can be used not for per-tenant limits, but other parts of Loki too (only one additional usage for now is "multi" KV client). Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy, go mod vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed RuntimeConfig from Ingester dependencies. Ingester already depends on RuntimeConfig via Store and Overrides mechanism. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed extra new line. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Remove unused method. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix broken code after rebasing. Removed incorrect comment. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Updated docs with section about runtime configuration. Mostly copied from Cortex. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy, go mod vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Updated Cortex to current master. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Go mod tidy, vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
petruha	4f314f3df4	Fileglob parsing fixes (#1485) ... * Update doublestar module to version 1.2.2. * Add absolute path for each element in list. * Remove doublestar test and update vendore module version.	5 years ago
Marco Pracucci	ce407d3276	Add ingestion rate global limit support (#1486) ... * Added ingestion rate global limit support Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed code comments Signed-off-by: Marco Pracucci <marco@pracucci.com> * Updated config doc Signed-off-by: Marco Pracucci <marco@pracucci.com> * Updated changelog Signed-off-by: Marco Pracucci <marco@pracucci.com> * Updated doc Signed-off-by: Marco Pracucci <marco@pracucci.com>	5 years ago
Cyril Tovena	4153740a12	Loki Query Frontend (#1442) ... * Adds frontend to Loki. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improves tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes sneaky bug in entries sorting. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes the split by interval. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Tweak jsonnet deployments and add a way to lint/fmt. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fix timezone issue. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve tests and rollback change in loghttp package. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a flaky test that might run one more goroutine. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes windows build. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve tracing in the split by interval. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add test stream to proto conversion. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes flappy retry test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove err shadowing in stopQueryFrontend as it was confusing. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor grpc message size in the libsonnet config file. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Don't check auth header for GRPC TransferChunks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Query frontend (#3) * frontend codec merging optimizations * codec benchmarks * removes unused bounds code in queryrange ordering * [wip] splitby uses channels instead of sub batching intervals * splitBy channel limit test * single allocation for merging entries from a single stream * skip merging loki responses when limit is already hit * removes checks for unlimited queries in queryrange * removes splitByInterval{,.interval} spans * removes interval_batch_size from jsonnet lib * moves benchmark utils to own file * renames markers -> entries * priority queue comments * Removes unused logRequest. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Sets the cache interval to the same split interval. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Missing import libsonnet for the frontend. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Frontend should not be a cluster IP. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	5 years ago
Jens Erat	c9016f4b2a	fluent-bit-plugin: re-enable JSON marshal tests (#1455) ... My upstream bugfix to `json-iterator/go` was accepted. This commit updates to the newest upstream release, adds checks for the returned error message and reenables the commented-out JSON marshaller tests. Re-enabling the JSON marshaller tests that should return an error message, I realized `sendMessage` logged and ignored marshalling errors instead of passing them to the caller. Signed-off-by: Jens Erat <email@jenserat.de>	5 years ago
Cyril Tovena	7654c27c12	Adds configurable compression algorithms for chunks (#1411) ... * Adds L4Z encoding. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds encoding benchmarks Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds snappy encoding. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds chunk size test Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds snappy v2 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve benchmarks Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove chunkenc Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update lz4 to latest master version. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use temporary buffer in serialise method to avoid allocations when doing string -> byte conversion. It also makes code little more readable. We pool those buffers for reuse. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Added gzip -1 for comparison. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Initialize reader and buffered reader lazily. This helps with reader/buffered reader reuse. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Don't keep entries, extracted generateData function (mostly to get more understandable profile) Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Improve test and benchmark to cover all encodings. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds support for a new chunk format with encoding info. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Ingesters now support encoding config. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add support for no compression. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add docs Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove default Gzip for ByteChunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Removes none, snappyv2 and gzip-1 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Move log test lines to testdata and add supported encoding stringer Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Sebastian Widmer	f0f6f24926	pkg/promtail: IETF Syslog (RFC5424) Support (#1275) ... * IETF syslog (rfc5254) support * Upgrade go-syslog to correctly released v2.0.1 * Incorporate feedback from new linter * Update documentation with relevant RFC sections. * Incorporate feedback from review * Use context to shutdown server. * Use util.Backoff from cortex * Do not embed mutex into TestLabeledClient * Use strings.HasPrefix * Better naming of connectionsClosed -> openConnections * Incorporate further feedback from review * Callback instead of channel in ParseStream. Removes one running Goroutine per connection. * Improve parse error log level. * Move mutex above field it protects. * Use backoff.Ongoing() * Switch to "nontransparent" parser * Further improvements to the documentation	6 years ago
Adam Johnson	3ae6fecbfc	feat(loki): support IAM roles for EKS (#1340) ... * pull in cortex changes to support IAM roles for EKS * update go.sum after running tidy Signed-off-by: Adam Johnson <a.johnson@winton.com>	6 years ago
Owen Diehl	b5c03b0802	Fix duplicate labels (update cortex) (#1280) ... * Update to latest cortex to fix duplicate labels Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Start the licycler in the ingester Required since this cortex change in 00566f61af\#diff-dcd65a4ff2de424be70a561f141e067bR180 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * go mod tidy Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Move cortex to a fork for ARM support Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * updates cortex to master Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * go mod tidy/vendor Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	6 years ago
Joe Elliott	3d2c643bba	Rolled back to v0.8.0 (#1237) ... Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Srinidhi Kaushik	90b937c743	Update dependencies to use weaveworks/common upstream (#1226)	6 years ago
Cyril Tovena	d438bda94a	Update cortex to bring v11 schema (#1201) ... * Update cortex to bring v11 schema Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes change in ParseProtoReader Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * sync go mod Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Anderson Queiroz	3b802b2ce1	AWS EKS: Update the AWS Go SDK to version v1.23.13 (#1182) ... Fixes: #1019	6 years ago
Ed Welch	94b9c4ba16	Improve how we record file size metric to avoid a race in our file lagging alert (#1194) ... * Update the filetarget sync() code to report size based on what the tailer reports for size (or read the file size from the file name if no tailer exists yet) Update vendored (and forked) hpcloud tail library which includes a Size() function which reports size from the file handle and not file name * needed to tidy? * addressing code review comments	6 years ago
Peter Štibraný	3df5561336	Updated cortex to latest master. (#1088) ... * Updated cortex to latest master. * Updated cortex to latest master. * Fix tests after updating cortex. * go mod tidy and vendor	6 years ago
Joe Elliott	a48230d343	Reverted cloud.google.com/go to 0.44.1 (#1097) ... Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Joe Elliott	9faa8bcf1d	Go 1.13 (#1089) ... Updated everything to 1.13 Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Joe Elliott	fc6c5c063f	dep => go mod (#1062) ... * go mod files added. dep removed Signed-off-by: Joe Elliott <number101010@gmail.com> * Magically got prometheus version to stick Signed-off-by: Joe Elliott <number101010@gmail.com> * Cortex updated and prometheus updated Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated prometheus dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added additional deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Added two replaces from Gopkg.toml. Tests passing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated build image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golangci-lint to use vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod makefile step and referenced in drone and circle ci configs Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated loki-build-image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed linting error. Force go111module on for linting Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod vendor Signed-off-by: Joe Elliott <number101010@gmail.com> * Forced the inclusion of modtimevfs Signed-off-by: Joe Elliott <number101010@gmail.com> * Pin client-go due to issue with v12 go.mod Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod to drone Signed-off-by: Joe Elliott <number101010@gmail.com> * Re-readded correct golang client Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Pinned golang/x/net to avoid proxy errors Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed check-mod from all. Not necessary for tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go.sum to match new pin Signed-off-by: Joe Elliott <number101010@gmail.com> * Upgraded proto to match build image Signed-off-by: Joe Elliott <number101010@gmail.com> * Force check-mod to wait til after test and lint are successful Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to go builds to force usage of vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Turn on gomodules on all builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Added mod vendor to go builds to force usage of vendored dependencies" This reverts commit 65865a24c9a23133e0fa52942f2828ead7c22147. * Moved builds out of the gopath to enforce vendor usage Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Turn on gomodules on all builds" This reverts commit b5847f0158e928e935e0b3c1b1d4eaba840ca3dc. * Explicitly choose build image for docker driver Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golang.org/x/sys to match prometheus's version to fix promtail windows compilation Signed-off-by: Joe Elliott <number101010@gmail.com> * Added fluentbit dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added dependency management clause Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go version in contributing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added phony makefile targets Signed-off-by: Joe Elliott <number101010@gmail.com> * Circle CI is increasingly failing linting Signed-off-by: Joe Elliott <number101010@gmail.com> * Force vendored deps on builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Added logcli to gitignore Signed-off-by: Joe Elliott <number101010@gmail.com> * Reduced concurrency to help CircleCI Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved drone builds out of GOPATH to force use of go module Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to prevent go clean from redownloading all packages Signed-off-by: Joe Elliott <number101010@gmail.com> * Added to test as well Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod-vendor to final go commands Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved fluent-bit out of GO_PATH to force go modules Signed-off-by: Joe Elliott <number101010@gmail.com> * Pass mod vendor to the last holdout: go generate Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated ugorji/go to 1.17 to avoid version regression Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy and go mod vendor for ugorji/go Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago