327 Commits (9faa8bcf1dc819179bfc29d3d5d9a7d0878717c5)

Author	SHA1	Message	Date
Joe Elliott	fc6c5c063f	dep => go mod (#1062) ... * go mod files added. dep removed Signed-off-by: Joe Elliott <number101010@gmail.com> * Magically got prometheus version to stick Signed-off-by: Joe Elliott <number101010@gmail.com> * Cortex updated and prometheus updated Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated prometheus dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added additional deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Added two replaces from Gopkg.toml. Tests passing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated build image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golangci-lint to use vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod makefile step and referenced in drone and circle ci configs Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated loki-build-image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed linting error. Force go111module on for linting Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod vendor Signed-off-by: Joe Elliott <number101010@gmail.com> * Forced the inclusion of modtimevfs Signed-off-by: Joe Elliott <number101010@gmail.com> * Pin client-go due to issue with v12 go.mod Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod to drone Signed-off-by: Joe Elliott <number101010@gmail.com> * Re-readded correct golang client Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Pinned golang/x/net to avoid proxy errors Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed check-mod from all. Not necessary for tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go.sum to match new pin Signed-off-by: Joe Elliott <number101010@gmail.com> * Upgraded proto to match build image Signed-off-by: Joe Elliott <number101010@gmail.com> * Force check-mod to wait til after test and lint are successful Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to go builds to force usage of vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Turn on gomodules on all builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Added mod vendor to go builds to force usage of vendored dependencies" This reverts commit 65865a24c9a23133e0fa52942f2828ead7c22147. * Moved builds out of the gopath to enforce vendor usage Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Turn on gomodules on all builds" This reverts commit b5847f0158e928e935e0b3c1b1d4eaba840ca3dc. * Explicitly choose build image for docker driver Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golang.org/x/sys to match prometheus's version to fix promtail windows compilation Signed-off-by: Joe Elliott <number101010@gmail.com> * Added fluentbit dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added dependency management clause Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go version in contributing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added phony makefile targets Signed-off-by: Joe Elliott <number101010@gmail.com> * Circle CI is increasingly failing linting Signed-off-by: Joe Elliott <number101010@gmail.com> * Force vendored deps on builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Added logcli to gitignore Signed-off-by: Joe Elliott <number101010@gmail.com> * Reduced concurrency to help CircleCI Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved drone builds out of GOPATH to force use of go module Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to prevent go clean from redownloading all packages Signed-off-by: Joe Elliott <number101010@gmail.com> * Added to test as well Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod-vendor to final go commands Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved fluent-bit out of GO_PATH to force go modules Signed-off-by: Joe Elliott <number101010@gmail.com> * Pass mod vendor to the last holdout: go generate Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated ugorji/go to 1.17 to avoid version regression Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy and go mod vendor for ugorji/go Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Marco Pracucci	42040f2d5b	Skip ingester tailer filtering if no filter is set	6 years ago
Hiroshi Hatake	0cfa33d90d	fluent-bit shared object go plugin (#847) ... * Import fluent-bit-go-loki plugin sources * Add fluent-bit-go package constraint * Ensure dependencies with dep ``` $ dep ensure ``` * Update .gitignore * Move fluent-bit-go-loki to fluent-bit/fluent-bit-go-loki directory * Add documentation for fluent-bit-go-loki * Fix for lint * Fix for goimports errors * Display fluent-bit-go-loki plugin version * Use more descriptive description for fluent-bit Loki plugin * Remove needless newlines * Rectify fluent-bit-go-loki makefile Remove needless parts and rename meaningless target * Use more descriptive description in README * refactor(fluent-bit): move make targets into root Makefile * Organize imports * Reorder imports with goimports $ goimports -w out_loki.go * Remove needless LICENSE file Because it is the same for Loki repository. * Rely on Loki client retry mechanism * Make descriptions more clearly * Change default unit * BatchWait: msec -> sec * BatchSize: KiB -> Byte * Use logql to parse Loki native labels format * Use logger instead of fmt.Printf * Update documentation to reflect recent changes * Use prometheus version module to print version * Dump error message with logger * Use DYN_GO_FLAGS instead of custom FLAGS for fluent-bit-plugin * Support RemoveKeys parameter * use logql.ParseMatchers() * refactor fluent-bit go plugin * add ci step for docker image * fix lint issues * fix a failing url test * fix a failing url test * add build images for drone ci * Adding labelmap file to complex record to labels Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add helm chart and fix a bug with labelmap * New push path * dep sync * Move fluent-bit folder and use jsonnet for drone * fix build * Adds more documentation and include a basic config in the container Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * remove binaries * Some readme grammar fixes * docs: really nitpicky grammar change in clients README * Add helm documentation * fixes bad config * bump chart * fixes .gitignore	6 years ago
sh0rez	00245403d9	feat: configuration source precedence (#980) ... This implements more sophisticated handling of our configuration: - Correct precedence: Flags set on the command line now precede values from YAML, which in turn precede the defaults from the `flag` package - Sticks to the `flagext.Registerer` pattern, as it is partly vendored from Cortex / Weaveworks code we cannot change easily	6 years ago
Joe Elliott	778c0b6857	Update loki push path (#1049) ... This commit adds a /loki/api/v1/push endpoint to mirror the new v1 API, and updates all references to the old push endpoint to new the one.	6 years ago
Joe Elliott	7794d15b6a	Extended test query to expose broken query. Parsed logql correctly in validate (#1044) ... Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Joe Elliott	9614b021d1	Loki HTTP/JSON Model Layer (#1022) ... * First pass data model Signed-off-by: Joe Elliott <number101010@gmail.com> * Use prom model b/c we're serializing promql objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy query support and tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy label test Signed-off-by: Joe Elliott <number101010@gmail.com> * Added tail response marshalling and tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed marshallers and test Signed-off-by: Joe Elliott <number101010@gmail.com> * Expanded legacy test cases Signed-off-by: Joe Elliott <number101010@gmail.com> * Dropped streams nano test Signed-off-by: Joe Elliott <number101010@gmail.com> * First pass v1 new objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Added failing tail response tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added failing tailresponse test Signed-off-by: Joe Elliott <number101010@gmail.com> * Partial v1 tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved legacy labels test Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved legacy query tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved all legacy tests to new method Signed-off-by: Joe Elliott <number101010@gmail.com> * Added v1 tests and fixed stream marshalling bug Signed-off-by: Joe Elliott <number101010@gmail.com> * First pass new Model Signed-off-by: Joe Elliott <number101010@gmail.com> * Added vector test Signed-off-by: Joe Elliott <number101010@gmail.com> * Added matrix tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added conversions for all things except tailed responses Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed mixed case issues Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tail marshalling Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed unused testStream Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved TailResponse to loghttp Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed legacy tailresponse objects in favor of actual legacy tailresponse objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated v1 methods to take legacy tail objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Cleaned up tests. Added some comments Signed-off-by: Joe Elliott <number101010@gmail.com> * Versioned tail endpoint Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved readability on loghttp packages in http.go Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed new as a var name Signed-off-by: Joe Elliott <number101010@gmail.com> * Started all error messages with lowercase alerts Signed-off-by: Joe Elliott <number101010@gmail.com> * new => ret Signed-off-by: Joe Elliott <number101010@gmail.com> * Added comments on exported methods Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed two personal notes Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed legacy package name to loghttp Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved and renamed loghttp v1 package Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved marshalling code out of model Signed-off-by: Joe Elliott <number101010@gmail.com> * Added package comments Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy testing Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed DroppedStream slice to value type for consistency Signed-off-by: Joe Elliott <number101010@gmail.com> * gofmt'ed test files Signed-off-by: Joe Elliott <number101010@gmail.com> * Cleaned up linting issues Signed-off-by: Joe Elliott <number101010@gmail.com> * Minor comment cleanup Signed-off-by: Joe Elliott <number101010@gmail.com> * Adjusted GOGC to make CircleCI happy Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed legacy => loghttp for consistency Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed matrix error message to be correct Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved label query over to loghttp response Signed-off-by: Joe Elliott <number101010@gmail.com> * Added marshal loop tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added response type test Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tail response marshal/unmarshal Signed-off-by: Joe Elliott <number101010@gmail.com> * Passing unmarshal/marshal queryresponse tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed vector and matrix Signed-off-by: Joe Elliott <number101010@gmail.com> * Added output support for streams minus ordering Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tailing Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed output tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed query tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Order log output Signed-off-by: Joe Elliott <number101010@gmail.com> * Use labels instead of stream Signed-off-by: Joe Elliott <number101010@gmail.com> * Lowered parallelization for CircleCI Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Sandeep Sukhani	ec5bc7054d	limits: limits implementation for loki (#948) ... * limits implementation for loki Added following new limits: Length of query Number of active streams Number of streams matcher per query Ingestion rate * periodically reload ingestion rate limits to apply latest rates from overrides	6 years ago
Erwin van Eyk	297cc63e5d	Fix externalURL and path prefix issues (#1026)	6 years ago
polar3130	b2cd49a13e	fix typo in error messages (#1030)	6 years ago
Ward Bekker	3b965105ec	Improved Ingester out-of-order error for faster troubleshooting (#1008)	6 years ago
Edward Welch	04a615a72d	make promtail daemonset name configurable ... also adding a regex test for something else I was working on	6 years ago
polar3130	330d1a6fc0	Fix a link to correct doc and fix a typo (#1014) ... * fix typo * Fix a link to correct doc	6 years ago
Sandeep Sukhani	107dfd3607	Use grpc client config from cortex for Ingester to get more control (#755) ... * Use grpc client config from cortex for Ingester for more control over the connection * moved ingester client config block at right place in production ksonnet	6 years ago
Edward Welch	50456f928c	Revert "Updated stream json objects to be more parse friendly (#1010)" ... This reverts commit 6185d543	6 years ago
Joe Elliott	6185d54320	Updated stream json objects to be more parse friendly (#1010) ... * Added custom marshaller for Entry Signed-off-by: Joe Elliott <number101010@gmail.com> * Added comment and test Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed spelling error * Added a custom marshaller to map entries => values * Added support for labels Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed test to only check to the microsecond level Signed-off-by: Joe Elliott <number101010@gmail.com> * Extended Entry tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Swapped to nanosecs encoded as strings * Cleaned up docs with new loki paths. Removed references to deprecated endpoints * Added support for /loki/api/v1/push Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed obsolete comment Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated docs for new push endpoint Signed-off-by: Joe Elliott <number101010@gmail.com> * Ran gofmt on test file Signed-off-by: Joe Elliott <number101010@gmail.com> * Set GOGC=20 on test to avoid out of memory issue in circle ci Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Joe Elliott	47637852fb	Logcli: Add Support for New Query Path (#987) ... Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Cyril Tovena	364b5bc4f6	Make Loki HTTP API more similar to Prometheus	6 years ago
Edward Welch	e75f61c4a8	prefix all the new api paths with `/loki` to make routing/use a little easier and be more similar to cortex. ... also included the `tail` path with the new api such that we can eventually deprecate all the old paths	6 years ago
sh0rez	3467440b30	fix(distributor): parseError BadRequest (#975) ... When the label validation in the distributor failed, it was reported as 500 Internal Server Error instead of 400 BadRequest. Fixes that.	6 years ago
Marco Pracucci	29f08ccb1c	Fix panic in tailer due to race condition between send() and close() (#986)	6 years ago
Edward Welch	a2e2272653	use a global variable to wrap debug log statments in the pipeline to prevent unnecessary allocations when debug logging is not enabled	6 years ago
Marco Pracucci	62d5122a11	Promtail exports metrics on sent and dropped log entries	6 years ago
Cyrill Troxler	6b5180535f	Set default value for BatchWait as ticker does not accept 0	6 years ago
Cyrill Troxler	dca8a4fb17	Use ticker instead of timer for batch wait ... Fixes #280	6 years ago
Cyril Tovena	8f9cd97880	Improve API compatibility. ... - keep supporting regex for old endpoint. - use reserved keyword for removed property in grpc	6 years ago
Ganesh Vernekar	184927ccdc	Split files and make yacc more redable ... Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Cyril Tovena	625c4f1f53	Move to milliseconds for samples to follow prometheus convention	6 years ago
Cyril Tovena	8bd9f752aa	Adds more tests for range queries	6 years ago
Cyril Tovena	d43348c895	Fix bug in topk, add more tests, improve iterator	6 years ago
Cyril Tovena	51761ece2c	adds tests for instant queries.	6 years ago
Cyril Tovena	7d63795350	fix linter & better comments	6 years ago
Cyril Tovena	7f3db9d7f0	LogQL: Vector and Range Vector Aggregation. ... - adds avg,min,max,top,bottomk,stddev,stdvar,count - updates api documentation - adds tests Improve yacc & go lexer to understand duration Remove support for regexp in all queries Clean up querier and logselector	6 years ago
Sandeep Sukhani	b687ec6e5b	vendoring: update cortex to latest master (#938) ... * update cortex to latest master using v3.4.0-rc.1 for go.etcd.io/etcd v0.0.2 for github.com/prometheus/procfs v1.0.0 for github.com/prometheus/client_golang latest master for github.com/weaveworks/common with changes from https://github.com/weaveworks/common/pull/153 * fixed failing tests * use large instance for test and lint jobs in CircleCI * running only 6 test binaries in parallel * removed resource type change for CircleCI * changed GOGC to 10 for lint makefile target	6 years ago
Robert Fratto	9484632d3d	pkg/querier: fix concurrent access to querier tail clients (#944)	6 years ago
Marco Pracucci	22a1bd5f45	promtail: do not send the last batch - to ingester - if empty (#934) ... * Do not send the last batch from promtail on graceful shutdown if the batch is empty * Tested promtail Client.Handle()	6 years ago
Robert Fratto	4c418430be	promtail: add "max_age" field to configure cutoff for journal reading (#921) ... The journal scrape config in promtail has been updated to support a "max_age" field. max_age determines the oldest journal entry promtail will read when starting the journal reader. When unspecified, max_age defaults to 7h. Even if a position in the journal is saved in the promtail positions file, that position may be ignored if the entry corresponding to that position is older than the max_age cutoff time. Example promtail config for getting up to the last 12h of journal entries: server: http_listen_port: 9080 grpc_listen_port: 0 positions: filename: /tmp/positions.yaml clients: - url: http://localhost:3100/api/prom/push scrape_configs: - job_name: journal journal: max_age: 12h path: /var/log/journal labels: job: systemd-journal relabel_configs: - source_labels: ['__journal__systemd_unit'] target_label: 'unit'	6 years ago
Aditya C S	da4964dd70	Add pipeline unit testing to promtail	6 years ago
Robert Fratto	fc8ae5ab92	loki: shutdown module before shutting down HTTP server (#827) ... Shutting the module down before shutting down the HTTP server enables metrics to continue to be collected while the module terminates. This change syncs up with how Cortex modules are shut down; the module is always terminated before the HTTP server. Fixes #819.	6 years ago
Marco Pracucci	e5d6f3c317	Fix panic in tailer when an ingester is removed from the ring while tailing (#897)	6 years ago
Ed	173e8ea2a4	fix(canary): Fix Flaky Tests (#889) ... * make the missing entry lookup synchronous during tests to hopefully avoid races around the entry order and test verifications * playing around with timings some more to get all tests to pass even on a really slow machine	6 years ago
Cyril Tovena	a4070992d9	fix a bug where nil line buffer would be put back	6 years ago
Robert Fratto	001d36f510	pkg/promtail/positions: remove executable bit from positions file (#893) ... File permissions on creation of positions.yaml will now be 0600 instead of 0700. Fixes #892.	6 years ago
Marco Pracucci	b9185324f4	Improved Tailer loop (dropped entries, logic refactoring, introduced tests)	6 years ago
Michael Dai	3d1f39fab6	Pipeline: Fixed labels process test with same objects (#869) ... * Fixed labels process test with same objects Signed-off-by: jojohappy <sarahdj0917@gmail.com>	6 years ago
Marco Pracucci	c955702b32	Fixed out of order entries allowed in a chunk on edge case (#859)	6 years ago
Sandeep Sukhani	2b8a3d9794	live-tailing: preload all the historic entries before query context is cancelled (#862) ... * preload all the historic entries before query context is cancelled * fix lint error	6 years ago
Marco Pracucci	82a7eae2bc	Removed unused (and skipped) TestGZIPCompression (#858)	6 years ago
Aditya C S	b5d8c77b99	Readiness probe for querier (#854)	6 years ago
Sandeep Sukhani	f38fa2d184	gofmt file	6 years ago