248 Commits (cb20afaa26d1e8fd285554129e32843a21afbe5a)

Author	SHA1	Message	Date
Ed Welch	691d27c8c7	Update Loki build image (#2467) ... * updating build-image version * updating drone config * updating a few other references to the build image	5 years ago
Ed Welch	204e61afe3	Loki-Canary: Backoff retries on query failures, add histograms for query performance. (#2413) ... * Canary needs to wait before spot-checking entries some reasonable amount of time else the nature of the async thread doing the testing can ask for them from Loki only a few ms after they are written and they would have no chance of being present. Adding two histograms for tracking the performance of the spot check and metric test queries * Adding backoff to queries if they fail specifically waiting 5mins when a 429 is received from Loki * fix order of parameters	5 years ago
Aditya C S	43f2ef33b3	logstash code refactor and doc improvements (#2410) ... * logstash code refactor and doc improvements * added back gradlew assemble	5 years ago
Cyril Tovena	4e2b37b2f8	bump fluentd version to release a new gem. (#2407) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Sándor Guba	35feea5a58	make fluentd verson more flexible for updatess (#2399)	5 years ago
Cyril Tovena	45af549aeb	Update fluentd docs and fixes links for the website.i (#2391) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	3d8688fae4	Fluent bit docs (#2390) ... * Improve documentation and fix all links for fluentbit. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Wrong file Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Reset fluentd Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add reference to ECS. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	23f64ef7e1	Docker driver doc (#2389) ... * wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Merge all documentation for docker-driver in one place. Fixes #2198 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	81c7bd33f5	Fixes the fluentbit batchwait backward compatiblity. (#2375) ... We move from second `1` to duration `1s`, but we can keep that compatible. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Ed Welch	76ceec45d9	If an entry is missing on the websocket, start checking after `wait` but continue checking until `max-wait` (#2369) ... Refactored out a useful pruneList function	5 years ago
Aditya C S	4c6090756f	Loki Logstash Plugin (#1822) ... * Logstash plugin * include_labels * include_labels * Removes binary. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve documentation and remove the push path. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Move to cmd. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more precision for jruby. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs/clients/logstash/README.md * p Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * ignore Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * remove ignore file/ Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More precision for installing jruby Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Rename without Grafana Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * A lot of refactoring and testing. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * change delay logic * Fully tested version. Now testing/writing docs and charts. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Forgot to save merge. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * working version. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Makefile + easier docker build. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * adds ci to build logstash image. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix build for logstash. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds example with helm charts. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix target to send 10 logs with logstash. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improved documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * add missing helm add repo for external repo Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review comment. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes loki service in Promtail. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update loki-stack version Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Martin Dojcak	ca94b2cc91	Fix fluent-bit exit callback deadlock when buffering for loki is enabled (#2365)	5 years ago
Martin Dojcak	eb39e8283f	fluent-bit: Add more loki client configuration options (#2367) ... * Add more loki client configuration options to flb output plugin * Fix/Add config option tests	5 years ago
Ed Welch	2b145da7d7	Canary tweaks (#2350) ... * tweak the canary a bit, add a metric for websocket pings, fix some logging, improve the mutex locking a bit, add a read timeout on the websocket, add a lastmessage time checker to restart websocket * fix constructor * fix counter name change metric count reporting to two separate metrics, expected vs actual * add note to upgrade guide for metric name changes * fixed tests.... * Update docs/operations/upgrade.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	5 years ago
Ed Welch	81b9decfb5	Loki-Canary: Add query spot checking and metric count checking (#2344) ... * Add spot check capability to canary to query certain values over time to make sure they are still in Loki. Add a context timeout to Loki queries * Updating canary to use /loki/api/v1 endpoints adding a metric test query * fixing metric names to have _total for counters * fix parsing of timestamp from log line and don't use entry timestamp * Tweak the metric test query to adjust the range to the time the canary has been running to get accurate results until the requested range has been reached. Add a total spot check entry counter to get a ratio of success * Add some debugging logs to troubleshoot missing entries * add more tests, fix some log lines * update docs, make the spot-check timer separate from prune entries * just how many paramaters can I add to the constructor.... * run prune entries on a separate thread, only run one at a time.	5 years ago
Cyril Tovena	2548d6986d	Remove mount in the docker driver. (#2295) ... * Remove mount in the docker driver. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint json Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Johnathan Falk	0ace187e17	Change ignoring empty line to debug logging. (#2343) ... Signed-off-by: Johnathan Falk <johnathan.falk@gmail.com>	5 years ago
Ed Welch	0b5996021f	Loki: add -print-config-stderr flag to dump loki's runtime config to stderr (#2336) ... * Loki: add -print-config-stderr flag to dump loki's runtime config to stderr * add ability to print inline config for better readability in grafana * add an option to log the config in reverse order to make viewing in Grafana easier. Also added this capability to Promtail	5 years ago
Ed Welch	b24a143043	Promtail: Loki Push API (#2296) ... * refactor to put targets in separate packages * rename scrape package to scrapeconfig * Push implementation, working with logging * adding new vendored library * adding tests * adding documentation and an additional config option. * fixing journal test * rename scrape config entry * Update docs/clients/promtail/configuration.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * PR feedback Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	5 years ago
Cyril Tovena	3defe3a5bd	Default to 3100 port for Loki without requiring NET_BIND_SERVICE capability. (#2294) ... Mistakes were made. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	19be2c4732	Fixes the lint issue merge to master. (#2290) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Aditya C S	384520436d	Fix fluent-bit \n and \t characters (#2286)	5 years ago
Owen Diehl	7682b32cba	Revert "Revert "Update go from 1.13 to 1.14. (#2013)"" (#2029) ... * Revert "Revert "Update go from 1.13 to 1.14. (#2013)"" This reverts commit e8fece69e3. * Update cortex to latest. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update go.sum and doc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Lukas Grimm	17390fd45d	Canary: make stream configurable (#2259) ... * tried to fix canary in non kubernetes env * update doc	5 years ago
Cyril Tovena	1e19aa4715	Add performance profile flags for logcli. (#2248) ... * Add performance profile flags for logcli. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update the logcli documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove comment. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * remove unused import. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	732fe6d14f	Adds default -config.file for the promtail docker images. (#2196) ... Closes #2194. This is already the case for Loki. I think it's better to have this since most people would expect it as default at least for docker. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	79a90dc4f6	Docker driver relabeling (#2199) ... * Add relabeling to docker driver using `loki-relabel-config` log option. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add docs. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Periklis Tsirakidis	815c47533e	Replace memberlist service in favor of cortex provided service (#2224) ... * Bump cortex deps to include non-experimental memberlist service * Replace memberlist service in favor of cortex provided service * Fix route method param for querytee tool * Register memberlist flags * Add documentation and example for memberlist	5 years ago
Takayuki Watanabe	e2a7941e32	Add server TLS certificate verification (#2171) ... * Add server TLS certificate verification * Generalize use of server TLS certificate verification * Fix typo * Bump fluent-plugin-grafana-loki gem version: 1.2.12 -> 1.2.13 * Replace option: verify_tls -> insecure_tls * Update document * Update docs/clients/fluentd/README.md Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Sandeep Sukhani	f04bc99e05	query tee proxy with support for comparison of responses (#2211)	5 years ago
Takayuki Watanabe	59f312545c	Improve fluentd plugin development experience (#2179) ... * Add Gemfile.lock and .rspec_status to .gitignore * Converge gem definition in gemspec file * Add missing `bundle exec` to README * s/FluentD/Fluentd/g	5 years ago
Periklis Tsirakidis	6e55277552	Add flag to disable tracing (#2191)	5 years ago
Owen Diehl	1530120c2e	Logql comparison ops (#2182) ... * comparison binops * starts comparison operator docs * implements bool modifier for comparison binops * vector-vector binop merging cases, including bool modifier * logql docs formatting * resets protos * linter fixes * goimports local * gofmt	5 years ago
Cyril Tovena	bedab491ac	Allows to change the log driver mode and buffer size. (#2116) ... Before those configs where not allowed, but docker can use them to have a different log delivery behaviour. see https://docs.docker.com/config/containers/logging/configure/#configure-the-delivery-mode-of-log-messages-from-container-to-log-driver Potentially fixes #2017 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	a01787af19	Update old link for pipeline stages. (#2172) ... Fixes #2165	5 years ago
Cyril Tovena	4c8199ec21	Validate local storage config for the logcli (#2126) ... * Allows to pass inlined pipeline stages to the docker driver. Fixes #1296 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add validation before using storage config for the logcli. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Martin Dojcak	b27f92e86b	Add FIFO queue persistent buffering for fluent bit output plugin (#2142) ... * Add FIFO queue persistent buffering for fluent bit output plugin * Fix configuration names in logging * Fix typos & suggested improvments	5 years ago
Aditya C S	360d9a7f97	Support querying labels on time range in logcli (#2083)	5 years ago
Or Sela	835dd29316	fluentd plugin: support placeholders in tenant field (#2074)	5 years ago
Cyril Tovena	c2f100c334	Allows to pass inlined pipeline stages to the docker driver. (#2076) ... * Allows to pass inlined pipeline stages to the docker driver. Fixes #1296 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * doc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Bohdan Kmit	f5b9cffc2e	Docker driver: Allow to provision external pipeline files to plugin (#2054) ... * Use latest Alpine version * Allow to provision external pipeline files to plugin * Add custom pipeline documentation * Untabify on reviewer request Co-authored-by: Mike Futerko <mike@maytech.net>	5 years ago
Emil 'Skeen' Madsen	33a81528ec	fluent-plugin-loki: Restructuring and CI (#2006) ... * Move fluentd/fluent-plugin-grafana-loki to cmd/fluentd * Integrate fluentd with Makefile * Added drone-ci job for fluentd * Fix typo fluent-bit --> fluentd in .drone/drone.jsonnet Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> * Change default LOKI_URL in FluentD Dockerfile Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> * Utilize upstream configuration file * Test setup, adapt default configuration * Regen drone.yml * Change default label to job=fluentd, rather than env=dev Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Owen Diehl	2d31d07680	Revert "Update go from 1.13 to 1.14. (#2013)" (#2028) ... This reverts commit 8d6dc2c4ca.	5 years ago
Cyril Tovena	8d6dc2c4ca	Update go from 1.13 to 1.14. (#2013) ... * Update go from 1.13 to 1.14. See https://golang.org/doc/go1.14 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * modules.txt. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Ed Welch	c7a1a1fb2e	Loki: Adds an `interval` paramater to query_range queries allowing a sampling of events to be returned based on the provided interval (#1965) ... * Adds an `interval` paramater to query_range queries allowing a sampling of events to be returned based on the provided interval Signed-off-by: Ed Welch <edward.welch@grafana.com> * lint Signed-off-by: Ed Welch <edward.welch@grafana.com> * extract function for parsing parameter Signed-off-by: Ed Welch <edward.welch@grafana.com> * Clarify the experimental nature of interval in the docs. Signed-off-by: Ed Welch <edward.welch@grafana.com> * make min time a constant Signed-off-by: Ed Welch <edward.welch@grafana.com> * Expand the docs around the experimental nature of interval a little more Signed-off-by: Ed Welch <edward.welch@grafana.com>	5 years ago
Ed Welch	e90e3b769f	incorporate some better defaults into table-manager configs (#1976) ... * Set less aggressive poll interval and a larger grace period for creating new periodic tables in the ksonnet config. Add some rate limiting and retry settings for the table manager bigtable arguments. Signed-off-by: Ed Welch <edward.welch@grafana.com> * Removing the dynamodb provisioning of 0 from config files Signed-off-by: Ed Welch <edward.welch@grafana.com>	5 years ago
Ben Grabham	97788abcc5	[Loki]: Cleanup dockerfile (#1949) ... * Reduce the amount of layers and data in the Dockerfile * Change the loki user and data directory to match the default helm configuration * Add an nsswitch.conf to stop localhost DNS requests * Copy changes to Dockerfile.cross except setep setep is probably not needed anymore due to 3100 being the default port. Also, I'm not sure the amd64 binary works on other arch binaries. * Install ca-certificates earlier and split creating user and creating dir steps * Install libcap earlier and don't bother deleting it as it doesn't make the image smaller * Revert config changes * Revert the dockerfile changes needed for the config revert	5 years ago
Peter Štibraný	e60164bef6	Convert Loki modules to services (#1804) ... * Loki now uses module services to start and stop its work. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use services methods to initialize some components. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use Cortex' NewModuleService. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted server to a service. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted distributor to service. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use table manager service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * querier service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * query-frontend service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Merged stopping method into shutdown Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted ingester to a service. It now starts all background tasks in Starting state. Stopping needs little work, as does reacting on lifecycler errors. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Loki Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * If lifecycler fails, ingester fails too. It now doesn't call os.Exit, but shuts down gracefully and enters Failed state. That triggers Loki to shutdown completely. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Ignore ErrStopProcess errors from services This is a signal that Loki should stop. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use single /ready handler It checks the state of all services, and asks ingester for its own check as well. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed unused value. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Lint Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix test. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Go mod tidy, vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Tailers, not trailers. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Adds return for the healtcheck in case of error. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	4a208ce6e6	Improve the local config to work locally and inside docker. (#1963) ... * Improve the local config to work locally and inside docker. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add a file specifically for docker loki config. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	e4d78b984b	Log docker driver config map. (#1903) ... This will allow users to verify which config option are passed to the driver. See dicussions here https://github.com/grafana/loki/issues/1868, it seems that it can be confusing when using docker-compose. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago