177 Commits (6e79cee85d8d5b62e524f08ae54a7d0f2f55a10b)

Author	SHA1	Message	Date
Robert Fratto	ca311c4d52	expose underlying promtail client (#2910)	5 years ago
RangerCD	b9534c65b3	Fix typos (#2853)	5 years ago
Cyril Tovena	e6778e9827	Update to go 1.15 (#2877) ... * Upgrade to 1.15. Also upgrade linter and fixes some lint issues. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update CI. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update generated files. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moar lint 🤦 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * moar lint and tests fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moar tests fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Sandeep Sukhani	071a20803b	Revendor cortex to latest master (#2794)	5 years ago
Cyril Tovena	62272cdaa1	Fixes all lint errors. (#2768) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Owen Diehl	607ccc4d77	Service discovery refactor (#2761) ... * promtail sd loading refactor + tests * flippin bools like flipper * adds back base scrapeconfig test * passes all service discovery variants	5 years ago
Owen Diehl	54c0c5febb	Revendor Cortex (#2755) ... * replaces prometheus & revendors cortex * compat * compat * sd refactor * updates docker * removes prom replace * adds delimiters back to object addrs in boltdb	5 years ago
Kaviraj	690a387cce	fix: Remove depricated `entry_parser` from scrapeconfig (#2752)	5 years ago
Calle Pettersson	426bac7f5b	Add counter promtail_batch_retries_total (#2723) ... Signed-off-by: Calle Pettersson <calle@cape.nu>	5 years ago
Ed Welch	8ea6c381cb	Promtail: Fix deadlock on tailer shutdown. (#2717) ... * to avoid a deadlock on shutdown, leave the goroutine running which reads the Lines channel from the tailer and it will exit when the tailer closes the channel. * remove the flag I added a few weeks ago and instead when updating the position information of the file gracefully fail if the file no longer exists. * update the vendoring of the tail library, more re-work on how tailing shutdown and errors are handled. * fix mistake * updating tail lib to merged commit hash * go mod tidy * update tail library with additional mutex lock * break up the tailer goroutine into two separate goroutines to make it a little more clear and remove some special case logic. * don't forget to close your channels! * make the stop method in the tailer only execute at most one time. * update hp-tail again with improvements to the mutex locking in closeFile * replace the more complicated channel based single execution code with a sync.Once	5 years ago
Ed Welch	d3bf21e774	Promtail: (and also fluent-bit) change the max batch size to 1MB (#2710) ... * change the max batch size to 1MB for all the defaults including helm and fluent-bit, attempt to centralize this config a little where possible. * fix test	5 years ago
Ed Welch	2162a0c1b1	Do not mark the position if the file is removed, doing so only results in an error because you can't get the file size of a removed file. (#2658)	5 years ago
Ed Welch	0b1dbe26fe	Promtail: Add a stream lagging metric (#2618) ... * add a metric which displays how far a stream lags the last timestamp in a batch * register your metrics * recorded metric in the wrong spot, moved to the correct spot * cardinality is much too high if we add all the stream labels, instead just use filename. * work on using an interface to set/remove metrics * reworking interface again * for now just register this metric based on filename only, we can improve this in the future if the need arises * change the timeout to 1min for expiring the metric * add host label * include host label when removing metric * change the batch size for promtail from 100kb to 1Mb, this will not affect sending for lower volumes which are still driven by the BatchWait setting. Through the addition of this metric it was found that for higher volume files > 100kb/sec this was much too low causing too many batches to be unnecessarily sent.	5 years ago
Robert Fratto	ee67b9c707	Expose UserAgent and fix User-Agent version source (#2603) ... * Use prometheus/common/version for reporting build version in User Agent header Promtail wasn't using github.com/prometheus/common/version for the User-Agent header when obtaining the build version. Instead, it imported github.com/grafana/loki/pkg/build which modifies the Prometheus version package on init. This causes problems for importers that also import and modify github.com/prometheus/common/version with their own build information (i.e., grafana/agent). This PR changes Promtail to obtain the build info from the common dependency to obtain the correct build information regardless of how Promtail is being run. * expose UserAgent to allow it to be modifiable by importers	5 years ago
Calle Pettersson	82a8cf425f	Add a counter for empty syslog messages (#2565)	5 years ago
Ed Welch	b8b24f3129	clone labels when sending them to Handle function for stdin target (#2570)	5 years ago
Ed Welch	ee28dbff2d	implement shutdown for the no-op server (#2542) ... (cherry picked from commit b4c6065b97)	5 years ago
Ed Welch	62da8bb858	restart the tailer if we fail to read and upate current position (#2532) ... (cherry picked from commit b6d9fd5a00)	5 years ago
Ed Welch	b4c6065b97	implement shutdown for the no-op server (#2542)	5 years ago
Ed Welch	b6d9fd5a00	restart the tailer if we fail to read and upate current position (#2532)	5 years ago
Ed Welch	44f8dc3e46	enable the MustExist flag when starting a tailer on a fail, this will cause the tail to fail fast and allow promtail to attempt to re-open the file. (#2491)	5 years ago
Ed Welch	7418d42141	Promtail: force the log level on any Loki Push API target servers to match Promtail's log level. (#2475) ... * force the log level on any Loki Push API target servers to match Promtail's log level. * also include logformat	5 years ago
Ed Welch	36164bbe1a	Promtail: use --client.external-labels for all clients (#2474) ... * --client.external-labels command line argument will now apply the labels to all clients configured in the clients config section of the config file. * do better Ed	5 years ago
Ed Welch	096976511f	Make a clone of the LabelSet before handling, the handler contains a full pipeline and can modify the labels uniquely per each line which will lead to some concurrent map access panics if you don't pass a copy of the labels to each Handle invocation. (#2464)	5 years ago
Robert Fratto	b5f0c75c03	pkg/promtail: propagate a logger rather than using util.Logger globally (#2438) ... * pkg/promtail: propagate a logger rather than using util.Logger globally This commit allows for creating promtail with a custom log.Logger instance which will be propagated and used consistently throughout the Promtail package. This allows for clients to provide a Promtail-specific logger. Address a portion of #2405. * remove applyOptions function	5 years ago
Robert Fratto	3451dd7c39	add RegisterFlagsWithPrefix to config structs (#2414) ... All config structs now have a RegisterFlagsWithPrefix function that allows specifying a prefixed string to append to all flag names. This may be used by clients to namespace support for Promtail in a larger application (e.g., grafana/agent). When implementing this PR, I noticed a few config structs were incorrectly registering to the global flagset rather than the flagset passed to RegisterFlags. This PR also includes a fix for that and ensures every option is registered to the flagset in the parameter. Related to #2405.	5 years ago
Aditya C S	8809de6087	refactor promtail target manager code (#2345)	5 years ago
Ed Welch	b24a143043	Promtail: Loki Push API (#2296) ... * refactor to put targets in separate packages * rename scrape package to scrapeconfig * Push implementation, working with logging * adding new vendored library * adding tests * adding documentation and an additional config option. * fixing journal test * rename scrape config entry * Update docs/clients/promtail/configuration.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * PR feedback Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	5 years ago
Roman Grebennikov	de644b52c3	promtail: fix high CPU usage on large kubernetes clusters. (#1118) ... * promtail: add api-level node filter for kubernetes discovery * promtail: fix hardcoded host value in k8s node filter	5 years ago
Calle Pettersson	baa264392b	Set user agent on outgoing http requests (#2242) ... Signed-off-by: Calle Pettersson <calle@cape.nu>	5 years ago
Cyril Tovena	d08ceef16b	Fixes iterators boundaries. (#2136) ... The start should be inclusive and the end is supposed to be exclusive. If start and end are equal, then only one entry can be returned which need to also be equal to start. Fixes #2124 Those issues were edge cases where the boundaries of a block or iterator would be equal to the start. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Owen Diehl	1530120c2e	Logql comparison ops (#2182) ... * comparison binops * starts comparison operator docs * implements bool modifier for comparison binops * vector-vector binop merging cases, including bool modifier * logql docs formatting * resets protos * linter fixes * goimports local * gofmt	5 years ago
Cyril Tovena	18f1b0640d	Bytes aggregations (#2150) ... * Add bytes_rate and bytes_over_time. Still need to works on test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More lint fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More comments. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedbacks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix bad conflic resolution. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Peter Štibraný	e4a5360586	Update vendored Cortex to master (1.1+) (#2149) ... * Updated Cortex to master (post 1.1) Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use new modules system from Cortex. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Treat query errors reported by chunk store as client errors. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed obsolete tests. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix race for ct variable. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix test. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Sebastian Widmer	d175fa58c8	Update go-syslog dependency to accept any encoding in message (#2127)	5 years ago
Aditya C S	ff0a4c35f4	Set JournalTarget Priority value to keyword (#2087)	5 years ago
Aditya C S	f984e71192	Fix Promtail journal seeking known position (#2111)	5 years ago
Ed Welch	f206165c4f	output a debug log if the requested path in the scrape config doesn't match any files (#2069) ... Signed-off-by: Ed Welch <edward.welch@grafana.com>	5 years ago
Aditya C S	11937d3c3a	Targets not required in promtail config (#2026) ... * Targets not required in promtail config * changing the defaulted value from `dummy` to `localhost`, I think this will be more intuitive for people, also re-worded the docs a bit. Signed-off-by: Ed Welch <edward.welch@grafana.com> Co-authored-by: Ed Welch <edward.welch@grafana.com>	5 years ago
Cyril Tovena	9988ce4b9f	Improve protobuf serialization (#2031) ... * Generate custom proto type for Streams. This is the first step to improve protobuf serialization. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve protobuf serialization allocations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Saves allocations by stringifying labels only once. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Aditya C S	83873d1c6f	Fix promtail client default values (#2049)	5 years ago
Josh Soref	0d9cf4946a	Spelling (#2014) ... * spelling: access * spelling: actual * spelling: asynchronously * spelling: circleci * spelling: circle * spelling: cognitive * spelling: comment * spelling: computation * spelling: correct * spelling: daemon * spelling: elliott * spelling: enabled * spelling: encbuf * spelling: extracted * spelling: explicitly * spelling: independent * spelling: languages * spelling: loose * spelling: message * spelling: metrics * spelling: mismatch * spelling: nonexistent * spelling: output * spelling: pipeline * spelling: pipe line * spelling: position * spelling: prepend * spelling: query * spelling: receive * spelling: received * spelling: reducible * spelling: registry * spelling: response * spelling: retryable * spelling: scanning * spelling: sourced * spelling: supported * spelling: template * spelling: timeout * spelling: unsupported * spelling: adapter * spelling: deferring * spelling: expirable	5 years ago
Cyril Tovena	34f9c601c9	Adds config to disable HTTP and GRPC server in Promtail. (#2004) ... Fixes #1991. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	658dcb8416	Support stdin target via flag instead of automatic detection. (#1935) ... This seems to causes issue on some environment so let's use a flag instead. Fixes #1921 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
alexanderGalushka	5e752bf343	feat: tms readiness check bypass implementation ... fix: renamed flag to health_check_target, default to true fix: flattened nested if statements	5 years ago
Cyril Tovena	8e80160db7	Removes file size check for pipe, not provided by linux. (#1893) ... * Removes file size check for pipe, not provided by linux. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Disable color on windows. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> (cherry picked from commit 75730782a7)	5 years ago
Cyril Tovena	75730782a7	Removes file size check for pipe, not provided by linux. (#1893) ... * Removes file size check for pipe, not provided by linux. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Disable color on windows. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Aditya C S	e713a8d83a	Fix flaky promtail test (#1865)	5 years ago
Edward Welch	3c479c8f9c	promtail retry 429 rate limit errors, clarifying risks in the docs of configuring multiple client sections in promtail, also increased the backoff and retry settings in promtail. ... Signed-off-by: Edward Welch <edward.welch@grafana.com> (cherry picked from commit 6841c418ef)	5 years ago
Edward Welch	6841c418ef	promtail retry 429 rate limit errors, clarifying risks in the docs of configuring multiple client sections in promtail, also increased the backoff and retry settings in promtail. ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	5 years ago