apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Michael Sverdlik	5206e3acd0	Docs: Hint about potential high cardinality metrics (#5603 ) * Hint about potential high cardinality metrics Add warning about potential high cardinality metrics due to the file path being included in some of the metrics. Depending on the setup, such config can result in a very large amount of label values, which is not recommended by Prometheus. This can result in increased storage, slow down of queries, extra costs and so on. Address what has been discussed in #5553. * Reword observability.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Gábor Farkas	0075085e9a	docs: describe ordering in http query results (#6024 ) * docs: describe ordering in http query results * improved text Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * better text Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * better text Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Joan López de la Franca Beltran	5e465ad74e	Docs: Fix design docs index page (#6008 ) * Fix design docs index * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Christian Haudum	d5c337c4d3	docs: Fix broken link from unpack parser to pack stage (#6207 ) and make hyperlinks more obvious Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Kaviraj Kanagaraj	a58dc3f6a8	doc(storage-example-configs): AWS single store configs (#6196 ) * doc(storage-example-configs): AWS single store configs Our example configs are bit misleading here. Particularly AWS deployment. https://grafana.com/docs/loki/latest/storage/#examples As a new user, I come here and all I can conclude is, we cannot use s3 as a single store under AWS deployment. Doc seems to recommend AWS deployment always needs S3 + DynamoDB. (in fact it was real confusion for new user from the community) https://grafana.slack.com/archives/CEPJRLQNL/p1652867888720979 Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Update schema_config.configs.store Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Dylan Guedes	440fb7d74c	Loki: Modifies TableManager to use IndexGateway ring (#5972 ) * Add two metrics to the IndexGateway. - Add a new `query_readiness_duration_seconds` metric, that reports query readiness duration of a tablemanager/index gateway instance. We should use it later to report performance against the ring mode - Add a new `usersToBeQueryReadyForTotal` metric, that reports number of users involved in the query readiness operation. We should use it later to correlate number of users with the query readiness duration. * Remove `usersToBeQueryReadyForTotal`. - It will report all users always for now, so it isn't too helpful the way it is. * Rename metric help text to not mislead people. * Log queryReadiness duration. * Fix where log message and duration and triggered. * Add two metrics to the IndexGateway. - Add a new `query_readiness_duration_seconds` metric, that reports query readiness duration of a tablemanager/index gateway instance. We should use it later to report performance against the ring mode - Add a new `usersToBeQueryReadyForTotal` metric, that reports number of users involved in the query readiness operation. We should use it later to correlate number of users with the query readiness duration. * Remove `usersToBeQueryReadyForTotal`. - It will report all users always for now, so it isn't too helpful the way it is. * Rename metric help text to not mislead people. * Log queryReadiness duration. * Fix where log message and duration and triggered. * Use boundaries to skip users in TableManager. - Separate the assigning of indexClient in the IndexGateway to allow initializing the shipper with the IndexGateway ring - Add new TenantBoundariesClient entity, that answers if a given tenantID should be ignored or not - Use the TenantBoundariesClient implemented by the IndexGateway in the downloads TableManager * Add IndexGateway configuration docs. * Add tests for pkg/util/ring TokenFor() and IsAssignedKey() Signed-off-by: JordanRushing <rushing.jordan@gmail.com> * Small whitespace fix in pkg/util/ring_test.go Signed-off-by: JordanRushing <rushing.jordan@gmail.com> * Apply suggestions from code review Rewrite QueryIndex error phrase. Co-authored-by: JordanRushing <rushing.jordan@gmail.com> * Join users list in a single string. - This is necessary since go-kit doesn't support array type. * Tweak queryReadiness log messages. - As suggested by Ed on https://github.com/grafana/loki/pull/5972#discussion_r859734129 and https://github.com/grafana/loki/pull/5972#discussion_r859736072 * Implement new RingManager. - Adds a new entity to indexgateway named RingManager, responsible for managing the ring and the lifecycler used by the indexgateway. The ringManager is guaranteed to be initiatiated before the Shipper and before the IndexGateway. - Remove the readiness logic from the IndexGateway. - Pass the RingManager as the TenantBoundaries implementation of the Shipper * Return non-empty ringmanager for all modes. * Fix lint regarding error not being check. * Fix lint due to wrong import order. * Implement support for client and server mode for the ring manager. * Fix ring manager services registration. * Add option to configure whether or not to log gateway requests. * Check if IndexGateway is enabled instead of active. * Tune RingManager behavior. - Instantiate ring buffers inside IsAssignedKey instead of reusing same buffer to avoid issues with concurrency. - Remove unnecessary details from IndexGateway mode docs. - Return error when wrongly instantiating a RingManager when IndexGateway is in simple mode. * Rewrite `TenantInBoundaries` as a func instead of interface. * Fix lint. - Fix YAML tag - Remove else clause * Use distributor instead of querier in test. - Since the querier needs to wait for the index gateway ring, it isn't suitable for this test anymore. * Add ring mode guard clause. * Return code erased by mistake. * Rename TenantInBoundaries to IndexGatewayOwnsTenant. * Log len of users instead of all of them to avoid log msgs. * Add docstrings to new public functions. * Apply suggestions from code review Document that tenant filtering is only applied during query readiness. Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Modify IsAssignedKey to expect address directly instead of lifecycler. - Also removes HasDedicatedAddress, since it isn't necessary anymore * Log error message when IsInReplicationSet fails. * Modify IsAssignedKey to return true by default. * Remove wrong assigning of Service for clientMode. * Log gateway requests before error checks. * Remove unnecessary endpoint registration. * Fix lint. * Update pkg/util/ring.go Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: JordanRushing <rushing.jordan@gmail.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	4 years ago
Karen Miller	b90c460769	Rename fudge_duplicate_timestamp to be increment_duplicate_timestamp (#6120 ) * Rename fudge_duplicate_timestamp to be increment_duplicate_timestamp * run `gofmt -d -w pkg/validation/limits.go` Co-authored-by: Christian Simon <simon@swine.de>	4 years ago
Michel Hollands	e5eff96266	Change deletion_enabled setting to deletion_mode (#6190 ) * Change deletion_enabled setting to deletion_mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Empty commit	4 years ago
Daniel Kovacs	e9d2a30733	Fixed reader to support both authentication and tenant-id at the same time. (#5719 )	4 years ago
Kaviraj Kanagaraj	e3a43c6972	query-scheduler: Now proper support for `querier_forget_delay` (#6174 ) * query-scheduler: Now proper support for `querier_forget_delay` Since now shuffle sharding is supported on the `query-path` in Loki, it makes sense to enable this config. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Update pkg/scheduler/scheduler.go Co-authored-by: Christian Simon <simon@swine.de> * Update docs/sources/configuration/_index.md Co-authored-by: Christian Simon <simon@swine.de> Co-authored-by: Christian Simon <simon@swine.de>	4 years ago
Periklis Tsirakidis	6e9517e4b5	ruler: Add support for alertmanager header authorization (#6136 )	4 years ago
Sergio Rua	069156380f	Update SSL config for cassandra (#6145 ) Documentation was missing the references to the SSL key and cert	4 years ago
Aditya C S	c6af640798	feat: support simple selection of labels in json expr parser (#6151 )	4 years ago
Justin Burnham	97c7d7ca3c	LogQL: add the `default` sprig template function to logql label/line formatter (#6163 ) * add the `default` sprig template function to logql label/line formatter With the `default` sprig template function, we will be able to make it easier to output a default value if the source string is otherwise empty. Useful for json fields that can be missing, like HTTP headers in a log line that aren't required as in the following: ``` {job="access_log"} \| json \| line_format `{{.http_request_headers_x_forwarded_for \| default "-"}}` ``` Previously this could only be done (that I found) with the much more verbose ``` {{regexReplaceAllLiteral "^$" .http_request_headers_x_forwarded_for "-"}} ``` * update changelog entry * add pr number * enhance docs * Update docs/sources/logql/template_functions.md Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> * Update docs/sources/logql/template_functions.md Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	4 years ago
Michel Hollands	f39552c7fa	Add docs for updated deletion api (#5763 ) * Add docs for updated deletion api Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix parameter name Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Clarify that only the delete requests for the current tenant are shown Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update docs/sources/api/_index.md Co-authored-by: Christian Simon <simon@swine.de> * Update docs/sources/operations/storage/logs-deletion.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Update docs/sources/operations/storage/logs-deletion.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Update docs/sources/operations/storage/logs-deletion.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Specify all delete endpoints use authentication Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add description of delete modes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Christian Simon <simon@swine.de> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
psytester	40b2373398	corrected copy&paste error in list of labels (#6146 ) response_size was twice mentioned	4 years ago
Karen Miller	31ca108a3c	Docs: redo PR 5109 (#6119 )	4 years ago
Danny Kopping	8838530976	Document `/services` API endpoint (#6111 ) * Documented /services API endpoint Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Grammar Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
JordanRushing	45dd588d8f	Update consistent hash ring docs with new index gateway ring (#6041 ) * Update consistent hash ring docs with new index gateway ring Signed-off-by: JordanRushing <rushing.jordan@gmail.com> * Update docs/sources/fundamentals/architecture/rings.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
arcosx	e2fdc3437b	Feat: add Baidu Cloud BOS as storage backends for Loki #4788 (#5848 ) * feat: add baidu bce bos storage support Signed-off-by: arcosx <arcosx@outlook.com> * add baidu bce bos as chunk storage backend Signed-off-by: arcosx <arcosx@outlook.com> * fix: some doc error && rewrite bad code Signed-off-by: arcosx <arcosx@outlook.com> * fix: add the `BceServiceError` source link Signed-off-by: arcosx <arcosx@outlook.com> * Update CHANGELOG.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Signed-off-by: arcosx <arcosx@outlook.com> * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update pkg/storage/chunk/client/baidubce/bos_storage_client.go Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update pkg/storage/chunk/client/baidubce/bos_storage_client.go Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update pkg/storage/chunk/client/baidubce/bos_storage_client.go Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Artem Timchenko	ebb39db864	lambda-promtail: Add multi-tenancy support (#6102 ) * lambda-promtail: adding multi-tenancy support * update changelog and fix import order * fix import order * fix import order	4 years ago
Ed Welch	4f75939efb	Revert "Loki: Change live tailing to only allow mutating the log line not the number of streams. (#6063 )" (#6097 ) This reverts commit `98fda9b0a0`.	4 years ago
n4mine	14b0873f2b	Revert "Docs: fix pic location in `sources/fundamentals/architecture/deployment-modes.md` (#6089 )" (#6093 ) This reverts commit `1d25e17b32`.	4 years ago
Gábor Farkas	0a68bd2d4e	docs: update the labels timespan info (#6019 ) * docs: update the labels timespan info * updated docs Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * updated docs Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
n4mine	1d25e17b32	Docs: fix pic location in `sources/fundamentals/architecture/deployment-modes.md` (#6089 )	4 years ago
Karen Miller	2264b73311	Docs: remove Cortex references (#6079 ) * Docs: remove Cortex references * Bug fix: (after PR 6015) Cortex to Loki update	4 years ago
Christian Haudum	7fff660144	Add UDP protocol support to Promtail's syslog target (#5790 ) Until now, the syslog target in Promtail did only support TCP. This PR adds support for sending syslog messages to Promtail (via ng-syslog or rsyslog) via UDP. UPD for syslogs can be used if you prefer speed over guaranteed delivery. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Ed Welch	98fda9b0a0	Loki: Change live tailing to only allow mutating the log line not the number of streams. (#6063 ) * don't create more streams when live tailing logs, process the log line with the pipeline to allow mutating the content of the line but don't split it into new streams. Signed-off-by: Edward Welch <edward.welch@grafana.com> * update changelog Signed-off-by: Edward Welch <edward.welch@grafana.com> * don't mutate the incoming streams Signed-off-by: Edward Welch <edward.welch@grafana.com> * do not reuse the pipeline because it caches labels Signed-off-by: Edward Welch <edward.welch@grafana.com> * add note to upgrade guide Signed-off-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Corentin Altepe	d9327ecd0c	Added NLog-Targets-Loki as unofficial C# client to Loki (#6061 ) Added a link to the HTTP C# client [NLog-Targets-Loki](https://github.com/corentinaltepe/nlog.loki) in Loki's clients documentation in the unofficial clients sections, right next to its Serilog counterpart.	4 years ago
W.T. Chang	9c19b0f548	Fix broken link on deployment-modes.md (#6068 ) The configuration link should go another parent level further in order to get `docs/loki/latest/configuration/`, currently it points to `docs/loki/latest/fundamentals/configuration/`.	4 years ago
Paschalis Tsilias	179d741c5f	promtail: Add support for exclusion patterns in `static_config` (#5943 ) * PR base Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add CHANGELOG entry; updated documentation for new label Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Follow logic from t.path in exlusion case; always handle matching Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Make error message unique Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Only run if t.pathExclude is set Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Revert "Only run if t.pathExclude is set" This reverts commit 17abb006ce141071c4bca4b7d133bea2aa314594.	4 years ago
Karen Miller	a5b9a9a91c	Docs: Improve wording of getting started guide (#6060 ) - Add the -d option to all docker-compose up commands - Make the wording consistent about the CWD	4 years ago
Karen Miller	f031427bec	Docs: PR 5387 redo - Warn ECS users to avoid plain text creds (#6051 )	4 years ago
Karen Miller	c4ebfd3020	Docs: revise getting started guide (#5939 ) * Revise the getting started section * Docs: revise getting started guide * Intermediate commit of getting started material. * Intermediate revision of the getting started guide. Add instructions to download, configure, and deploy the test environment. * Add sample queries and instructions for breaking down the test env * Revise prose using reviewer comments. * update to use promtail and flog in the docker compose file Signed-off-by: Edward Welch <edward.welch@grafana.com> * Revise guide to put all parts into Docker containers. * Add README files for examples directory. Revise getting started instructions. * Address DylanGuedes review comments * Getting started guide: incorporate suggestions from ssncferreira Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Alexandre de Verteuil	149227b889	Minor fixes in docs/sources/api/_index.md (#5915 )	4 years ago
Ed Welch	fc8c4f0592	Loki: Add a configurable ability to fudge incoming timestamps for guaranteed query sort order when receiving entries for the same stream that have duplicate timestamps. (#6042 ) * Add a per tenant ability to fudge duplicate timestamps in the distributor Signed-off-by: Edward Welch <edward.welch@grafana.com> * update docs Signed-off-by: Edward Welch <edward.welch@grafana.com> * lint Signed-off-by: Edward Welch <edward.welch@grafana.com> * update changelog Signed-off-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Trevor Whitney	2a04ac9af7	Remove references to cortex (#6015 )	4 years ago
Travis Patterson	9cef86b162	Invalidate caches on delete (#5661 ) * Generate cache invalidation numbers in the delete store * Get cache generation numbers from the store on request * changlog * rename tombstones to something more meaningful * User invisible module * query frontend relies on a compactor to get the cache generation number * fix serialization * source -> name * lint errors * lint errors * log non-200 responses * add jsonnet changes * lint * review feedback * review feedback * client rename	4 years ago
Christian Haudum	0c72c8ce7a	Other: trigger build of loki-build-image:0.20.3 (#5924 ) * Trigger build for Loki build image version 0.20.3 In commit `435391661f` the version was increased from 0.20.2 to 0.20.3 in the `.drone/drone.jsonnet` file, but not in the `.drone/drone.yml`, which caused the image not to be built. In order to trigger the build of 0.20.3, contents of the `loki-build-image/` folder need to change, therefore the comment in the Dockerfile. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Update documentation how to build and use the loki-build-image The existing documentation was not clear about the two-step process for building and using the image. This PR tries to improve that. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Update documentation how to build and use the loki-build-image Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
johgsc	27a536718f	Update _index.md (#5934 ) changed URLs for [Fluentd](https://github.com/grafana/loki/tree/main/clients/cmd/fluentd) and [Fluent Bit](https://github.com/grafana/loki/tree/main/clients/cmd/fluent-bit) since the old URLs where leading to a 404 on GitHub.	4 years ago
Periklis Tsirakidis	2b4ebbd578	Add SIG operator team to governance doc (#4977 ) * Add SIG operator team to governance doc * Update docs/sources/community/governance.md Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * Update docs/sources/community/governance.md Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com>	4 years ago
Karen Miller	03d4238700	Fix SSD Docker installation (#5916 ) * Fix SSD Docker installation * Clarify endpoints and ports	4 years ago
Marcus Teixeira	23ff2e9ddb	docs: fix query_ingester_only config (#5873 ) * docs: fix query_ingester_only config * fix: change msg when query_store_only and query_ingester_only is enabled	4 years ago
kaige0207	bbebc6b2fb	Update _index.md (#5887 ) Modify the command line description of the grpc_server_max_send_msg_size parameter	4 years ago
Nicolas Vanheuverzwijn	f22dbc2190	Update _index.md (#5871 ) The new default for : - `chunk_retain_period` is `0s`, not `30s` - `chunk_idle_period` is `30m`, not `1h` - `chunk_target_size ` is `1572864`, not `1048576` According to https://grafana.com/docs/loki/v2.4/configuration/#ingester_config	4 years ago
Ed Welch	c6c301ec27	Docs: Improve the new schema docs (#5869 ) * improve the new schema docs Signed-off-by: Edward Welch <edward.welch@grafana.com> * fix capitalization Signed-off-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Sandeep Sukhani	87ed13715a	document flag for disabling broad index queries (#5880 )	4 years ago
Callum Styan	800b7c6345	[promtail] Mark promtail single client config flags and config file block as deprecated (#5842 ) * Mark promtail single client config flags and config file block as deprecated. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update some promtail docs examples that still use `client` instead of `clients`. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix imports. Signed-off-by: Callum Styan <callumstyan@gmail.com> * properly format url as an array item since we changed to clients from client Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Alexandre de Verteuil	b08ddc5f93	Minor fix in docs/sources/logql/log_queries.md (#5868 )	4 years ago
Ed Welch	4e421206f0	Docs: Add usage reporting section to 2.5 release notes (#5865 )	4 years ago

1 2 3 4 5 ...

929 Commits (0dfd2cbbcc15d888466b520ee9345be9b7c8ee8a)