apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Karen Miller	27477bf5f5	Docs: improve API docs (#6485 ) * Docs: improve API docs * Implement review suggestions.	3 years ago
Christian Simon	b8c9672de6	Update loki's default port to 3100 (#6349 ) * Update loki's default port to 3100 I was suprised this was not already the case. I think it would be a good change to make, but I am also contious of the breaking nature of this. * Add note to the changelong * Add upgrade note * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Michel Hollands	f80e487dcf	Remove whole stream deletion mode (#6435 ) * Remove whole-stream-deletion mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove whole-stream-deletion from docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update the changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Sort changelog entries Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove link to wrong configuration Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix integration test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Set default deletion mode to disabled Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove extra white line in documentation Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix default value in docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add DeletionEnabled method on mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename test Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Karen Miller	167c95c320	Docs: revise multi-tenancy prose (#6355 ) * Docs: revise multi-tenancy prose * Update multi-tenancy docs using review comments	3 years ago
Conor Evans	b65e11ec9b	fix(documentation): correct spelling and scrape_config (#6454 ) Signed-off-by: Conor Evans <coevans@tcd.ie>	3 years ago
Mohamed-Amine Bouqsimi	4f4b42ba73	Add TLS config to query frontend (#6444 ) * added tls config to query frontend * changelog * remove commented code * lint	3 years ago
Salva Corts	1cfc30fb23	Evenly spread queriers across available nodes (#6415 ) * Evenly spread queriers across available nodes * Fix lint issue * Add entry to the CHANGELOG and the Upgrade Guide * Make topology spread configurable * Apply CR feedback	3 years ago
Michel Hollands	b4e6c59936	Compactor: add per tenant compaction delete enabled flag (#6410 ) * Add per tenant compaction delete enabled flag Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove changes in wrong place Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add compactor deletion enabled field Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use limit in compactor Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use http middleware and add test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix lint issue Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Revert to default setting if no override Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add default value command line option Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update the docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Enable access to deletion API for integration test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename flag to allow_deletes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update per review comments Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Christian Haudum	a56a88bd35	Add unwrapped aggregation `rate_counter()` (#6412 ) `rate_counter()` takes an unwrapped range as input, but unlike `rate()` it treats the values extracted from the log lines as "counter metric" like in Prometheus' `rate()` function. This is a replacement for the reverted change of #5013 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Christian Haudum	b315ed0ddf	Sum values in unwrapped rate aggregation instead of treating them as counter (#6361 ) * Revert unwrapped rate aggregation to previous implementation This PR reverts the implementation done in #5013 to the original implementation that sums the extracted values from the log lines instead of treating them like a Prometheus counter metric. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Move changelog entry Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Remove unused/dead code Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Clean changelog Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Karen Miller	ba7e36afd1	Docs: Edits for several PRs that did not have a docs review (#6398 )	3 years ago
Karen Miller	1ab642f970	Docs: add unpopulated v2.6 release notes section (#6354 )	3 years ago
Christian Haudum	c12a1f4f43	feature: Add ingester handler for shutdown and forget tokens (#6179 ) * Update grafana/dskit to 07166f9 Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add /ingester/shutdown handler This handler replaces the deprecated /ingester/flush_shutdown handler and can be used to gracefully shut down a Loki instance and delete the file that persists the tokens of the ingester ring. In production environments you usually want to persist ring tokens so that during a restart of an ingester instance, or during rollout, the tokens from that instance are not re-distributed to other instances, but instead kept so that the same streams end up on the same instance once it is up and running again. For that, the tokens are written to a file that can be specified via the `-ingester.tokens-file-path` argument. In certain cases, however, you want to forget the tokens and re-distribute them when shutting down an ingester instance. This was already possible by calling `/ingester/flush_shutdown`, deleting the tokens file and terminating the process. The new handler `/ingester/shutdown` combines these manual steps into a single handler. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add changelog entry Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Trevor Wood	aed11c25e6	fix(loki): allow global and per tenant sigv4 config (#6358 ) Signed-off-by: Trevor Wood <Trevor.G.Wood@gmail.com>	3 years ago
Wu Zhiyuan	f61cbe4288	GCS: Add service_account option (#6366 ) * gcs client: add service_account support * Update docs for GCS's service_account * Fixed lint check failed	3 years ago
Owen Diehl	3f4a663dbe	Tsdb/index sampling endpoint (#6347 ) * index sampling endpoint * fix bug in withTenantLabelMatcher * show zero values in stats * handles new errcase * fixes comments	3 years ago
Michał Jaroń	57d543eff7	Update _index.md - an unofficial Loki client added (#6312 ) I would like to add mjaron-tinyloki-java to unofficial Loki clients. It is standalone small library without any dependencies. It may be an alternative to other libraries which use many dependencies and require more configuration to send logs.	3 years ago
Jan-Otto Kröpke	a1e0298a57	Support tenant id from value (#6290 ) Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>	3 years ago
Christian Haudum	9a82f5c04a	Look up config file in default locations (#6160 ) * Look for config.yaml file in ./ and ./config directories This change makes Loki look for a config file in the current directory or in the config/ sub-directory. This is a pre-requisite for making an easy-to-use tarball distribution that also ships a config.yaml Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Rename YAMLFlag to ConfigFileLoader The `YAMLFlag` function is solely used for loading config files. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Look for config.yaml file in ./ and ./config directories * fixup! fixup! Look for config.yaml file in ./ and ./config directories Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! fixup! fixup! Look for config.yaml file in ./ and ./config directories Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Trevor Wood	0470e4e7d3	feat(fluentd): allow fluentd_thread label to be configurable (#6240 ) Signed-off-by: Trevor Wood <Trevor.G.Wood@gmail.com>	3 years ago
Karsten Jeschkies	ff5ce4e129	Document multi-tenant queries. (#5994 ) * Document multi-tenant queries. * Update docs/sources/operations/multi-tenancy.md Co-authored-by: Aleksey <a.alexey.lazarev@gmail.com> * Update docs/sources/operations/multi-tenancy.md Co-authored-by: JordanRushing <rushing.jordan@gmail.com> * Update docs/sources/operations/multi-tenancy.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/multi-tenancy.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Aleksey <a.alexey.lazarev@gmail.com> Co-authored-by: JordanRushing <rushing.jordan@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Christian Haudum	b2b4fdec71	Update docs for xk6-loki log formats (#6134 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Nicklas	6922190cde	Documentation: Explain --config.expand-env=true double slash with slash substitution (#6139 ) * docs: Promtail configuration explain expand-env slash pitfall * docs: link to correct envsubst go package Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	3 years ago
Bryan Boreham	dc185f6ac2	Fix wording about pattern parser (#6264 ) The last sentence was saying the opposite of what it meant. Also fix 'start' to match singluar 'expression'.	3 years ago
Travis Patterson	9b2786bde3	Fix delete updates (#6194 ) * Fix delete updates * Update pkg/storage/stores/shipper/compactor/deletion/delete_requests_client.go Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> * Add compactor address to the storage config * review comments * Review feedback Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Steve Hipwell	7cc6aa4021	feat: Add support for Azure user assigned identity (#5891 ) Signed-off-by: Steve Hipwell <steve.hipwell@gmail.com>	3 years ago
Sergey	51dabcf2ee	docs: Default loki service port. (#6212 ) Loki service port is set to 80 by default and not 3000.	3 years ago
Aditya C S	410c1a0d38	feat(ruler): support alert relabeling (#6220 )	3 years ago
Karen Miller	0b947d956f	Docs: Reimplement PR 5649 on troubleshooting timeout errors (#6183 )	3 years ago
Richard Sartor	eedfcd5170	Fix the signature for substr (#6236 )	3 years ago
Trevor Whitney	65c6e254bd	Add upgrade guide regarding azure container name change (#6118 ) * Add upgarde guide regarding azure container name change * Add loki header to master upgrade section	3 years ago
Callum Styan	b6ac00e1a3	[Promtail] drop lines with malformed json (#6099 ) * Refactor json pipeline stage to have a Run function Signed-off-by: Callum Styan <callumstyan@gmail.com> * Allow for dropping log lines in json pipeline stage if they are not valid json strings Signed-off-by: Callum Styan <callumstyan@gmail.com> * fix linting Signed-off-by: Callum Styan <callumstyan@gmail.com> * Clean up new malformed json test. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address some review feedback. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Refactor json pipeline stage test to use testify assert instead of t.Errorf Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update changelog and docs with new drop malformed option Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update CHANGELOG.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs based on Karen's comments. Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Michael Sverdlik	5206e3acd0	Docs: Hint about potential high cardinality metrics (#5603 ) * Hint about potential high cardinality metrics Add warning about potential high cardinality metrics due to the file path being included in some of the metrics. Depending on the setup, such config can result in a very large amount of label values, which is not recommended by Prometheus. This can result in increased storage, slow down of queries, extra costs and so on. Address what has been discussed in #5553. * Reword observability.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Gábor Farkas	0075085e9a	docs: describe ordering in http query results (#6024 ) * docs: describe ordering in http query results * improved text Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * better text Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * better text Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Joan López de la Franca Beltran	5e465ad74e	Docs: Fix design docs index page (#6008 ) * Fix design docs index * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Christian Haudum	d5c337c4d3	docs: Fix broken link from unpack parser to pack stage (#6207 ) and make hyperlinks more obvious Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Kaviraj Kanagaraj	a58dc3f6a8	doc(storage-example-configs): AWS single store configs (#6196 ) * doc(storage-example-configs): AWS single store configs Our example configs are bit misleading here. Particularly AWS deployment. https://grafana.com/docs/loki/latest/storage/#examples As a new user, I come here and all I can conclude is, we cannot use s3 as a single store under AWS deployment. Doc seems to recommend AWS deployment always needs S3 + DynamoDB. (in fact it was real confusion for new user from the community) https://grafana.slack.com/archives/CEPJRLQNL/p1652867888720979 Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Update schema_config.configs.store Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Dylan Guedes	440fb7d74c	Loki: Modifies TableManager to use IndexGateway ring (#5972 ) * Add two metrics to the IndexGateway. - Add a new `query_readiness_duration_seconds` metric, that reports query readiness duration of a tablemanager/index gateway instance. We should use it later to report performance against the ring mode - Add a new `usersToBeQueryReadyForTotal` metric, that reports number of users involved in the query readiness operation. We should use it later to correlate number of users with the query readiness duration. * Remove `usersToBeQueryReadyForTotal`. - It will report all users always for now, so it isn't too helpful the way it is. * Rename metric help text to not mislead people. * Log queryReadiness duration. * Fix where log message and duration and triggered. * Add two metrics to the IndexGateway. - Add a new `query_readiness_duration_seconds` metric, that reports query readiness duration of a tablemanager/index gateway instance. We should use it later to report performance against the ring mode - Add a new `usersToBeQueryReadyForTotal` metric, that reports number of users involved in the query readiness operation. We should use it later to correlate number of users with the query readiness duration. * Remove `usersToBeQueryReadyForTotal`. - It will report all users always for now, so it isn't too helpful the way it is. * Rename metric help text to not mislead people. * Log queryReadiness duration. * Fix where log message and duration and triggered. * Use boundaries to skip users in TableManager. - Separate the assigning of indexClient in the IndexGateway to allow initializing the shipper with the IndexGateway ring - Add new TenantBoundariesClient entity, that answers if a given tenantID should be ignored or not - Use the TenantBoundariesClient implemented by the IndexGateway in the downloads TableManager * Add IndexGateway configuration docs. * Add tests for pkg/util/ring TokenFor() and IsAssignedKey() Signed-off-by: JordanRushing <rushing.jordan@gmail.com> * Small whitespace fix in pkg/util/ring_test.go Signed-off-by: JordanRushing <rushing.jordan@gmail.com> * Apply suggestions from code review Rewrite QueryIndex error phrase. Co-authored-by: JordanRushing <rushing.jordan@gmail.com> * Join users list in a single string. - This is necessary since go-kit doesn't support array type. * Tweak queryReadiness log messages. - As suggested by Ed on https://github.com/grafana/loki/pull/5972#discussion_r859734129 and https://github.com/grafana/loki/pull/5972#discussion_r859736072 * Implement new RingManager. - Adds a new entity to indexgateway named RingManager, responsible for managing the ring and the lifecycler used by the indexgateway. The ringManager is guaranteed to be initiatiated before the Shipper and before the IndexGateway. - Remove the readiness logic from the IndexGateway. - Pass the RingManager as the TenantBoundaries implementation of the Shipper * Return non-empty ringmanager for all modes. * Fix lint regarding error not being check. * Fix lint due to wrong import order. * Implement support for client and server mode for the ring manager. * Fix ring manager services registration. * Add option to configure whether or not to log gateway requests. * Check if IndexGateway is enabled instead of active. * Tune RingManager behavior. - Instantiate ring buffers inside IsAssignedKey instead of reusing same buffer to avoid issues with concurrency. - Remove unnecessary details from IndexGateway mode docs. - Return error when wrongly instantiating a RingManager when IndexGateway is in simple mode. * Rewrite `TenantInBoundaries` as a func instead of interface. * Fix lint. - Fix YAML tag - Remove else clause * Use distributor instead of querier in test. - Since the querier needs to wait for the index gateway ring, it isn't suitable for this test anymore. * Add ring mode guard clause. * Return code erased by mistake. * Rename TenantInBoundaries to IndexGatewayOwnsTenant. * Log len of users instead of all of them to avoid log msgs. * Add docstrings to new public functions. * Apply suggestions from code review Document that tenant filtering is only applied during query readiness. Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Modify IsAssignedKey to expect address directly instead of lifecycler. - Also removes HasDedicatedAddress, since it isn't necessary anymore * Log error message when IsInReplicationSet fails. * Modify IsAssignedKey to return true by default. * Remove wrong assigning of Service for clientMode. * Log gateway requests before error checks. * Remove unnecessary endpoint registration. * Fix lint. * Update pkg/util/ring.go Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: JordanRushing <rushing.jordan@gmail.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	3 years ago
Karen Miller	b90c460769	Rename fudge_duplicate_timestamp to be increment_duplicate_timestamp (#6120 ) * Rename fudge_duplicate_timestamp to be increment_duplicate_timestamp * run `gofmt -d -w pkg/validation/limits.go` Co-authored-by: Christian Simon <simon@swine.de>	3 years ago
Michel Hollands	e5eff96266	Change deletion_enabled setting to deletion_mode (#6190 ) * Change deletion_enabled setting to deletion_mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Empty commit	3 years ago
Daniel Kovacs	e9d2a30733	Fixed reader to support both authentication and tenant-id at the same time. (#5719 )	3 years ago
Kaviraj Kanagaraj	e3a43c6972	query-scheduler: Now proper support for `querier_forget_delay` (#6174 ) * query-scheduler: Now proper support for `querier_forget_delay` Since now shuffle sharding is supported on the `query-path` in Loki, it makes sense to enable this config. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Update pkg/scheduler/scheduler.go Co-authored-by: Christian Simon <simon@swine.de> * Update docs/sources/configuration/_index.md Co-authored-by: Christian Simon <simon@swine.de> Co-authored-by: Christian Simon <simon@swine.de>	3 years ago
Periklis Tsirakidis	6e9517e4b5	ruler: Add support for alertmanager header authorization (#6136 )	3 years ago
Sergio Rua	069156380f	Update SSL config for cassandra (#6145 ) Documentation was missing the references to the SSL key and cert	3 years ago
Aditya C S	c6af640798	feat: support simple selection of labels in json expr parser (#6151 )	3 years ago
Justin Burnham	97c7d7ca3c	LogQL: add the `default` sprig template function to logql label/line formatter (#6163 ) * add the `default` sprig template function to logql label/line formatter With the `default` sprig template function, we will be able to make it easier to output a default value if the source string is otherwise empty. Useful for json fields that can be missing, like HTTP headers in a log line that aren't required as in the following: ``` {job="access_log"} \| json \| line_format `{{.http_request_headers_x_forwarded_for \| default "-"}}` ``` Previously this could only be done (that I found) with the much more verbose ``` {{regexReplaceAllLiteral "^$" .http_request_headers_x_forwarded_for "-"}} ``` * update changelog entry * add pr number * enhance docs * Update docs/sources/logql/template_functions.md Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> * Update docs/sources/logql/template_functions.md Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	3 years ago
Michel Hollands	f39552c7fa	Add docs for updated deletion api (#5763 ) * Add docs for updated deletion api Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix parameter name Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Clarify that only the delete requests for the current tenant are shown Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update docs/sources/api/_index.md Co-authored-by: Christian Simon <simon@swine.de> * Update docs/sources/operations/storage/logs-deletion.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Update docs/sources/operations/storage/logs-deletion.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Update docs/sources/operations/storage/logs-deletion.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Specify all delete endpoints use authentication Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add description of delete modes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Christian Simon <simon@swine.de> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
psytester	40b2373398	corrected copy&paste error in list of labels (#6146 ) response_size was twice mentioned	3 years ago
Karen Miller	31ca108a3c	Docs: redo PR 5109 (#6119 )	3 years ago
Danny Kopping	8838530976	Document `/services` API endpoint (#6111 ) * Documented /services API endpoint Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Grammar Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago

... 4 5 6 7 8 ...

830 Commits (b2ea1214fcce6934c02d4301773777a6ff746f6d)