3504 Commits (6922190cdec135eb2c52417335d51ccbe6d615b1)

 

All Branches   

Search

Author	SHA1	Message	Date
Nicklas	6922190cde	Documentation: Explain --config.expand-env=true double slash with slash substitution (#6139) ... * docs: Promtail configuration explain expand-env slash pitfall * docs: link to correct envsubst go package Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	3 years ago
Bryan Boreham	dc185f6ac2	Fix wording about pattern parser (#6264) ... The last sentence was saying the opposite of what it meant. Also fix 'start' to match singluar 'expression'.	3 years ago
Anatoly Laskaris	9c49a0222d	docs: Add Nomad deployment examples (#6258) ... * Add Loki monolith example * Commentout traefik in monolithic setup * Typos * Add ruler configuration * Add loki-simple * Fix consul prefix * Add loki-distributed example * Add missing README * Add explanation why use Consul Connect and Vault PKI at the same time * Simplify distributed setup * Readme small fixes * Update production/nomad/README.md * Add Nomad mention * Commit suggestions * Add ability to specify Loki version from command line * Wording fix	3 years ago
Robert Jacob	e61f8dccdd	operator: Document how to connect Grafana to gateway (#6250)	3 years ago
Travis Patterson	9b2786bde3	Fix delete updates (#6194) ... * Fix delete updates * Update pkg/storage/stores/shipper/compactor/deletion/delete_requests_client.go Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> * Add compactor address to the storage config * review comments * Review feedback Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Sandeep Sukhani	3b3fcf6c87	update drone branch match for 3 digit k release (#6263)	3 years ago
Steve Hipwell	7cc6aa4021	feat: Add support for Azure user assigned identity (#5891) ... Signed-off-by: Steve Hipwell <steve.hipwell@gmail.com>	3 years ago
Callum Styan	0cb3994859	Memberlist related jsonnet (#6253) ... * Add memberlist.libsonnet and modify loki services so we can set the port when using memberlist for the ring. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Missed the + for the ruler ring config here. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix linting. Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Sergey	51dabcf2ee	docs: Default loki service port. (#6212) ... Loki service port is set to 80 by default and not 3000.	3 years ago
Marco Pracucci	afa53f1939	Loki mixin: publish compiled version of the mixin (#6254) ... * Publish compiled mixin Signed-off-by: Marco Pracucci <marco@pracucci.com> * Update Makefile Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> * Updated build image version Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	3 years ago
Marco Pracucci	58718a548c	Add jsonnet tools to build image (#6255) ... Signed-off-by: Marco Pracucci <marco@pracucci.com>	3 years ago
Kaviraj Kanagaraj	a795b93897	chore(cache): Make exprimental messages more clear for cache. (#6219) ... Currently we see following message when any of the experimental feature is used for cache (say fifocache or redis). But the problem it doesn't say __what_kind__ of cache is it exactly. So this leads to confusion to track what kind of cache is the source of the experiment feature. This PR fixes it by adding more clear message. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Aditya C S	410c1a0d38	feat(ruler): support alert relabeling (#6220)	3 years ago
Karen Miller	0b947d956f	Docs: Reimplement PR 5649 on troubleshooting timeout errors (#6183)	3 years ago
Ed Welch	3e77423f91	Migrate Tool: Update to work with boltdb-shipper, improve logging output. (#6237) ... * update migrate tool Signed-off-by: Edward Welch <edward.welch@grafana.com> * improve logging don't use any caches Signed-off-by: Edward Welch <edward.welch@grafana.com> * comment out some more log lines Signed-off-by: Edward Welch <edward.welch@grafana.com> * change stat types Signed-off-by: Edward Welch <edward.welch@grafana.com> * update docs and comments as well as set some lower times on a few boltdb-shipper timers Signed-off-by: Edward Welch <edward.welch@grafana.com> * remove changes to the Makefile which weren't meant to make it to the PR Signed-off-by: Edward Welch <edward.welch@grafana.com> * remove some commented code Signed-off-by: Edward Welch <edward.welch@grafana.com> * fix tests Signed-off-by: Edward Welch <edward.welch@grafana.com> * lint Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Richard Sartor	eedfcd5170	Fix the signature for substr (#6236)	3 years ago
Trevor Whitney	65c6e254bd	Add upgrade guide regarding azure container name change (#6118) ... * Add upgarde guide regarding azure container name change * Add loki header to master upgrade section	3 years ago
Sandeep Sukhani	6ac955cc7d	make boltdb-shipper use the generic indexshipper for managing index in object storage (#6226)	3 years ago
Ludovic Cleroux	67f104a783	Add NRC in Adopters (#6242)	3 years ago
Callum Styan	b6ac00e1a3	[Promtail] drop lines with malformed json (#6099) ... * Refactor json pipeline stage to have a Run function Signed-off-by: Callum Styan <callumstyan@gmail.com> * Allow for dropping log lines in json pipeline stage if they are not valid json strings Signed-off-by: Callum Styan <callumstyan@gmail.com> * fix linting Signed-off-by: Callum Styan <callumstyan@gmail.com> * Clean up new malformed json test. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address some review feedback. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Refactor json pipeline stage test to use testify assert instead of t.Errorf Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update changelog and docs with new drop malformed option Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update CHANGELOG.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs based on Karen's comments. Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Callum Styan	3d2282745b	Runtime reloadable config; ring migration setup (#6214) ... * Clean up setting of memberlist and multikv config for Loki services that use the ring. Also sets the multi client runtime config function for all services that use the ring. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add a test for the multi kv setup Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix lint issues. Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Owen Diehl	c032da9639	Tsdb/inverted index (#6233) ... * begins adding a factor-of-two dynamically sharded inverted index * bit prefix inverted index tests * goimports	3 years ago
Travis Patterson	e313d8a873	Remove __name__ from label calls (#6229)	3 years ago
Cyril Tovena	67e7285bf5	Improve labels computation during LogQL pipeline (#6110) ... * Improve labels computation during LogQL pipeline * Fixes a bug that was mutating the base labels	3 years ago
Susana Ferreira	0dfd2cbbcc	Fix label extraction pushdown on instant queries (#6215) ... * Add downstream label extraction tests * Fix label extraction downstream pushdown	3 years ago
Periklis Tsirakidis	fd627f2a49	operator: Add rules support (#5986)	3 years ago
Michael Sverdlik	5206e3acd0	Docs: Hint about potential high cardinality metrics (#5603) ... * Hint about potential high cardinality metrics Add warning about potential high cardinality metrics due to the file path being included in some of the metrics. Depending on the setup, such config can result in a very large amount of label values, which is not recommended by Prometheus. This can result in increased storage, slow down of queries, extra costs and so on. Address what has been discussed in #5553. * Reword observability.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Gábor Farkas	0075085e9a	docs: describe ordering in http query results (#6024) ... * docs: describe ordering in http query results * improved text Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * better text Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * better text Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Joan López de la Franca Beltran	5e465ad74e	Docs: Fix design docs index page (#6008) ... * Fix design docs index * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Sandeep Sukhani	1923668a5d	run store queries in ingester when using tsdb as index store (#6209)	3 years ago
Owen Diehl	f14f9dd8c4	prevents sharding avg and avg_over_time when their children arent shardable. (#6204)	3 years ago
Christian Haudum	d5c337c4d3	docs: Fix broken link from unpack parser to pack stage (#6207) ... and make hyperlinks more obvious Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Kaviraj Kanagaraj	a58dc3f6a8	doc(storage-example-configs): AWS single store configs (#6196) ... * doc(storage-example-configs): AWS single store configs Our example configs are bit misleading here. Particularly AWS deployment. https://grafana.com/docs/loki/latest/storage/#examples As a new user, I come here and all I can conclude is, we cannot use s3 as a single store under AWS deployment. Doc seems to recommend AWS deployment always needs S3 + DynamoDB. (in fact it was real confusion for new user from the community) https://grafana.slack.com/archives/CEPJRLQNL/p1652867888720979 Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Update schema_config.configs.store Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Dylan Guedes	440fb7d74c	Loki: Modifies TableManager to use IndexGateway ring (#5972) ... * Add two metrics to the IndexGateway. - Add a new `query_readiness_duration_seconds` metric, that reports query readiness duration of a tablemanager/index gateway instance. We should use it later to report performance against the ring mode - Add a new `usersToBeQueryReadyForTotal` metric, that reports number of users involved in the query readiness operation. We should use it later to correlate number of users with the query readiness duration. * Remove `usersToBeQueryReadyForTotal`. - It will report all users always for now, so it isn't too helpful the way it is. * Rename metric help text to not mislead people. * Log queryReadiness duration. * Fix where log message and duration and triggered. * Add two metrics to the IndexGateway. - Add a new `query_readiness_duration_seconds` metric, that reports query readiness duration of a tablemanager/index gateway instance. We should use it later to report performance against the ring mode - Add a new `usersToBeQueryReadyForTotal` metric, that reports number of users involved in the query readiness operation. We should use it later to correlate number of users with the query readiness duration. * Remove `usersToBeQueryReadyForTotal`. - It will report all users always for now, so it isn't too helpful the way it is. * Rename metric help text to not mislead people. * Log queryReadiness duration. * Fix where log message and duration and triggered. * Use boundaries to skip users in TableManager. - Separate the assigning of indexClient in the IndexGateway to allow initializing the shipper with the IndexGateway ring - Add new TenantBoundariesClient entity, that answers if a given tenantID should be ignored or not - Use the TenantBoundariesClient implemented by the IndexGateway in the downloads TableManager * Add IndexGateway configuration docs. * Add tests for pkg/util/ring TokenFor() and IsAssignedKey() Signed-off-by: JordanRushing <rushing.jordan@gmail.com> * Small whitespace fix in pkg/util/ring_test.go Signed-off-by: JordanRushing <rushing.jordan@gmail.com> * Apply suggestions from code review Rewrite QueryIndex error phrase. Co-authored-by: JordanRushing <rushing.jordan@gmail.com> * Join users list in a single string. - This is necessary since go-kit doesn't support array type. * Tweak queryReadiness log messages. - As suggested by Ed on https://github.com/grafana/loki/pull/5972#discussion_r859734129 and https://github.com/grafana/loki/pull/5972#discussion_r859736072 * Implement new RingManager. - Adds a new entity to indexgateway named RingManager, responsible for managing the ring and the lifecycler used by the indexgateway. The ringManager is guaranteed to be initiatiated before the Shipper and before the IndexGateway. - Remove the readiness logic from the IndexGateway. - Pass the RingManager as the TenantBoundaries implementation of the Shipper * Return non-empty ringmanager for all modes. * Fix lint regarding error not being check. * Fix lint due to wrong import order. * Implement support for client and server mode for the ring manager. * Fix ring manager services registration. * Add option to configure whether or not to log gateway requests. * Check if IndexGateway is enabled instead of active. * Tune RingManager behavior. - Instantiate ring buffers inside IsAssignedKey instead of reusing same buffer to avoid issues with concurrency. - Remove unnecessary details from IndexGateway mode docs. - Return error when wrongly instantiating a RingManager when IndexGateway is in simple mode. * Rewrite `TenantInBoundaries` as a func instead of interface. * Fix lint. - Fix YAML tag - Remove else clause * Use distributor instead of querier in test. - Since the querier needs to wait for the index gateway ring, it isn't suitable for this test anymore. * Add ring mode guard clause. * Return code erased by mistake. * Rename TenantInBoundaries to IndexGatewayOwnsTenant. * Log len of users instead of all of them to avoid log msgs. * Add docstrings to new public functions. * Apply suggestions from code review Document that tenant filtering is only applied during query readiness. Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Modify IsAssignedKey to expect address directly instead of lifecycler. - Also removes HasDedicatedAddress, since it isn't necessary anymore * Log error message when IsInReplicationSet fails. * Modify IsAssignedKey to return true by default. * Remove wrong assigning of Service for clientMode. * Log gateway requests before error checks. * Remove unnecessary endpoint registration. * Fix lint. * Update pkg/util/ring.go Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: JordanRushing <rushing.jordan@gmail.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	3 years ago
Karen Miller	b90c460769	Rename fudge_duplicate_timestamp to be increment_duplicate_timestamp (#6120) ... * Rename fudge_duplicate_timestamp to be increment_duplicate_timestamp * run `gofmt -d -w pkg/validation/limits.go` Co-authored-by: Christian Simon <simon@swine.de>	3 years ago
Owen Diehl	3c334001cd	creates a single tsdb for entire wal recovery (#6203) ... * creates a single tsdb for entire wal recovery * tsdb wal replay builds TSDBs with unrounded timestamps to avoid clobbering	3 years ago
Sandeep Sukhani	ef4bc8f1ed	copy all the fixes/improvements that went into boltdb-shipper that are missing from generic index-shipper (#6200)	3 years ago
Gerard Vanloo	5eb4c0d66a	operator: Changing GCP secret file path (#6199) ... * Changing GCP secret file path * Update changelog	3 years ago
irizzant	bb2eccce3c	[loki microservices]: allow custom gateway server configuration (#6138) ... * feat(loki)!: allow custom server configuration Signed-off-by: irizzant <i.rizzante@gmail.com> * doc(loki): fix documentation * refactor: move server_snippet under gateway.libsonnet Signed-off-by: irizzant <i.rizzante@gmail.com>	3 years ago
Michel Hollands	e5eff96266	Change deletion_enabled setting to deletion_mode (#6190) ... * Change deletion_enabled setting to deletion_mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Empty commit	3 years ago
Dylan Guedes	fe7ced807b	Fix CHANGELOG entries. (#6193) ... - The compactor address entry wasn't part of v2.5.0, so I moved it to the main section instead - Two PRs were linking to a wrong PR link	3 years ago
Christian Haudum	b073ec9017	security: Redact credentials when marshalled to YAML (#6186)	3 years ago
Christian Haudum	8518560cf2	Fix flaky gRPC server shutdown in tests (#6191) ... * Use `t.Cleanup(cleanup)` instead of `defer cleanup()` Which seems to reduce the likelyhood of the flaky gRPC server shutdown Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Do not fail test if gRPC server graceful shutdown fails Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Sandeep Sukhani	62f5e598dd	tsdb index gateway (#6158) ... * enable new grpc calls on index gateway and add GetSeries call * use index gateway for tsdb index store queries * add missing file * lint * disable IndexGatewayClientStore for stores other than tsdb until we are ready to enable it again	3 years ago
Owen Diehl	4f87db7f38	strips out tenant label from LabelNames queries (#6181) ... * strips out tenant label from LabelNames queries * prevent tenant labelvalues query	3 years ago
Michel Hollands	b6ff9d5889	Add integration test for delete request (#6188) ... * Add integration test for delete request Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add line filter Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Daniel Kovacs	e9d2a30733	Fixed reader to support both authentication and tenant-id at the same time. (#5719)	3 years ago
Alan Conway	3547d8948c	Fix typo in operator/docs/hack_operator_make_run.md (#6180)	3 years ago
Owen Diehl	6b97de765e	tsdb creation supports fingerprint overrides (#6178)	3 years ago
Sandeep Sukhani	ead8c345e6	skip empty object key returned by some s3 compatible storage while listing boltdb-shipper index (#6123)	3 years ago