3518 Commits (0470e4e7d38e7b364acfd230b20f4237c8ef05d0)

 

All Branches   

Search

Author	SHA1	Message	Date
Sandeep Sukhani	ef4bc8f1ed	copy all the fixes/improvements that went into boltdb-shipper that are missing from generic index-shipper (#6200)	3 years ago
Gerard Vanloo	5eb4c0d66a	operator: Changing GCP secret file path (#6199) ... * Changing GCP secret file path * Update changelog	3 years ago
irizzant	bb2eccce3c	[loki microservices]: allow custom gateway server configuration (#6138) ... * feat(loki)!: allow custom server configuration Signed-off-by: irizzant <i.rizzante@gmail.com> * doc(loki): fix documentation * refactor: move server_snippet under gateway.libsonnet Signed-off-by: irizzant <i.rizzante@gmail.com>	3 years ago
Michel Hollands	e5eff96266	Change deletion_enabled setting to deletion_mode (#6190) ... * Change deletion_enabled setting to deletion_mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Empty commit	3 years ago
Dylan Guedes	fe7ced807b	Fix CHANGELOG entries. (#6193) ... - The compactor address entry wasn't part of v2.5.0, so I moved it to the main section instead - Two PRs were linking to a wrong PR link	3 years ago
Christian Haudum	b073ec9017	security: Redact credentials when marshalled to YAML (#6186)	3 years ago
Christian Haudum	8518560cf2	Fix flaky gRPC server shutdown in tests (#6191) ... * Use `t.Cleanup(cleanup)` instead of `defer cleanup()` Which seems to reduce the likelyhood of the flaky gRPC server shutdown Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Do not fail test if gRPC server graceful shutdown fails Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Sandeep Sukhani	62f5e598dd	tsdb index gateway (#6158) ... * enable new grpc calls on index gateway and add GetSeries call * use index gateway for tsdb index store queries * add missing file * lint * disable IndexGatewayClientStore for stores other than tsdb until we are ready to enable it again	3 years ago
Owen Diehl	4f87db7f38	strips out tenant label from LabelNames queries (#6181) ... * strips out tenant label from LabelNames queries * prevent tenant labelvalues query	3 years ago
Michel Hollands	b6ff9d5889	Add integration test for delete request (#6188) ... * Add integration test for delete request Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add line filter Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Daniel Kovacs	e9d2a30733	Fixed reader to support both authentication and tenant-id at the same time. (#5719)	3 years ago
Alan Conway	3547d8948c	Fix typo in operator/docs/hack_operator_make_run.md (#6180)	3 years ago
Owen Diehl	6b97de765e	tsdb creation supports fingerprint overrides (#6178)	3 years ago
Sandeep Sukhani	ead8c345e6	skip empty object key returned by some s3 compatible storage while listing boltdb-shipper index (#6123)	3 years ago
Kaviraj Kanagaraj	e5a89fa6da	fix(querier): Record `result` length after response validation. (#6176) ... Without this fix, it causes panic, if series or label endpoint returns error. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Kaviraj Kanagaraj	e3a43c6972	query-scheduler: Now proper support for `querier_forget_delay` (#6174) ... * query-scheduler: Now proper support for `querier_forget_delay` Since now shuffle sharding is supported on the `query-path` in Loki, it makes sense to enable this config. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Update pkg/scheduler/scheduler.go Co-authored-by: Christian Simon <simon@swine.de> * Update docs/sources/configuration/_index.md Co-authored-by: Christian Simon <simon@swine.de> Co-authored-by: Christian Simon <simon@swine.de>	3 years ago
Periklis Tsirakidis	6e9517e4b5	ruler: Add support for alertmanager header authorization (#6136)	3 years ago
irizzant	6fb86c4def	fix(tanka): add customizable common env vars (#6124) ... Signed-off-by: Ivan Rizzante <i.rizzante@gmail.com>	3 years ago
Sergio Rua	069156380f	Update SSL config for cassandra (#6145) ... Documentation was missing the references to the SSL key and cert	3 years ago
Aditya C S	c6af640798	feat: support simple selection of labels in json expr parser (#6151)	3 years ago
dependabot[bot]	c6b7752bbb	Bump golangci/golangci-lint-action from 3.1.0 to 3.2.0 (#6170) ... Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.1.0 to 3.2.0. - [Release notes](https://github.com/golangci/golangci-lint-action/releases) - [Commits](https://github.com/golangci/golangci-lint-action/compare/v3.1.0...v3.2.0) --- updated-dependencies: - dependency-name: golangci/golangci-lint-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
Justin Burnham	97c7d7ca3c	LogQL: add the `default` sprig template function to logql label/line formatter (#6163) ... * add the `default` sprig template function to logql label/line formatter With the `default` sprig template function, we will be able to make it easier to output a default value if the source string is otherwise empty. Useful for json fields that can be missing, like HTTP headers in a log line that aren't required as in the following: ``` {job="access_log"} | json | line_format `{{.http_request_headers_x_forwarded_for | default "-"}}` ``` Previously this could only be done (that I found) with the much more verbose ``` {{regexReplaceAllLiteral "^$" .http_request_headers_x_forwarded_for "-"}} ``` * update changelog entry * add pr number * enhance docs * Update docs/sources/logql/template_functions.md Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> * Update docs/sources/logql/template_functions.md Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	3 years ago
Marco Cadetg	aa3f06f36b	update auth header env variable usage help (#6169)	3 years ago
Michel Hollands	f39552c7fa	Add docs for updated deletion api (#5763) ... * Add docs for updated deletion api Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix parameter name Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Clarify that only the delete requests for the current tenant are shown Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update docs/sources/api/_index.md Co-authored-by: Christian Simon <simon@swine.de> * Update docs/sources/operations/storage/logs-deletion.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Update docs/sources/operations/storage/logs-deletion.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Update docs/sources/operations/storage/logs-deletion.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Specify all delete endpoints use authentication Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add description of delete modes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Christian Simon <simon@swine.de> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Marco Cadetg	73b39a97bc	specify logcli authorization header by flag (#6164) ... * specify logcli authorization header by flag #6144 * default AuthHeader to Authorization in client lib * use constant for default auth header * for consistency use LOKI_AUTH_HEADER	3 years ago
psytester	40b2373398	corrected copy&paste error in list of labels (#6146) ... response_size was twice mentioned	3 years ago
Yusuf Papurcu	dc782de864	Removed double space in Log (#6166)	3 years ago
Sandeep Sukhani	6e3bf3f7d0	tsdb query fixes (#6142) ... * query both downloads and uploads manager when non-nil * fix reading of index file with nodename having a `-` * remove __name__ matcher at a layer above to avoid passing it during tsdb head queries	3 years ago
Sashank Agarwal	301ea42314	operator: Add method to get authenticated from GCP (#6125)	3 years ago
Ed Welch	4bf1648da1	Loki: When processing logs during tailing, don't reuse the query pipeline (#6152) ... * when tailing, don't reuse the query pipeline because it has an unbounded label cache Signed-off-by: Edward Welch <edward.welch@grafana.com> * update changelog Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Christian Haudum	ee4ab1b394	Fix certain binary expression queries by optimizing through push down (#6132) ... * Add test cases that fail Instant queries of the following type fail and return an `unimplemented` error: ``` sum(count_over_time({foo="bar"} | logfmt | duration > 2s [3s])) / sum(count_over_time({foo="bar"} [3s])) ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Fix certain binary expressions for instant queries If either the left hand side or the right hand side of a binary expression is a noop, we need to return the original expression so the whole expression is a noop as well, and thus not executed using the downstream engine. Otherwise, a binary expression that has a noop on either side, results in an `unimplemented` error when executed using the downstream engine. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Optimize instant vector aggregations with log extraction stage This change optimizes queries that use a vector aggregation without grouping around a range aggregation with a label extraction stage such as `json` or `logfmt`. Since the vector aggregation can be pushed down to the downstream query, the downstream query does not create a massive amount of streams, even though it contains a generic label extraction stage. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Add test cases that fail Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! fixup! Add test cases that fail Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Karen Miller	31ca108a3c	Docs: redo PR 5109 (#6119)	3 years ago
Irina	c2e43dd8f5	Bump k8s.io/api, k8s.io/client-go, k8s.io/apimachinery from 0.22.7 to 0.23.6 (#6037)	3 years ago
Kaviraj Kanagaraj	24f8b19e91	Remove unused grpc health check endpoints (#6113) ... We have generic `grpc_healthcheck` endpoint that get registered for all the components during initialization stage. Looks like these health checks that are part of actual component's struct itself came from legacy. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Owen Diehl	729bf3fc91	loads previously created TSDBs into shipper on startup (#6117) ... * loads previously created TSDBs into shipper on startup * noopTSDBManager impl * info logs for initial tsdb loading	3 years ago
Owen Diehl	756fc19577	fix mutability bug with labels during tsdb flushing (#6116)	3 years ago
Periklis Tsirakidis	41647d1302	operator: Ruler enhancement proposal (#5985)	3 years ago
Owen Diehl	55ea89f9fd	Tsdb/head wal feedback (#6107) ... * pr feedback, improved mutex unlocking * tenant heads use sync.Map * tenant heads bench * Revert "tenant heads use sync.Map" This reverts commit 64adebc535. * removes old code the removed TSDBs outside the shipper + minor fixes	3 years ago
Danny Kopping	8838530976	Document `/services` API endpoint (#6111) ... * Documented /services API endpoint Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Grammar Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Sandeep Sukhani	06e309bae6	do not increment sync operation count metric on per index set (#6109)	3 years ago
JordanRushing	45dd588d8f	Update consistent hash ring docs with new index gateway ring (#6041) ... * Update consistent hash ring docs with new index gateway ring Signed-off-by: JordanRushing <rushing.jordan@gmail.com> * Update docs/sources/fundamentals/architecture/rings.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
arcosx	e2fdc3437b	Feat: add Baidu Cloud BOS as storage backends for Loki #4788 (#5848) ... * feat: add baidu bce bos storage support Signed-off-by: arcosx <arcosx@outlook.com> * add baidu bce bos as chunk storage backend Signed-off-by: arcosx <arcosx@outlook.com> * fix: some doc error && rewrite bad code Signed-off-by: arcosx <arcosx@outlook.com> * fix: add the `BceServiceError` source link Signed-off-by: arcosx <arcosx@outlook.com> * Update CHANGELOG.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Signed-off-by: arcosx <arcosx@outlook.com> * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update pkg/storage/chunk/client/baidubce/bos_storage_client.go Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update pkg/storage/chunk/client/baidubce/bos_storage_client.go Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update pkg/storage/chunk/client/baidubce/bos_storage_client.go Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Christian Haudum	18058d2b55	Serve correct status of ingester/distributor in gRPC healthcheck endpoint (#6094) ... * Serve correct status of ingester in gRPC healthcheck endpoint The gRCP healthcheck endpoint is used by clients that determin whether a connection to the service is healthy or not. In case the ingester is not in "running" state and is not "active" in the ring, this healthcheck should return "not serving". Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Serve correct status of distributor in gRPC healthcheck endpoint	3 years ago
Artem Timchenko	ebb39db864	lambda-promtail: Add multi-tenancy support (#6102) ... * lambda-promtail: adding multi-tenancy support * update changelog and fix import order * fix import order * fix import order	3 years ago
Owen Diehl	b45efd4c2d	TSDB shipper + WAL (#6049) ... * begins speccing out TSDB Head * auto incrementing series ref + mempostings * mintime/maxtime methods * tsdb head IndexReader impl * head correctly populates ref lookup * tsdb head tests * adds prometheus license to tsdb head * linting * [WIP] speccing out tsdb head wal * fix length check and adds tsdb wal encoding tests * exposes wal structs & removes closed semantics * logs start time in the tsdb wal * wal interface + testing * exports walrecord + returns ref when appending * specs out head manager * tsdb head manager wal initialization * tsdb wal rotation * wals dont use node name, but tsdb files do * cleans up fn signature * multi tsdb idx now just wraps Index interfaces * no longer sorts indices when creating multi-idx * tenantHeads & HeadManger index impls * head mgr tests * bugfixes & head manager tests * tsdb dir selection now helper fns * period utility * pulls out more code to helpers, fixes some var races * head recovery is more generic * tsdb manager builds from wals * pulls more helpers out of headmanager * lockedIdx, Close() on idx, tsdbManager update * removes mmap from index reader implementation * tsdb file * adds tsdb shipper config and refactors initStore * removes unused tsdbManager code * implements stores.Index and stores.ChunkWriter for tsdb * chunk.Data now supports an Entries() method * moves walreader to new util/wal pkg to avoid circular dep + tsdb storage alignment * tsdb store * passes indexWriter to chunkWriter * build a tsdb per index bucket in according with shipper conventions * dont open tsdb files until necessary for indexshipper * tsdbManager Index impl * tsdb defaults + initStore fix for invalid looping * fixes UsingTSDB helper * disables deleteRequestStore when using TSDB * pass limits to tsdb store * always start headmanager for tsdb Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * fixes copy bug Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * more logging Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * fixes duplicate tenant label bug Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * debug logs, uses label builder, removes __name__=logs for tsdb Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * tsdb fixes labels at earlier pt Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * account for setting tenant label in head manager test * changing tsdb dir names * identifier interface, builder to tsdb pkg * tsdb version path prefix * fixes buildfromwals identifier * fixes tsdb shipper paths * split buckets once per user set * refactors combining single and multi tenant tsdb indices on shipper reads * indexshipper ignores old gzip logic * method name refactor * remove unused record type * removes v1 prefix in tsdb paths and refactores indices method * ignores double optimization in tsdb looking for multitenant idx, shipper handles this * removes 5-ln requirement on shipper tablename regexp * groups identifiers, begins removing multitenant prefix in shipped files * passses open fn to indexshipper * exposes RealByteSlice * TSDBFile no longer needs a file descriptor, parses gzip extensions * method signature fixing * stop masquerading as compressed indices post-download in indexshipper * variable bucket regexp * removes accidental configs committed * label matcher handling for multitenancy and metricname in tsdb * explicitly require fingerprint when creating tsdb index * only add tenant label when creating multitenant tsdb write fingerprints without synthetic tenant label strip out tenant labels from queries * linting + unused removal * more linting :( * goimports * removes uploadername from indexshipper * maxuint32 for arm32 builds * tsdb chunk filterer support * always set ingester name when using object storage index Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	3 years ago
Travis Patterson	03153e89bf	clean up the pr diff message (#6098) ... * clean up the pr diff message * add dependency to clone-main to avoid possible race * don't start tests before everything is cloned * formatting change	3 years ago
Ed Welch	4f75939efb	Revert "Loki: Change live tailing to only allow mutating the log line not the number of streams. (#6063)" (#6097) ... This reverts commit 98fda9b0a0.	3 years ago
n4mine	14b0873f2b	Revert "Docs: fix pic location in `sources/fundamentals/architecture/deployment-modes.md` (#6089)" (#6093) ... This reverts commit 1d25e17b32.	3 years ago
Gábor Farkas	0a68bd2d4e	docs: update the labels timespan info (#6019) ... * docs: update the labels timespan info * updated docs Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * updated docs Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Travis Patterson	d97724e8dd	Introduce coverage to PR pipelines (#5357) ... * Add code coverage diff to CI * remove test package from coverage diff * review feedback * lint * Env variable test * faster experiment * drone vars * drone vars * how does the shell work? * add pr comment * escape json * report diff in comment * properly format json * quote post body * review feedback * add querier/queryrange to coverage	3 years ago