1274 Commits (a3fcd3cc759ef8f29be1d6abe8d6e2c8fecc334c)

Author	SHA1	Message	Date
RangerCD	87cbe50b59	Lock consistencyMtx outside locking chunkMtx (#5210) ... * Lock consistencyMtx outside locking chunkMtx * Remove unnecessary RLock	4 years ago
RangerCD	4ee686c22c	Remove ErrStreamMissing (#5234)	4 years ago
Callum Styan	bfef7bad44	Start refactor of storage (chunks) clients metrics into structs (#5057) ... * Refactor azure metrics to be a struct (+ add an overall client metrics struct), add an azure client metric for total egress bytes. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add changelog entry for new azure metric. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fixes based on Kavi's review. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix one more merge conflict that I missed in the merge commit. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix failing tests. Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Kaviraj Kanagaraj	7713ff78e2	Fix `cortexpb` -> `logproto` rename in some tests (#5231) ... * Fix `cortexpg` -> `logproto` rename in some tests Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix TimestampMs rename Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * queryrange test Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix tests on queryrangebase pacakage\ Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix some typos Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Callum Styan	f24b84c15a	[querier] Abort a query in limitedRoundTripper if parallelism is 0 (#5190) ... * Abort a query in limitedRoundTripper if parallelism is 0. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix failing tests related to fakeLimits change. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Change limitedRoundTripper parallelism error message. Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Dylan Guedes	3f496e7336	Copy `cortex/pkg/cortexpb` package dependency into Loki (#5151) ... * Fork cortexpb into Cortex `pkg/logproto`. * Use `pkg/logproto` instead of Cortex `pkg/cortexpb`. * Rename `cortexpb` usage to `logproto`. * Fix encoding/decoding test. * Replace cortexpb usage in the ruler/base package. * Remove exemplars from logproto.	4 years ago
Dylan Guedes	3f78d876ff	Copy `cortex/pkg/prom1` package dependency into Loki (#5196) ... * Fork cortex `prom1`. * Use forked `prom1` instead of Cortex one.	4 years ago
Susana Ferreira	50ca4d5186	Addition of cortex' queryrange tests (#5183)	4 years ago
Trevor Whitney	99e046c42f	Default max_outstanding_per_tenant to 2048 (#5204) ... * Default max_outstanding_per_tenant to 2048 * Update changelog	4 years ago
李国忠	d88d20e44f	[instrument] cache: add tracing instrumentation when error !=nil (#5202)	4 years ago
Sandeep Sukhani	0426870f3f	fix broken boltdb-shipper test (#5222)	4 years ago
Sandeep Sukhani	cd1537d06e	step align start and end time of the original query while splitting it (#5217) ... * step align start and end time of the original query while splitting it * also step align queries with step > split interval	4 years ago
Sandeep Sukhani	d87f2a1c8a	per user index changes on the write path (#5193) ... * per user index changes on the write path - add a flag to enable building per user index from the ingesters - when enabled, ingesters would build boltdb files with buckets per tenant	4 years ago
Dylan Guedes	61ff29a574	Copy `cortex/pkg/chunk/purger` package dependency into Loki (#5169) ... * Fork cortex `pkg/chunk/purger`. * Replace `chunk/purger` usage. - Move away from Cortex implementation to instead use the new forked Loki implementation. * Remove unused code from `storage/chunk/purger`. * Remove unused vendored dependencies. * Remove `chunk/purger` implementation. - Loki already has the `compactor/deletion`, so the `chunk/purger` isn't really used.	4 years ago
RangerCD	b26b73d5be	Remove instance level streams lock (#5179) ... * Feat(ingester): remove instance level streams lock * Fix lint * Solve Load and Delete consistency problem while pushing to ingester	4 years ago
Arve Knudsen	a2dd731586	Chore: Add pkg/util/log package (#5187) ... * Chore: Add pkg/util/log package Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix documentation of loki_log_messages_total Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
李国忠	7fca88debd	[enhancement] querier : Add "query_memory_only" to make loki have option to rely only on memory availability like google monarch. (#5093) ... * [enhancement] querier : Add "query_memory_only" to make loki have option to rely only on memory availability like google monarch. * Update pkg/querier/querier.go Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * [enhancement] querier : Add "query_memory_only" to make loki have option to rely only on memory availability like google monarch. #5093 Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	4 years ago
Sandeep Sukhani	88b5a767bc	changes on read path for supporting per user index with boltdb-shipper (#5073)	4 years ago
Sandeep Sukhani	49ffe5291d	add objects list caching for boltdb-shipper index store to reduce object storage list api calls (#5160)	4 years ago
Trevor Whitney	9ff24ac736	Remove temporary ruler ring key fix (#5189) ... Ring keys have been moved out of dskit into their respective projects. Edit commit drone hack	4 years ago
Owen Diehl	1b5b8a3109	log headblock conversion error during replay cleanup (#5185)	4 years ago
Kaviraj	5b5d469038	Support `X-Query-Tags` in LogCLI (#5145) ... * Add support for `--query-tags` LogCLI Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Split it into helper func Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Cleanup the tests Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix linter Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix `getHTTPRequestHeader()` in test assert Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Sandeep Sukhani	dad7fca2fd	align metric queries by step and other queries by split interval (#5181)	4 years ago
Cyril Tovena	1a7614ff70	Avoid splitting large range vector aggregation. (#5172) ... * Reduce split by for large range vector Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/querier/queryrange/split_by_interval.go Co-authored-by: Christian Haudum <christian.haudum@gmail.com> * improve fn name. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
李国忠	6c85cda61a	Handle `context` cancellation in some of the `querier` downstream requests (#5178)	4 years ago
Kaviraj	5dc888710a	Support "POST" method for distributor ring (#5171) ... Able to "forget" the unhealthy member Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Kaviraj	dddace862d	Add nil check for Ruler BasicAuth config. (#5168) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
李国忠	2c20ce6857	[optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" (#4891) ... * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * Update pkg/storage/chunk/cache/instrumented.go Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz> * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * Update pkg/querier/queryrange/roundtrip.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update roundtrip.go Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Danny Kopping	0368a09d4c	Ruler: Fix remote write basic auth credentials (#5144) ... * Retrieving original password to avoid marshaling issues Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Fixing test Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding CHANGELOG entry Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Dylan Guedes	4b1269afab	Remove unused `storegateway` base (#5141)	4 years ago
Piggy	e5dcadee8a	Fix test in retention/expiration_test.go on slow machines (#5137) ... Signed-off-by: Piggy NL <piggynl@outlook.com>	4 years ago
Owen Diehl	8c348614e3	Querier worker/inflight metrics (#5124) ... * refactors querier worker metrics to use metrics struct * metrics and logs for querier worker execution * loki_querier_worker_concurrency metric * alter log line * wires up prometheus.registerer args to prevent duplicate registration errors on tests	4 years ago
Dylan Guedes	fb457e5fc6	Copy `cortex/pkg/querier` package dependency into Loki (#5122) ... * Fork cortex querier and its dependencies. * Use forked querier instead of cortex implementation. * Fix files wrongly using cortex spanlogger. * Fix file wrongly using cortex gateway. * Remove storegateway from vendor. * Fix test. * Use seriesFunc const to fix lint. * Fix linting issues.	4 years ago
Kaviraj	37d0c6c39f	Fix cancel issue between Query Frontend and Query Schdeduler (#5113) ... * Fix cancel issue between Query Frontend -> Query Schdeduler Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * remove assert check for message type Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Handle the case where Frontend is running without Frontend workers Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make it into `sendRequestCancel()` func Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove debug logs Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix note Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove workers count debug log on test Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add bug link to test Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fixes the case where we cancel after enqueuing Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/lokifrontend/frontend/v2/frontend.go Co-authored-by: Oleg Zaytsev <mail@olegzaytsev.com> * Switch to a buffered channel instead Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add error on unknown response status from scheduler Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Oleg Zaytsev <mail@olegzaytsev.com>	4 years ago
Sashank Agarwal	7ff1b45260	remove split_queries_by_interval validation (#5125)	4 years ago
Dylan Guedes	7d501729c3	Add missing forked gateway tests (#5118) ... * Add missing tests for `storegateway`. * Fork cortex tsdb testutil package. * Fix import lint.	4 years ago
Dylan Guedes	634baa8aa8	Copy `cortex/pkg/querier/stats` package dependency into Loki (#5047)	4 years ago
Dylan Guedes	0e3df1938f	Copy querier subpackages dependency into Loki (#5068) ... * Fork cortex `querier/series`. * Fork cortex `querier/chunkstore`. * Fork cortex `querier/lazyquery`. * Use Loki forked cortex code. * Copy cortex `querier/queryrange` as `querier/queryrange/queryrangebase`. * Fork cortex `pkg/querier/queryrange` to `pkg/querier/queryrange/queryrangebase`. * Tell protoc to use gogo.types instead of pb.Any. * Use `pkg/querier/queryrange/queryrangebase`. * Remove cortex `queryrange` vendoring. * Fix lint. * Use `_ = ...` syntax instead of comment to fix lint.	4 years ago
Dylan Guedes	3d7e9cee36	Copy `cortex/pkg/ruler` package dependency into Loki (#5089) ... * Fork cortex `ruler/rulestore/local`. * Fork cortex ruler into Loki as package `ruler/base`. * Use forked cortex ruler instead of vendored. * Update `vendor/modules.txt` and Makefile instruction. * Merge `pkg/ruler/base/rulestore` with `pkg/ruler/rulestore`. * Move `pkg/ruler/base/rulespb` to `pkg/ruler/rulespb`. * Rename `cortex_ruler` usage to `base_ruler`. * Fix lint issues. * Add forked ruler tests. * Use own `pkg/storage/bucket` instead of Cortex one. * Swap to `pkg/storage/chunk/{aws,gcp,etc}`. * Update with correct prometheus import. * Change error comparison. * Code style changes	4 years ago
Cyril Tovena	fc49a87d99	Fixes HTTP2 removal for GCS (#5103) ... * Fixes metric name for hedging object store requests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Only allow to disable http2 for GETs Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes http2 removal Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Address review feedback Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Address review feedback Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Trevor Whitney	3091ccd3eb	improve default config values (#5077) ... * improve default config values * change defaults for upstream query range package * update changelog * remove max_tranfer_retries fix as it is covered in another PR, 4792 * increase query ingesters within by 1h This provides an additional buffer on top of the max_chunk_age. Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * add comment reminder to remove query range config hack Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * rollback change to align_queries_with_step and parallelise_shardable_queries * add upgrading docs * re-enable parallelise_shardable_queries by default * add parallelise_shardable_queries back to upgrading doc Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Owen Diehl	e823a5037f	better defaults for flush queue parallelism (#5091) ... * better defaults for flush queue length * changelog	4 years ago
Cyril Tovena	181f4b698d	Fixes metric name for hedging object store requests (#5098) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
李国忠	29cccbe576	[enhancement] querier cache: WriteBackCache should be off query path (#5083) ... * [enhancement] querier cache: WriteBackCache should be off query path #5072 * [enhancement] querier cache: WriteBackCache should be off query path #5083 * Update pkg/storage/chunk/chunk_store_utils.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * [enhancement] querier cache: WriteBackCache should be off query path #5083 * [enhancement] querier cache: WriteBackCache should be off query path #5083 * Update pkg/storage/chunk/cache/cache.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * [enhancement] querier cache: WriteBackCache should be off query path #5083 * [enhancement] querier cache: WriteBackCache should be off query path #5083 * Update pkg/storage/chunk/chunk_store_utils.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * [enhancement] querier cache: WriteBackCache should be off query path #5083 Co-authored-by: Danny Kopping <dannykopping@gmail.com>	4 years ago
Kaviraj	c5aa9f197d	Handle `context` cancellation in some of the `querier` downstream requests (#5080) ... * Fix deadlock in disconnecting querier Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * todo Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add more logs to queriers cancellation Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * new span Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixe log Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Exit earlier for batch iterator Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add missing return Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add store context cancellation Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a possible cancellation issue Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Rmove code to find issues Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add splitMiddleware back to handler Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove query split Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove split middleware and context cancel check for chunks * Handle context cancel properly on `getChunk()` via select. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Use context in getChunk without starting new goroutine Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Just normal ctx.Err() check instead of using Done channel Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove debug logs Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Update pkg/querier/http.go Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove unused imports Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	21e61315df	LogQL: add __line__ function to | line_format template (#4879) ... * LogQL: add __line__ function to | line_format template Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * add changelog entry Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs/sources/logql/template_functions.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Clone the function map to avoid a race between multiple expr Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Sas Swart	79c337e234	Add the option to configure memory ballast for Loki (#5081) ... * Add memory ballast * Add changelog entry * Add documentation Add additional information to Flag description * Ensure that ballast_bytes can be configured from config file Add documentation * Add Operational scalability documentation related to memory ballast * Restore config.ballast-bytes cli flag * Move config.ballast-bytes cli flag declaration Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Dylan Guedes	0cdb66013b	Copy `cortex/pkg/storegateway` package dependency into Loki (#5088) ... * Fork Cortex `pkg/storegateway` into Loki. * Update forked `pkg/storegateway` imports. Modify it to use Loki's `spanlogger` instead of Cortex implementation. * Include `thanos-io` in the list of protos to include. * Remove `bucket_store_inmemory_server.go` as its content isn't used. * Update `go.mod` with new explicit dependency.	4 years ago
Cyril Tovena	d6727398d0	Removes split-by upstream validation (#5087) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Arve Knudsen	d87dbf9f8a	Chore: Upgrade to latest Cortex (#5085) ... * Chore: Upgrade to latest Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Chore: Upgrade to Prometheus@692a54649ed7 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build after upgrading dskit Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago