18 Commits (95b885691d4ea1d94aa2f9ea1d426bc90b3b4759)

Author	SHA1	Message	Date
Christian Simon	ea62af58e9	Introduce a ChunkRef type as part of logproto (#5643) ... * Change protobuf definitions * Adapt to an embedded ChunkRef struct * Move the Checksum to the ChunkRef	3 years ago
Cyril Tovena	79f0e349f5	Refactor LogQL syntax into its own package (#5539) ... * Refactor LogQL syntax into its own package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Merge upstream Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/loghttp/series.go Co-authored-by: Susana Ferreira <ssncferreira@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	3 years ago
Cyril Tovena	c8bdb37ef3	update loki go version to 1.17.8 (#5543) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Callum Styan	bfef7bad44	Start refactor of storage (chunks) clients metrics into structs (#5057) ... * Refactor azure metrics to be a struct (+ add an overall client metrics struct), add an azure client metric for total egress bytes. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add changelog entry for new azure metric. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fixes based on Kavi's review. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix one more merge conflict that I missed in the merge commit. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix failing tests. Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Arve Knudsen	a2dd731586	Chore: Add pkg/util/log package (#5187) ... * Chore: Add pkg/util/log package Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix documentation of loki_log_messages_total Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 years ago
JordanRushing	291360ddcb	Simpler new chunk key v12 (#5054) ... * starts hacking new chunk paths * Create new chunk key for S3; update parsing and add basic test and benchmark Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Fix `TestGrpcStore` to work with new chunk key structure Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Add a function to SchemaConfig that returns the correct PeriodConfig for a given time. Signed-off-by: Callum Styan <callumstyan@gmail.com> Add schema config to object client so it we can use the right external key function based on the schema version. Signed-off-by: Callum Styan <callumstyan@gmail.com> Fix failing compactor tests by passing SchemaConfig to chunkClient Remove empty conditional from SchemaForTime Define schema v12; wire-in ChunkPathShardFactor and ChunkPathPeriod as configurable values Add PeriodConfig test steps for new values ChunkPathShardFactor and ChunkPathPeriod in v12 Update test for chunk.NewExternalKey(); remove completed TODO Set defaults for new ChunkPathPeriod and ChunkPathShardFactor SchemaConfig values; change FSObjectClient to use IdentityEncoder instead of Base64Encoder Use IdentityEncoder everywhere we use FSObjectClient for Chunks Add ExternalKey() function to SchemaConfig; update ObjectClient for Chunks * Finish plumbing through the chunk.ExternalKey -> schemaConfig.ExternalKey change. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Clean up lint failures Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up chunk.go and fix tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Break SchemaConfig.ExternalKey() into smaller functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Quickly fix variable name in SchemaConfig.newerExternalKey() Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up ExternalKey conditional logic; add better comments; add tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix a bug where we are prepending userID to legacy Chunk keys but never parsing it; refactor key tests and benchmarks Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Add small SchemaConfig test Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Update docs and CHANGELOG.md Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Correctly return an error when failing to parse a chunk external key Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Revert IdentityEncoder to Base64Encoder after testing Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix assignment in test for linter Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove leftover comments from development Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Change v12 version comment format; remove redundant login in `ParseExternalKey()` Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Change remaining v12 comment style Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Pass chunk.SchemaConfig to new parallel chunk client * Simplify chunk external key prefixes for schema v12 * Fix broken benchmark; add benchmarks for old parsing conditional Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove unneeded lines from upgrading doc * Add benchmarks for root chunk external key parsing functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * memoizes schema number calculations * Resolve linter issue with PeriodConfig receiver name Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Kaviraj	38c42a26c6	Move `cortex/pkg/tenant` dependency to Loki. (#4944)	3 years ago
Marco Pracucci	c53457feb9	Upgrade Cortex, Prometheus and Thanos (#4830) ... * Upgrade Cortex, Prometheus and Thanos Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed imported ordering Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed data type used for series ref when recovering from checkpoint/WAL Signed-off-by: Marco Pracucci <marco@pracucci.com> * Upgraded Thanos to fix arm build Signed-off-by: Marco Pracucci <marco@pracucci.com> * Run go mod tidy Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Callum Styan	096cf0cfaf	Update golang and loki-build-image image versions. (#4481) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Ed Welch	11e215d79f	Loki: Common Config (#4347) ... * refactor the pkg/util/cfg package to simplify and make it accessable outside of this package. create a pkg/cfg which handles building the Loki config, this is created separately from pkg/util (which is Apache 2 licensed) to maintain proper separations of AGPL and Apache 2 code Update the main.go for several of the apps with the refactoring * refactor how dynamic/common config is loaded Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Add common config logic for path prefix Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Missed a file, update common config with prefix property Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * fix test, remove loki from args Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Clean up test names Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * fix typo in filename, make linter happy Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Update pkg/loki/config_wrapper.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/loki/config_wrapper.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/loki/config_wrapper_test.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * remove temp files in test Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Cyril Tovena	32e55c3043	Fork cortex chunk storage into Loki. (#3842) ... The chunk package has been forked from this commit for future reference and licensing. 09fe17c0e6	4 years ago
Nick Pillitteri	b830d65bdd	Use the Cortex wrapper for getting tenant ID from a context (#3973) ... This change replaces all uses of the weaveworks/common method for getting tenant/org ID with the Cortex wrapper which performs some extra validation. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	4 years ago
Cyril Tovena	490c19e38c	Deprecate max_look_back_period in the chunk storage. (#3677) ... * Deprecate max_look_back_period in the chunk storage. From now on it will be applied at the frontend, ruler and querier level. The querier validation was missing so I added it. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Ed Welch	6e67d1ea5d	Loki/Promtail: Client Refactor (#3623) ... * move docker driver * move fluent-bit output plugin * move fluentd plugin * move logstash plugin * move promtail * match stage uses simple logql label matcher and filter expressions only * remove remaining logql imports * remove helpers package and move contents to util package * move validation and runtime out of util package * fix imports * move build and cfg packages into util * move ParseRequest function to util package * create a new package for logql models and move marshal/unmarshal code to util folder * move ErrorLabel to logqlmodel * move stats package to logqlmodel * decouple loghttp from logql by moving validation to logql package * move gcplog to tools * move PackedEntryKey constant to logqlmodel * fix stats proto import and rebuild protos * fix windows promtail tests * update drone to new dockerfile locations	4 years ago
Ed Welch	36ec92ce5c	Update Go and Alpine versions (#3481) ... * Upgrade go to 1.16 and alpine to 3.13 * update drone to use new build image version * gomod Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	306cc72438	Revendor/cortex (#3446) ... * Upgrade Cortex in Loki Signed-off-by: Marco Pracucci <marco@pracucci.com> * proto compat * gofmt * changelog for deprecated querier.split-queries-by-day Co-authored-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Jacob Lisi	923671a17e	chore: update cortex to latest and fix refs (#3295) ... * chore: update cortex to latest and fix refs Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * tidy mod file Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>	4 years ago
Ed Welch	b4eeddc710	Loki migrate-tool (#3151) ... * basics working, needs some more cleanup and more flags for config and a Dockerfile Signed-off-by: Edward Welch <edward.welch@grafana.com> * add label matchers start end time params and ability to remap the chunk to a different user Signed-off-by: Edward Welch <edward.welch@grafana.com> * adding dockerfile and makefile instructions Signed-off-by: Edward Welch <edward.welch@grafana.com> * push out some of the limits show errors from index query * for testing have docker container not run migrate command but instead stay running by tailing /dev/null * big overhaul to add parallelization * adding better stats * adding retry logic * tweaking retry logic * tweaking retry logic * work around changes in latest cortex Signed-off-by: Ed Welch <edward.welch@grafana.com> * fix up imports and changes since the last time this was merged added a readme fixed up the makefile added some comments * fixing some of my garbage * fixing more of my garbage * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * tabs not spaces Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago