apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Karsten Jeschkies	45ffebd4d8	Return HTTP 400 when multiple tenants are present in push. (#5800 )	3 years ago
Ed Welch	c158b2c5f5	Loki: Revert #4845 which changed the format of errors from the API (#5772 ) * This reverts the changes from #4845, unfortunately changing the content type from plain text to JSON is a breaking change of the API and we should wait to do this at a major release. Signed-off-by: Ed Welch <edward.welch@grafana.com> * fix tests which were updated after initial change. Signed-off-by: Ed Welch <edward.welch@grafana.com>	3 years ago
Karsten Jeschkies	46956d0bf6	Use dskit/tenant intead of loki/pkg/tenant. (#5724 )	3 years ago
Cyril Tovena	79f0e349f5	Refactor LogQL syntax into its own package (#5539 ) * Refactor LogQL syntax into its own package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Merge upstream Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/loghttp/series.go Co-authored-by: Susana Ferreira <ssncferreira@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	3 years ago
RangerCD	daacfd0b70	Fix two remote_timeout configs in ingester_client block (#5418 ) * Fix two remote_timeout configs in ingester_client block * Add default value of remote_timeout in document	3 years ago
Michel Hollands	d8ed87d04c	Update dskit version (#5392 ) * Update dskit version Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * update changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add debug printing to see interfaces in CI. This will be reverted Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add comment to remember to remove Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use netutil.PrivateNetworkInterfacesWithFallback for the other rings Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Move import to place indicated by linter Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove debug printing Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Determine local interfaces in tests Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rearrange import to satify linter Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update build image in CircleCI Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Change changelegog message to show customer visible changes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add doc tag and use private network interfaces in common config Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add comment about filtered out non private IP addresses Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update the documentation of the interface names settings Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Cyril Tovena	bbaef790db	Add usage report into Loki. (#5361 ) * Adds leader election process Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fluke Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixes the kv typecheck * wire up the http client * Hooking into loki services, hit a bug * Add stats variable. * re-vendor dskit and improve to never fail service * Intrument Loki with the package * Add changelog entry Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes compactor test Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add configuration documentation Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/usagestats/reporter.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * Add boundary check Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add log for success report. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/usagestats/reporter.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	3 years ago
Danny Kopping	20041b74d6	Revert "Simplify code for write path in distributor (#5248 )" (#5337 ) This reverts commit `10b93cebe5`. We discovered that these changes can lead to log deduplication issues. Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Christian Haudum	10b93cebe5	Simplify code for write path in distributor (#5248 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Christian Haudum	8d2b3f7aa5	Return early if push payload does not contain data (#5229 ) Optimize for: * payload does not contain any streams * a stream does not contain any entries * none of the streams contain any entries Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Arve Knudsen	a2dd731586	Chore: Add pkg/util/log package (#5187 ) * Chore: Add pkg/util/log package Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix documentation of loki_log_messages_total Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 years ago
Arve Knudsen	d87dbf9f8a	Chore: Upgrade to latest Cortex (#5085 ) * Chore: Upgrade to latest Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Chore: Upgrade to Prometheus@692a54649ed7 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build after upgrading dskit Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 years ago
Kaviraj	cecadf2a18	Chore: Remove `cortex/util/test` dependency (#5050 ) * Remove `cortex/util/test` dependency Copy `util/test.Poll` into Loki Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * go mod vendor Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Kaviraj	564f833a30	Improve error message if incoming logs timestamp is far too behind. (#5040 ) * Improve error message if incoming logs timestamp is far too behind. This is part of JSON response giving out as the response to HTTP /push endpoint. Old message ``` {"code":400,"status":"error","message":"entry for stream '{foo=\"bar\"}' has timestamp too old: 1970-01-01 01:00:00.5 +0100 CET"} ``` New message ``` {"code":400,"status":"error","message":"entry for stream '{foo=\"bar\"}' has timestamp too old: 2021-12-28 01:48:45.5 +0100 CET, accepts timestamp from: 2021-12-29 09:48:45.737756651 +0100 CET"} ``` main rationale being, hard to know what is the closest timestamp that Loki is expecting without going through the config. Also config is not straight forward (its `latest stream's timestamp - ingester.max-chunk-age/2`) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Tweaks the error message Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix `distributor/validator_test`. 1. Make `time.Now()` mockable. 2. Make time format consistent across error messages. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Rename `getValidationContextFor` to `getValidationContextForTime` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Dylan Guedes	98ecfefd89	Add distributor ring page (#4938 ) * Implement new IngestionRateStrat Enum. - This new IngestionRateStrat is used internally to represent the chosen ingestion rate strategy, to avoid using plain strings. * Add a ring to the distributor. As of now, the distributor only has a pointer to its lifecycler. Since the ring HTTP handler isn't implemented for the lifecycler but it is for the ruler, this extends the distributor by also holding a ring instance. * Implement /distributor/ring page. * Reuse existing global strat const. * Add distributor ring HTTP page tests. * Add missing CHANGELOG entry. * Document missing ring status pages. * Fork Cortex util/http.go into source code.	3 years ago
Dylan Guedes	92c05493e1	Copy `cortex/pkg/distributor` package dependency into Loki (#4983 ) * Fork cortex `pkg/distributor/distributor_ring.go` Copy content from `pkg/distributor/distributor_ring.go` into Loki's `pkg/distributor/distributor_ring.go`. * Fork cortex `pkg/distributor/ingester_client_pool.go`. Copy Cortex `pkg/distributor/ingester_client_pool.go` into Loki's `pkg/distributor/clientpool/ingester_client_pool.go`. We couldn't copy it directly to `pkg/distributor/ingester_client_pool.go` because we distributor imports a package that imports the ingester_client_pool, which leads to import cycle. * Stop using Cortex `pkg/distributor`. Instead, use Loki's `pkg/distributor` and `pkg/distributor/clientpool`. * Remove unused Cortex distributor pkg. * Use random ports for Loki test.	3 years ago
Kaviraj	38c42a26c6	Move `cortex/pkg/tenant` dependency to Loki. (#4944 )	3 years ago
Christian Haudum	e573a4d1a2	Change error responses from plain text to JSON (#4845 ) * Return HTTP errors consistently as JSON response The API returned errors in plain text where the error message was the response body. However, the content type header was set to `application/json`. To make the API responses consistent, errors are now returned as JSON as well. ``` { "message": string, "status": string, "code": int } ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Return HTTP errors consistently as JSON response Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Return 404 Not Found responses as JSON Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Return 404 Not Found responses as JSON Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Callum Styan	822d60507b	log the user ID in the rate limit error message. (#4852 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Marco Pracucci	c53457feb9	Upgrade Cortex, Prometheus and Thanos (#4830 ) * Upgrade Cortex, Prometheus and Thanos Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed imported ordering Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed data type used for series ref when recovering from checkpoint/WAL Signed-off-by: Marco Pracucci <marco@pracucci.com> * Upgraded Thanos to fix arm build Signed-off-by: Marco Pracucci <marco@pracucci.com> * Run go mod tidy Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Susana Ferreira	2d24e2ea64	Migration to dskit/ring (#4641 )	4 years ago
Dylan Guedes	0d66319a1f	Revert distributor defaulting to inmemory. (#4638 ) - [PR 4440](https://github.com/grafana/loki/pull/4440/files) modified the distributor setup to use inmemory by default (instead of consul). That was necessary because, by changing the default rate limiting to global, a ring was necessary, and since by default the ring was set to consul, the project couldn't run with the default configs. That's not the case anymore: right now we are reusing the same config set for the ingester (which users already have properly configured otherwise they wouldn't be able to run the project). List of changes: - Remove changelog entries - Remove upgrading guide entries - Remove added tests (unnecessary checks) - Remove value overriding	4 years ago
Dylan Guedes	3c16658ee0	Loki: Append loopback to ingester net interface default list (#4570 ) * Move LoopbackInterfaceName to its own package. - Create new package `util/net` and puts `LoopbackInterfaceName` there - Modify distributor_test to use `util/net` package * By default, append loopback to ingester interfaces. - Append the loopback net interface to the list of net interfaces used by the ingester ring. The append only occurs if the ingester net interfaces are default - Add CHANGELOG entry * Test ring net interface behavior. - Add tests to check the current net interface name resolution. The tests checks if the ingester ring net interface names are correctly copied to other rings	4 years ago
JordanRushing	ca67292b54	Introduce `overrides-exporter` module to Loki (#4520 ) * Introduce overrides-exporter module from cortex into loki; add some test configs for local dev * Fix panic by adding server dependency to overrides-exporter module * Add tests for limits exporter * Remove dummy configs for testing * Add more limits to export * Add more limits; move mockTenantLimits to exporter_test.go * Introduce overrides-exporter module from cortex into loki; add some test configs for local dev * Fix panic by adding server dependency to overrides-exporter module * Add tests for limits exporter * Remove dummy configs for testing * Add more limits to export * Add more limits; move mockTenantLimits to exporter_test.go * Use reflection; add more limits to export * Fix import to satisfy linter * refactors TenantLimits for clarity & efficiency. Also fixes tenantLimits nil check * Fix breaking test with nil check in runtime_config Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Danny Kopping	cff8324f24	Replacing go-kit/kit/log with go-kit/log (#4484 ) * Replacing go-kit/kit/log with go-kit/log Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the f&^$ing linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding depguard Cleaning go mod Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating weaveworks after https://github.com/weaveworks/common/pull/226 was merged Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the mod checker Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Dylan Guedes	d3d63e1778	Loki: Override distributor's default ring KV store (#4440 ) * Override default KV store for the compactor. - Currently, the default KVstore is consul. This changes it to inmemory to make it easier for new users to run the project without dependencies. - Refactor default flag value tests. Now, it will check for a populated map instead of checking for flags during iteration. * Move distributor flag override to RegisterFlags. * Add missing CHANGELOG and upgrading guide entries.	4 years ago
Arve Knudsen	79b95d976a	Chore: Use dskit/limiter (#4327 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Arve Knudsen	ae424deac1	Upgrade dskit (#4295 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Christian Haudum	8d561f6caf	Chore: Update Cortex and use kv package from grafana/dskit (#4282 )	4 years ago
Bryan Boreham	13a3df8826	Simplify Distributor.push (#4240 ) Instead of creating another goroutine to wait for the background `sendSamples()` to finish, give the channels it is sending on a buffer so that they can send without a listener. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	4 years ago
李国忠	6f78ffe33c	[fix] distributor: fix goroutine leak (#4238 ) Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>	4 years ago
Arve Knudsen	cfb4fc1f55	Use flagext from dskit (#4225 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Nick Pillitteri	79a7bdbabc	Chore: Use services and modules from grafana/dskit (#4196 ) * Update Cortex vendor to bcf15611345da65e03604a08a0213aaaaac49e54 Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Use services and modules packages from grafana/dskit Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Update promtail test to set default weaveworks server network type Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Punctuation Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	4 years ago
Travis Patterson	8a2081a39e	Distributor: Truncate rather than drop log lines (#4051 ) * Truncate rather than drop logs - When log lines are longer than maxLineSize, truncate them to maxLineSize rather than dropping them Fixes: #3747 * Review feedback: - Rename 'max_line_size_should_truncate' to 'max_line_size_truncate' - Add configurable truncated indicator - Add Truncated Line metric - Changed truncated metrics to have the name mutated_{samples,bytes}_total * Linter * Review Feedback - Make metric variable names match metric names - Add benchmarks - reduce truncation allocs by 100x * Linter * Remove truncation indicator	4 years ago
Michel Hollands	c44c12b930	Update Cortex version (#3992 ) * Update Cortex version Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Changed code due to new Cortex version Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Run go mod tidy Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	4 years ago
Nick Pillitteri	b830d65bdd	Use the Cortex wrapper for getting tenant ID from a context (#3973 ) This change replaces all uses of the weaveworks/common method for getting tenant/org ID with the Cortex wrapper which performs some extra validation. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	4 years ago
Sandeep Sukhani	7a7eeb0d63	skip applying retention when data in table is still within retention period and there are no overlapping delete requests (#3934 )	4 years ago
Danny Kopping	71889dcb19	Querier/Ingester: Fixing json expression parser bug (#3928 ) * Fixing json expression parser bug Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * UNRELATED CHANGE: pulling main to fix linter issue Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Cyril Tovena	22dcfcde33	Add retention label to loki_distributor_bytes_received_total metrics (#3840 ) * Add retention label on the loki_distributor_bytes_received_total metrics. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs/sources/operations/observability.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Move ParseRequest to an apache2 package. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add retention label on the loki_distributor_bytes_received_total metrics. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixes bad merge. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes bad merge. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Cyril Tovena	ae23b7e4ed	Rejects push requests with streams without labels. (#3643 ) * Rejects push requests with streams without labels. Fixes #3638 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Missing commit Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Kaviraj	6bcd9807d6	replace `time.Duration` -> `model.Duration` for `Limits`. (#3632 ) Rationale 1. support user-friendly duration format (e.g: "1h30m45s") in JSON value. 2. In consistent with Cortex and tempo.	4 years ago
Ed Welch	6e67d1ea5d	Loki/Promtail: Client Refactor (#3623 ) * move docker driver * move fluent-bit output plugin * move fluentd plugin * move logstash plugin * move promtail * match stage uses simple logql label matcher and filter expressions only * remove remaining logql imports * remove helpers package and move contents to util package * move validation and runtime out of util package * fix imports * move build and cfg packages into util * move ParseRequest function to util package * create a new package for logql models and move marshal/unmarshal code to util folder * move ErrorLabel to logqlmodel * move stats package to logqlmodel * decouple loghttp from logql by moving validation to logql package * move gcplog to tools * move PackedEntryKey constant to logqlmodel * fix stats proto import and rebuild protos * fix windows promtail tests * update drone to new dockerfile locations	4 years ago
Owen Diehl	965d48222b	adds distributor replication factor metric (#3594 )	4 years ago
Ed Welch	21b901db9b	Loki: Distributor log message bodySize should always reflect the compressed size (#3572 )	4 years ago
Michel Hollands	f49a510f38	Loki: Update cortex version and fix resulting changes (#3532 ) * Update cortex version and fix resulting changes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix scrapeconfig test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Tidy up go.sum Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix go.sum Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update pkg/promtail/scrapeconfig/scrapeconfig_test.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Remove unused fmt import Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Owen Diehl	306cc72438	Revendor/cortex (#3446 ) * Upgrade Cortex in Loki Signed-off-by: Marco Pracucci <marco@pracucci.com> * proto compat * gofmt * changelog for deprecated querier.split-queries-by-day Co-authored-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Ed Welch	8012362674	Loki: Per Tenant Runtime Configs (#3460 ) * Add per tenant configs mechanism * Add per tenant logging of stream creation * fix tests * enable per tenant configs on push request logging. * fixing up the stream log, changing log levels to debug	4 years ago
Dmitry Ukolov	6ea7122777	Distributor: append several tests for http parser. (#3416 )	4 years ago
Dr. Jan-Philip Gehrcke	0f881fbf78	distributor: fix snappy pb POST request handling (#3407 ) (#3408 ) Signed-off-by: Dr. Jan-Philip Gehrcke <jp@opstrace.com>	4 years ago
Ed Welch	407afce0e3	Loki: fix validation error and metrics (#3307 ) * Remove the helper functions for creating errors as they cause unintended value manipulation if the value contains any valid golang printf substitute characters, e.g. %s * Improve error message for invalid labels, increment metrics for lines dropped for invalid labels.	4 years ago

1 2 3

125 Commits (88c4781dc72b410a30e054f82d2c4aad0558afbb)