apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Eng Zer Jun	e2842c69c5	Replace deprecated `io/ioutil` with `io` and `os` packages (#7212 ) The `io/ioutil` package has been deprecated in Go 1.16 (See https://pkg.go.dev/io/ioutil). This PR replaces the existing `io/ioutil` functions with their new definitions in `io` and `os` packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	3 years ago
Travis Patterson	071118a090	revert stream push error response (#7160 ) We've changed approaches to how we want to do per-stream rate limiting. The new approach will rely on creating an api on the ingester so the distributors can decide how streams should be sharded. As a result, the work to return rate-limited streams in the error body is no longer needed.	3 years ago
Travis Patterson	3f0ecd3c7b	Implement all-or-nothing stream ingestion (#7094 ) Per-stream sharding will introduce automatic stream sharding on rate-limited streams. The ingester's current line-by-line behavior may result in duplicates when a partially-accepted stream is resharded and sent to other ingesters. This PR makes the ingester either rate limit the entire stream before attempting to write lines to a chunk. The PR 1. Separates validation, storage, and wal replay into their own functions 2. Implements a flag to rate limit the whole stream or not 3. Checks rate limits for the whole stream rather than line-by-line 4. Returns an error with information on the whole stream when rate limited	3 years ago
Travis Patterson	bc16450349	Revert "Hide duplicates caused by stream sharding (#7005 )" (#7031 ) @owen-d had a suggestion that was dramatically dimpler. Revert this whole thing and the simplified PR will be forthcoming	3 years ago
Travis Patterson	a98b7ebaf4	Hide duplicates caused by stream sharding (#7005 ) As streams are sharded and resharded across different ingesters, the likelihood of duplicate logs being ingested increases dramatically. This removes duplicates at query time by dynamically removing the `__shard_stream__` label for both log and sample queries so the `stream hash` used by the `MergeEntryIterator` will match when doing comparisons for duplicates. The `stream hash` is created in either the `store` or `ingester instance` at query time so the labels are removed there. Additionally, the `MergeEntryIterator` has been modified to remove duplicates in a stream rather than just across streams. For completeness, `__shard_stream__` has also been removed from Label and Series Queries.	3 years ago
李国忠	562ccd6932	[bugfix] fix flush.go nil pointer panic (#6993 ) What this PR does / why we need it: Fix a panic when reporting chunk statistics. The chunk would already be garbage-collected as it is marked as flushed before reporting its statistics. This fix it by first reporting its statistics and only then marking it as flushed. Panic description: ```go panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x177d158] goroutine 14455 [running]: github.com/grafana/loki/pkg/chunkenc.(MemChunk).Size(0xc00441c750?) /loki@v1.6.2-0.20220816085209-65ed7b60fbd4/pkg/chunkenc/memchunk.go:611 +0x18 github.com/grafana/loki/pkg/ingester.(Ingester).reportFlushedChunkStatistics(0xc00087a000, 0xc00f8efe80, 0xc0519d0870, {0x36b7008, 0xc01a593860}, {0x36b7008, 0xc01a5938c0}, {0x307e2f9, 0x7}) /loki@v1.6.2-0.20220816085209-65ed7b60fbd4/pkg/ingester/flush.go:395 +0x2f2 ```	3 years ago
Dylan Guedes	bba6c14459	Loki: Modify ingesters to return rate-limited streams to distributors (#6977 ) What this PR does / why we need it: Modify ingesters to return a list of rate-limited streams on its reply to `(distributor).Push`. This is done by using the gRPC details feature. I've updated the tests and apparently, the returned error message is exactly the same so this shouldn't be a breaking change. Also modifies `(stream).Push` to return a list of all entries with errors instead of returning the final gRPC error. This will allow us in the future to easily return to distributors the indices of all reject entries, so that distributors won't have to always retry all entries. This work is necessary for the new stream sharding feature.	3 years ago
Owen Diehl	57cec8e496	[TSDB] Ingester Index Sampling (#6852 ) * queryrangebase definitions protos & aliases Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * indexgatewaypb to logproto Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * updates queryrange.proto for new definitions file Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * s/indexgatewaypb/logproto/g Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * renames to indexseries for clobbering reasons Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * more indexgatewaypb removal compatibility work Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * proto fmt Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * rebuild imports Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * goimports -s Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * query ingesters for index stats Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * compatibility during rollouts for idx-gw+ingester-querier post proto refactor Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * proto compatibility with old indexgateway pkg path Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * test compat Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * gofmt Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Christian Simon	19c7315bcc	Improve error message when max error message is hit (#6834 ) before: `entry with timestamp 2022-08-02 12:32:37 +0000 UTC ignored, reason: 'entry too far behind' for stream: {foo="bar2"}` after: `entry with timestamp 2022-08-02 12:32:37 +0000 UTC ignored, reason: 'entry too far behind, oldest acceptable timestamp is: 2022-08-03T15:25:57Z' for stream: {foo="bar2"}`	3 years ago
Danny Kopping	66fdb3c572	Log stream flushes to see how many chunks we flush at a time (#6819 ) * Log stream flushes to see how many chunks we flush at a time Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Smuggling in an error wrapping which aides debugging Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Update docs/sources/upgrading/_index.md Co-authored-by: Ed Welch <ed@oqqer.com> * Removing unnecessary label Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Ed Welch <ed@oqqer.com>	3 years ago
Ed Welch	d7345c1c4c	Loki: Do not store exact log line duplicates (now including unordered inserts) (#6642 ) * do not insert log lines which are an exact duplicate in both timestamp and log content Signed-off-by: Edward Welch <edward.welch@grafana.com> * update test to now ignore dupes pushed out of order Signed-off-by: Edward Welch <edward.welch@grafana.com> * make the test unordered to be more thorough Signed-off-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Marco Pracucci	e7fb10f456	Loki: Update `grafana/dskit` and `weaveworks/common` (#6643 ) * Upgrade dskit and weaveworks/common Signed-off-by: Marco Pracucci <marco@pracucci.com> * 'go mod vendor' again Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Owen Diehl	094f8f3b20	[TSDB] fingerprint offsets is now correctly 16 byte aligned (#6630 ) * simpler shard factor calculations, disallows identity factor of 1 Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * log interval span as soon as it finishes Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * another shard bounds test Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * indexclient span logging Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * more index span logging Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * correctly match single shard Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * shardedPostings test Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * shardedPostings maxOffset is properly exclusive Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * minor posting idiomatic-ness Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * store 16 byte aligned offsets for fingerprint table Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Owen Diehl	8eed9a4dde	dont call getinstance twice in checkpointing (#6516 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Jesus Vazquez	db9dbe8e1d	Bump prometheus dependency (#6403 ) * Bump prometheus dependency We had to add a replace of prometheus in go mod because thanos is importing an older version and then replacing it. * Undo textparser.New variable mismatch * modtimevfs path update * Fix labels withlabels and withoutlabels usages WithoutLabels and WithLabels methods have dissappeared. Now labels.NewBuilder has to be used. * Fix RuleManager interface for Update() Update now accepts a new param called ruleGroupPostProcessFunc which is an optional function to be executed after rule group processing. In this case I've set it to nil in the usages i've found. * NewQueryRange now accepts QueryOpts The NewQueryRange function now accepts QueryOptions so I had to update all the calls. I've set the opts to nil since we're not using them. * gofmt	4 years ago
Christian Haudum	c12a1f4f43	feature: Add ingester handler for shutdown and forget tokens (#6179 ) * Update grafana/dskit to 07166f9 Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add /ingester/shutdown handler This handler replaces the deprecated /ingester/flush_shutdown handler and can be used to gracefully shut down a Loki instance and delete the file that persists the tokens of the ingester ring. In production environments you usually want to persist ring tokens so that during a restart of an ingester instance, or during rollout, the tokens from that instance are not re-distributed to other instances, but instead kept so that the same streams end up on the same instance once it is up and running again. For that, the tokens are written to a file that can be specified via the `-ingester.tokens-file-path` argument. In certain cases, however, you want to forget the tokens and re-distribute them when shutting down an ingester instance. This was already possible by calling `/ingester/flush_shutdown`, deleting the tokens file and terminating the process. The new handler `/ingester/shutdown` combines these manual steps into a single handler. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add changelog entry Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Mathew Heard	de418bf6ac	Reduce to a single code path through sendBatches regardless of limit (#6216 ) * Reduce to a single code path through sendBatches regardless of limit specified * simplify sendBatches by using -1 to specify no limit * test fix	4 years ago
Owen Diehl	4903df6ae5	Tsdb/inverted index wiring (#6252 ) * multi inverted index * delete in inverted index signature * multi inverted index * inverted index testing + skip nil indices * wires up period config aware multi-inverted-idx through ingesters * better inverted index validation * apply shipper defaults to last relevant period config, not current one. * more verbose error	4 years ago
Christian Simon	7f50a26656	Move ingester metrics around (#6275 )	4 years ago
Owen Diehl	c032da9639	Tsdb/inverted index (#6233 ) * begins adding a factor-of-two dynamically sharded inverted index * bit prefix inverted index tests * goimports	4 years ago
Sandeep Sukhani	1923668a5d	run store queries in ingester when using tsdb as index store (#6209 )	4 years ago
Ed Welch	4bf1648da1	Loki: When processing logs during tailing, don't reuse the query pipeline (#6152 ) * when tailing, don't reuse the query pipeline because it has an unbounded label cache Signed-off-by: Edward Welch <edward.welch@grafana.com> * update changelog Signed-off-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Kaviraj Kanagaraj	24f8b19e91	Remove unused grpc health check endpoints (#6113 ) We have generic `grpc_healthcheck` endpoint that get registered for all the components during initialization stage. Looks like these health checks that are part of actual component's struct itself came from legacy. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Owen Diehl	756fc19577	fix mutability bug with labels during tsdb flushing (#6116 )	4 years ago
Christian Haudum	18058d2b55	Serve correct status of ingester/distributor in gRPC healthcheck endpoint (#6094 ) * Serve correct status of ingester in gRPC healthcheck endpoint The gRCP healthcheck endpoint is used by clients that determin whether a connection to the service is healthy or not. In case the ingester is not in "running" state and is not "active" in the ring, this healthcheck should return "not serving". Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Serve correct status of distributor in gRPC healthcheck endpoint	4 years ago
Owen Diehl	b45efd4c2d	TSDB shipper + WAL (#6049 ) * begins speccing out TSDB Head * auto incrementing series ref + mempostings * mintime/maxtime methods * tsdb head IndexReader impl * head correctly populates ref lookup * tsdb head tests * adds prometheus license to tsdb head * linting * [WIP] speccing out tsdb head wal * fix length check and adds tsdb wal encoding tests * exposes wal structs & removes closed semantics * logs start time in the tsdb wal * wal interface + testing * exports walrecord + returns ref when appending * specs out head manager * tsdb head manager wal initialization * tsdb wal rotation * wals dont use node name, but tsdb files do * cleans up fn signature * multi tsdb idx now just wraps Index interfaces * no longer sorts indices when creating multi-idx * tenantHeads & HeadManger index impls * head mgr tests * bugfixes & head manager tests * tsdb dir selection now helper fns * period utility * pulls out more code to helpers, fixes some var races * head recovery is more generic * tsdb manager builds from wals * pulls more helpers out of headmanager * lockedIdx, Close() on idx, tsdbManager update * removes mmap from index reader implementation * tsdb file * adds tsdb shipper config and refactors initStore * removes unused tsdbManager code * implements stores.Index and stores.ChunkWriter for tsdb * chunk.Data now supports an Entries() method * moves walreader to new util/wal pkg to avoid circular dep + tsdb storage alignment * tsdb store * passes indexWriter to chunkWriter * build a tsdb per index bucket in according with shipper conventions * dont open tsdb files until necessary for indexshipper * tsdbManager Index impl * tsdb defaults + initStore fix for invalid looping * fixes UsingTSDB helper * disables deleteRequestStore when using TSDB * pass limits to tsdb store * always start headmanager for tsdb Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * fixes copy bug Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * more logging Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * fixes duplicate tenant label bug Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * debug logs, uses label builder, removes __name__=logs for tsdb Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * tsdb fixes labels at earlier pt Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * account for setting tenant label in head manager test * changing tsdb dir names * identifier interface, builder to tsdb pkg * tsdb version path prefix * fixes buildfromwals identifier * fixes tsdb shipper paths * split buckets once per user set * refactors combining single and multi tenant tsdb indices on shipper reads * indexshipper ignores old gzip logic * method name refactor * remove unused record type * removes v1 prefix in tsdb paths and refactores indices method * ignores double optimization in tsdb looking for multitenant idx, shipper handles this * removes 5-ln requirement on shipper tablename regexp * groups identifiers, begins removing multitenant prefix in shipped files * passses open fn to indexshipper * exposes RealByteSlice * TSDBFile no longer needs a file descriptor, parses gzip extensions * method signature fixing * stop masquerading as compressed indices post-download in indexshipper * variable bucket regexp * removes accidental configs committed * label matcher handling for multitenancy and metricname in tsdb * explicitly require fingerprint when creating tsdb index * only add tenant label when creating multitenant tsdb write fingerprints without synthetic tenant label strip out tenant labels from queries * linting + unused removal * more linting :( * goimports * removes uploadername from indexshipper * maxuint32 for arm32 builds * tsdb chunk filterer support * always set ingester name when using object storage index Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	4 years ago
Ed Welch	4f75939efb	Revert "Loki: Change live tailing to only allow mutating the log line not the number of streams. (#6063 )" (#6097 ) This reverts commit `98fda9b0a0`.	4 years ago
Ed Welch	98fda9b0a0	Loki: Change live tailing to only allow mutating the log line not the number of streams. (#6063 ) * don't create more streams when live tailing logs, process the log line with the pipeline to allow mutating the content of the line but don't split it into new streams. Signed-off-by: Edward Welch <edward.welch@grafana.com> * update changelog Signed-off-by: Edward Welch <edward.welch@grafana.com> * don't mutate the incoming streams Signed-off-by: Edward Welch <edward.welch@grafana.com> * do not reuse the pipeline because it caches labels Signed-off-by: Edward Welch <edward.welch@grafana.com> * add note to upgrade guide Signed-off-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Christian Simon	6c1a62d829	Ensure proto definitions are formatted [2/2] (#6000 ) * Update build image to 0.10.4 * Add a fmt target for proto defintions This uses buf to ensure the proto defintions are more consistently formatted. * Update formatted files	4 years ago
Christian Haudum	b6ce3826b4	Use new build image 0.20.3 (#5904 )	4 years ago
Travis Patterson	7299e3bdcd	Fix query filtering (#5951 ) - milli -> nano conversion in the querier - add deletes to store requests from the ingester	4 years ago
Travis Patterson	5939d5e9d8	Query filtering in the ingester and storage (#5629 ) * Add timestamp to pipeline and extractor interfaces * implement filter extractor and pipeline * implement storage filtering * implement ingester filtering * test cleanup * Refactor pipeline setup to util * linter * enable filtering based on mode * review feedback * review comments * review feedback * no need to include label filters in the test pipeline * review feedpack	4 years ago
Ed Welch	d1f6a1968e	Loki: Fix issue with recent change to flush code (#5932 ) * fix flush chunks to pass a reference so we can mutate the chunk contents Signed-off-by: Edward Welch <edward.welch@grafana.com> * update the rest of the helper methods to take a reference of the chunk Signed-off-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Dylan Guedes	b3b96568ed	Loki: Flush chunks one at a time (#5894 ) * Flush chunks one at time. - Calls `store.Put` (flushes chunk to the store) for each chunk individually until it doesn't work instead of calling it for all chunks a single time * Refactor chunk flushing into smaller functions. * Rearrange flushing locks. - Updates our locking strategy to release the chunk lock more frequently. This release the lock after closing and marking a chunk as flushed. * Change where we report `chunksFlushedPerReason` metric.	4 years ago
Cyril Tovena	8f02495453	Refactoring of the Loki storage (#5833 ) * Extract index interface from the series store * moving package around * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * moving package around * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve composite store for more generics index store * fixes first round of tests and ensure coverage is still good * implement series method * Fixes storage tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add test for GetSeries Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Make Loki compile Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint the whole project Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * tsdb util package usage Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more information of why SetChunkFilterer exists for the index Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes ruler/base package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes o2 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	fcb9812f5d	Fixes deduping when multiple timestamp are equal (#5799 ) * Fixes deduping when multiple timestamp are equal Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * update changelog Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes int64 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * that nit was not worth Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Ed Welch	b136d0dab3	Increase flush_op_timeout to 10m, 10s is often much too short for a big WAL replay which can generate a lot of chunks very quickly. (#5768 ) Signed-off-by: Ed Welch <edward.welch@grafana.com>	4 years ago
Karsten Jeschkies	46956d0bf6	Use dskit/tenant intead of loki/pkg/tenant. (#5724 )	4 years ago
Christian Simon	ea62af58e9	Introduce a ChunkRef type as part of logproto (#5643 ) * Change protobuf definitions * Adapt to an embedded ChunkRef struct * Move the Checksum to the ChunkRef	4 years ago
Cyril Tovena	59bd44a5df	Remove more chunkstore and schema version below v9 (#5650 ) * Remove more chunkstore and schema version below v9 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes remaining tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes remaining tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	79f0e349f5	Refactor LogQL syntax into its own package (#5539 ) * Refactor LogQL syntax into its own package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Merge upstream Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/loghttp/series.go Co-authored-by: Susana Ferreira <ssncferreira@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	4 years ago
Cyril Tovena	8f693e77b0	Update Prometheus to latest (#5545 ) * Update Prometheus to latest Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update promql.QueryTracker Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes querier test to use new grafana/regexp Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes test with new textparse signature Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	6dad54fef9	Fixes ingester sample deduping (#5470 ) * Fixes ingester sample deduping * Call get hash only once * Update pkg/iter/entry_iterator.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Cyril Tovena	5d8bc6129e	Correctly sets hash value for headblock iterator (#5423 )	4 years ago
Michel Hollands	d8ed87d04c	Update dskit version (#5392 ) * Update dskit version Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * update changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add debug printing to see interfaces in CI. This will be reverted Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add comment to remember to remove Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use netutil.PrivateNetworkInterfacesWithFallback for the other rings Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Move import to place indicated by linter Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove debug printing Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Determine local interfaces in tests Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rearrange import to satify linter Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update build image in CircleCI Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Change changelegog message to show customer visible changes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add doc tag and use private network interfaces in common config Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add comment about filtered out non private IP addresses Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update the documentation of the interface names settings Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	4 years ago
Owen Diehl	1837c9e0b2	Experimental TSDB index (#5376 ) * moves encoding to own pkg * decwrap impl * forks tsdb index * renames tsdb_enc * wraps all tsdb decbufs * use loki encbuf * custom chunk meta replacing ref-id for uint32 checksum * handle overlapping chunks in tsdb encoding * tsdb builder impl * tsdb querier beginnings to use modified tsdb * fixes nonexistent reference * lint/remove unused * adds tsdb testware * lint	4 years ago
Cyril Tovena	bbaef790db	Add usage report into Loki. (#5361 ) * Adds leader election process Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fluke Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixes the kv typecheck * wire up the http client * Hooking into loki services, hit a bug * Add stats variable. * re-vendor dskit and improve to never fail service * Intrument Loki with the package * Add changelog entry Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes compactor test Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add configuration documentation Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/usagestats/reporter.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * Add boundary check Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add log for success report. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/usagestats/reporter.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	4 years ago
Kaviraj Kanagaraj	2db309271c	Makes `tailer.droppedStreams` slice bounded. (#5334 ) * Makes `tailer.droppedStreams` slice bounded. For slow receiver, this dropped streams may cause ingester memory to keep increasing. Sometimes OOMing the ingester. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make droppedStreams slice size configurable Update tests Adding logging when slice is dropped Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add test for dropped stream Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Cyril Tovena	e3b74158a5	Fixes log deduplication when mutating Labels using LogQL (#5289 ) * Changes the iterator interface to include labelshash. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve unused args and fixes build Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Implement the new hash in the memchunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add the hash in logproto Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Insert the hash from ingester-querier Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Rename LabelsHash to StreamHash Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working through fixing tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor HeapIterator into merge and sort Iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Order alphabetically when ordering samples. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Do not JSON encode the hash for legacy model. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more regression tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update CHANGELOG.md Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback	4 years ago
Dylan Guedes	cf419d1cb8	Implement `LegacySample`, `LegacyLabelPair` proto messages (#5301 ) * Add LegacySample and modify protos to use it. * [Skip this commit] Regenerate protos. * Modify codebase to use LegacySample instead of Sample. For reference, take a look at PR https://github.com/grafana/loki/pull/5151. * Implement LegacyLabelPair. * Use new Legacy implementations accordingly. * Use LegacySample in compat tests. - Since `compat.go` was forked from Cortex, we can reliably change all mentions from `Sample` to `LegacySample` there.	4 years ago

1 2 3 4 5 ...

324 Commits (6e2b7d25898f321bfd6f94de05d6ce934c785da9)