apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Kaviraj Kanagaraj	7e3d2433ed	embedded-cache: Bring `fifocache` and `groupcache` into single tent. (#6821 ) * chore(groupcache): Rename groupcache into memcache{distributed: true} 1. Introduced new config called memorycache config. 2. It has flag `distributed`. By enabling it, we use groupcache Going forward. We will bring fifocache into memcache config(with distributed:false) This PR is just a POC. To show how we can simplify in-built memory cache config and bring it into single config. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Memorycache -> EmbeddedCache * Bring fifocache into embeddedcache tent Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix edge case with enabling fifocache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix missing flags for embedded cache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add changelog Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Warning message for fifocache config Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linter happy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix listenport issue Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * idk. review remarks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks 1. update changelog and upgrade guide 2. s/Embeddedcache/EmbeddedCache/g 3. /groupcache/ handler -> /embedded-cache/ 4. some typos 5. Fix some cache prefixes Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fixing corresponding tests Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Support extra timeouts on embedded-cache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove unwanted single struct field Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * fix(stats-collector): Let cache.New() wrap stats collector for distributed cache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Deprecate max_size_items from fifocache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks about the flag description Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Update docs/sources/upgrading/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Travis Patterson	3bbc5a92f1	Allow fifocache to be configured with groupcache (#6806 )	3 years ago
Kaviraj Kanagaraj	acc3a2fe23	chore(groupcache): Fix panic with stats collector wrapper (#6798 ) * chore(groupcache): Fix panic with stats collector wrapper Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * update Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Kaviraj Kanagaraj	e67dd6c91d	chore(groupcache): Support per cache group capacity (#6793 ) * chore(groupcache): Make per cache group capacity Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove unwanted handler got merged during rebase Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix typo on the comment Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Some standard renaming Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Travis Patterson	cc9c19846f	Fix cache instrumentation (#6753 )	4 years ago
Danny Kopping	b3f1965ff1	Introduce an "enable_groupcache" config to control initialization better (#6673 ) * Introduce an "enable_groupcache" config to control initialization better We initialize groupcache as a module, which happens after the config is parsed. A function named applyFIFOCacheConfig is called to enable fifocache if no cache is configured, and at the time of its calling the modules have not been initialized. This new "enable_groupcache" value is set if groupcache is configured in common, or this value is set manually. With this setting, we can prevent automatic fifocache settings. Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Add tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Simplifying condition Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Travis Patterson	68141444f8	Introduce Groupcache (#6644 ) * implement groupcache * use mailgun groupcache * use memberlist * implement peer discovery with the ring * implement retrieval and storage * use a ticker * Correctly configuring chunk cache Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Enabling results cache Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Fixing groupcache ring page Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * groupcache-poc: Add support for index cache. (#6535) * Hacked in prometheus metrics (#6544) Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * add http to addresses * always make a group for each cache * fix peer discovery * Fix groupcache config - Allow it to be enabled/disabled - Move all config to 'common' config - Always write to hot cache * Add http/2 support for groupcache (#6596) * Add groupcache latency metrics (#6619) * Add groupcache latency metrics * lint * lint * lint * Add groupcache test (#6620) * Add groupcache test * lint * Improve 'Fetch' allocs 11 -> 8 (#6623) * Move constants to the file where they're used * remove irrelevant comments Co-authored-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	4 years ago
Travis Patterson	7f24b0f6ab	Remove ourdated comment (#6597 )	4 years ago
Danny Kopping	36e0979cf5	Add cache usage statistics (#6317 ) * Adding cache statistics Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding metrics to metrics.go Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Creating new stats context for use in metric queries middleware Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Clean up unnecessary log fields Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Fixing tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding stats tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * CHANGELOG entry Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Documenting function Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Moving CHANGELOG entry to appropriate section Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Implementing a stats collector cache wrapper to simplify stats collection If we keep the stats collection in pkg/storage/chunk/cache/instrumented.go, then any implementation that wraps it will cause the stats collected to be incomplete. For example: NewBackground(cacheName, cfg.Background, Instrument(cacheName, cache, reg), reg)) - the background cache requests are not collected Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Fixing tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Christian Haudum	57b384df8f	Auto-expire old items from FIFO cache (#5148 ) * Remove validity from FIFO cache We can remove the validity setting of 1h in 2.4 for the chunks cache because it doesn't reduce memory usage and instead leads to valid chunks being ignore when querying the cache. Chunks are immutable so it doesn't make sense to not return them if they are present in the cache. Closes #4922 Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add periodic task to prune old FIFO cache elements As described in #4921 this PR adds a periodic task that prunes expired items from the FIFO cache to free up memory. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
李国忠	2c20ce6857	[optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" (#4891 ) * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * Update pkg/storage/chunk/cache/instrumented.go Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz> * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * [optimization] cache prometheus : fix "loki_cache_request_duration_seconds_bucket" ‘status_code’ label always equals "200" #4891 * Update pkg/querier/queryrange/roundtrip.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update roundtrip.go Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
李国忠	29cccbe576	[enhancement] querier cache: WriteBackCache should be off query path (#5083 ) * [enhancement] querier cache: WriteBackCache should be off query path #5072 * [enhancement] querier cache: WriteBackCache should be off query path #5083 * Update pkg/storage/chunk/chunk_store_utils.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * [enhancement] querier cache: WriteBackCache should be off query path #5083 * [enhancement] querier cache: WriteBackCache should be off query path #5083 * Update pkg/storage/chunk/cache/cache.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * [enhancement] querier cache: WriteBackCache should be off query path #5083 * [enhancement] querier cache: WriteBackCache should be off query path #5083 * Update pkg/storage/chunk/chunk_store_utils.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * [enhancement] querier cache: WriteBackCache should be off query path #5083 Co-authored-by: Danny Kopping <dannykopping@gmail.com>	4 years ago
Siavash Safi	cd9d9c9b42	Support Redis Cluster Configuration Endpoint (#4947 ) The go-redis options requires multiple redis addresses for a cluster. In some scenarios like AWS ElastiCache, a single configuration endpoint is used to resolve all cluster nodes. This change adds an extra step to resolve all nodes and pass them to go-redis. It remove the need to specify multiple cluster nodes in the configuration. No tests were added for this scenario since it requires patching the Go resolver during runtime. Signed-off-by: Siavash Safi <siavash.safi@gmail.com>	4 years ago
Dylan Guedes	77ea11724b	Loki: Enable FIFO cache by default (#4519 ) * Set default FIFO cache with reasonable defaults. Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Remove FIFO cache feature flag. - We noticed that we always want that behavior, so the logic is better implemented in the `ApplyDinamicConfig` method - Update documentation * Move FIFO cache default behavior. - Move FIFO cache default behavior implementation to ApplyDinamicConfig * Add tests for the new FIFO cache default behavior. - The tests tests the behavior of the four existing caches (query results, query index, write dedupe, and chunk results) - Add changelog entry - Fix lint issues regarding import order - Revert moving of EnableFifoCache check * Apply suggestions from code review - Change `EnableFifoCache` description to mention that it is only applied for chunk and query results cache Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Mauro Stettler <mauro.stettler@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Danny Kopping	cff8324f24	Replacing go-kit/kit/log with go-kit/log (#4484 ) * Replacing go-kit/kit/log with go-kit/log Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the f&^$ing linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding depguard Cleaning go mod Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating weaveworks after https://github.com/weaveworks/common/pull/226 was merged Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the mod checker Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Cyril Tovena	32e55c3043	Fork cortex chunk storage into Loki. (#3842 ) The chunk package has been forked from this commit for future reference and licensing. `09fe17c0e6`	5 years ago
Michel Hollands	c44c12b930	Update Cortex version (#3992 ) * Update Cortex version Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Changed code due to new Cortex version Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Run go mod tidy Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	5 years ago
Sandeep Sukhani	337db2fedf	revendor cortex to latest master (#2627 )	5 years ago
Sandeep Sukhani	4a358bc9ea	revendor cortex to latest master (#2463 )	6 years ago
Aditya C S	199e334126	Update loki to cortex master (#2030 )	6 years ago
Sandeep Sukhani	49f04ef4bc	update cortex to latest master (#1956 )	6 years ago
Peter Štibraný	6d5408c100	Update Cortex to latest master (#1869 ) * Update Cortex to master Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix compilation errors after Cortex update. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Update to Cortex v1.0.0-rc.0 Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Added missing vendored file. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Vendor Cortex 1.0.0 Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Configuration changes. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Mention Cortex 1.0 changes and link to docs. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix jsonnet files, and mention fields that were removed. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Cyril Tovena	3f3152427a	Update cortex vendoring to include frontend status code improvement. (#1746 ) * Update cortex vendoring to include frontend status code improvement. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes signature changes in runtimeconfig.NewRuntimeConfigManager. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Forgot to git add . :facelpalm: Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Owen Diehl	018e6bcdfd	bumps cortex to 0.6.1 master (#1612 )	6 years ago
Peter Štibraný	d4cae667ae	loki: use new runtimeconfig package from Cortex (#1484 ) * loki: use new runtimeconfig package from Cortex runtimeconfig is a generalization of previous "overrides" mechanism, and can be used not for per-tenant limits, but other parts of Loki too (only one additional usage for now is "multi" KV client). Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy, go mod vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed RuntimeConfig from Ingester dependencies. Ingester already depends on RuntimeConfig via Store and Overrides mechanism. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed extra new line. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Remove unused method. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix broken code after rebasing. Removed incorrect comment. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Updated docs with section about runtime configuration. Mostly copied from Cortex. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy, go mod vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Updated Cortex to current master. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Go mod tidy, vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Peter Štibraný	3df5561336	Updated cortex to latest master. (#1088 ) * Updated cortex to latest master. * Updated cortex to latest master. * Fix tests after updating cortex. * go mod tidy and vendor	6 years ago
Sandeep Sukhani	b687ec6e5b	vendoring: update cortex to latest master (#938 ) * update cortex to latest master using v3.4.0-rc.1 for go.etcd.io/etcd v0.0.2 for github.com/prometheus/procfs v1.0.0 for github.com/prometheus/client_golang latest master for github.com/weaveworks/common with changes from https://github.com/weaveworks/common/pull/153 * fixed failing tests * use large instance for test and lint jobs in CircleCI * running only 6 test binaries in parallel * removed resource type change for CircleCI * changed GOGC to 10 for lint makefile target	6 years ago
sh0rez	421265c90c	chore(vendor): update cortex vendor Updates cortex vendor to get the changes from cortexproject/cortex#1431. Required for making promtail build on Windows Signed-off-by: sh0rez <me@shorez.de>	7 years ago
Goutham Veeramachaneni	0e86bfd5c3	Add support for memcaches (#564 ) * vendor: update cortex Don't be blocked on https://github.com/cortexproject/cortex/pull/1345 to be merged and still maintain the fork Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * add memcached ksonnet Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * commit generated proto Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Tom Wilkie	e3e4a2d9ae	Update vendored Cortex. (#213 ) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Adam Shannon	eb390523e3	all: update dependencies (#211 ) * all: update dependencies * Update some more dependancies and pin the right versions for Prometheus deps. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	9f257c591a	[WIP] Persistence (#44 ) Persistence chunks using the Cortex chunk store.	7 years ago

16 Commits (6e2b7d25898f321bfd6f94de05d6ce934c785da9)