835 Commits (e8cce09de29dc58c38b333e72dfe11613d49fbb3)

Author	SHA1	Message	Date
Cyril Tovena	3640429532	Bypass sharding middleware when a query can't be sharded. (#2709) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	6500f82195	Improve lz4 compression (#2614) ... * Improve lz4 compression. - Move to v4. - Remove not required checksuming. - Default will now write 4M blocks, it's backward compatible for reads. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * vendor update Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Ed Welch	2162a0c1b1	Do not mark the position if the file is removed, doing so only results in an error because you can't get the file size of a removed file. (#2658)	5 years ago
BaiLian	0ac1ce4e5a	Querier APIs respond JSON Content-Type (#2665)	5 years ago
Amin Jamali	9fcb8b8e4a	Add fallback_formats for timestamp stage (#2615) ... Fixes grafana/loki#2590 Co-authored-by: Amin Jamali <ajamali@vmware.com>	5 years ago
Lars Lehtonen	0c9f7a58a4	pkg/querier: fix dropped error (#2671) ... Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>	5 years ago
Sandeep Sukhani	6053cdb9d9	fix store query bug when running loki in single binary mode with boltdb-shipper (#2655)	5 years ago
Sandeep Sukhani	79c027550c	run multiple queries per table at once with boltdb-shipper (#2656)	5 years ago
Owen Diehl	89b8ae4b49	improves numeric literal stringer impl (#2663)	5 years ago
Owen Diehl	5729d88302	exposes rule group validation fn (#2662)	5 years ago
Owen Diehl	21d3e69790	Enable local rules backend & disallow configdb. (#2661) ... * revendors cortex * vendoring compat * disallow configdb ruler backend * go mod tidy	5 years ago
Owen Diehl	6f8bfe0c79	Ruler docs + single binary inclusion (#2637) ... * starts alerting docs * ruler in single binary * make docs interactive * alerting docs * ruler prom alerts endpoint * Apply suggestions from code review Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * doc fixes * capitalize ruler * removes double spaces * Update docs/sources/alerting/_index.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Ed Welch <ed@oqqer.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Ed Welch <ed@oqqer.com>	5 years ago
Aurélien Rainone	21addf7935	pkg/distributor: fix test by looking for loopback interface (#2645) ... TestDistributor_PushIngestionRateLimiter failed if the system had no interface name in "eth0", "en0" or "lo0". Fix that by looking at the name of the loopback interface in the list of the system's network interfaces, then using that name in the test. Fixes #2356	5 years ago
Aurélien Rainone	85696d00eb	pkg/chunkenc: fix test using string(int) conversion (#2647) ... Since go1.15, there's a new vet check for code such as string(x) where x has an integer type other than rune. This vet check is enabled by default on go test. TestSerialization failed because of that, this commit replaces `string()` conversions with `strconv.Itoa` calls Fixes #2646	5 years ago
Owen Diehl	9e85757d89	adds prometheus ruler routes (#2650)	5 years ago
Ed Welch	0b1dbe26fe	Promtail: Add a stream lagging metric (#2618) ... * add a metric which displays how far a stream lags the last timestamp in a batch * register your metrics * recorded metric in the wrong spot, moved to the correct spot * cardinality is much too high if we add all the stream labels, instead just use filename. * work on using an interface to set/remove metrics * reworking interface again * for now just register this metric based on filename only, we can improve this in the future if the need arises * change the timeout to 1min for expiring the metric * add host label * include host label when removing metric * change the batch size for promtail from 100kb to 1Mb, this will not affect sending for lower volumes which are still driven by the BatchWait setting. Through the addition of this metric it was found that for higher volume files > 100kb/sec this was much too low causing too many batches to be unnecessarily sent.	5 years ago
Sandeep Sukhani	29790ea19b	fix an issue with building loki (#2642)	5 years ago
Sandeep Sukhani	5b361a0c89	improvements for boltdb-shipper compactor (#2640) ... 1. Run the compaction process on startup instead of waiting for the first tick. 2. Run the compactor service when using loki with boltdb-shipper in single binary mode without clustering.	5 years ago
Sandeep Sukhani	2f7e0e50c2	rpc for querying ingesters to get chunk ids from its store (#2601) ... * rpc for querying ingesters to get chunk ids from its store * fix broken tests * separate out code from querier to query ingester to a new type called IngesterQuerier * add AsyncStore which helps with querying and merging results from both chunk store and ingesters * query ingesters local store for labels when active index type is boltdb-shipper or query store is true * better naming * add some tests * add test case for duplicate chunk ids from ingesters * add some logs and spans * check for nil fetcher * use require.Same to assert that two pointers are pointing to same object * some more improvements 1. query chunk-refs parallely from store and ingesters. 2. limit look back for chunk refs and labels from ingesters to start of schema when using boltdb-shipper. * fixes suggested from PR review * fix mode not being set before cortex_storage.NewStore is being called * add IngesterQuerier as a dependency for store when target is either ingester or querier * do not cache index from ingester * rename tests * fix broken tests * update upgrade guide to rollout ingesters before queriers * upgrade guide changes suggested in PR review * runtime check for whether active index store is boltdb-shipper * consider previous schema as well for boltdb-shipper for calculating max look back	5 years ago
Sandeep Sukhani	337db2fedf	revendor cortex to latest master (#2627)	5 years ago
Lars Lehtonen	334cc1e59a	pkg/storage/stores/shipper/uploads: fix test error (#2620) ... Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>	5 years ago
Roger Steneteg	edb5fc5176	Promtail: adding pipeline stage for dropping labels (#2571) ... * adding pipeline stage for dropping labels, our use case for this is to move unbound labels into the log line and dropping the label, fixes #2421 Signed-off-by: Roger Steneteg <rsteneteg@ea.com> * Update docs/sources/clients/promtail/stages/labeldrop.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/clients/promtail/stages/_index.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * fixed inconsistent naming, removed unused logger in the labeldrop stage Signed-off-by: Roger Steneteg <rsteneteg@ea.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>	5 years ago
Sandeep Sukhani	e53d2032fe	fix a panic when trying to stop boltdb-shipper multiple times using sync.once (#2613)	5 years ago
Ed Welch	2015763327	Loki: Fix query-frontend ready handler (#2610) ... * query frontend should not have a default route as it usurps the ready handler, the ready handler was also not implemented completely for the query frontend. * enable ready check for query-frontend in ksonnet	5 years ago
Robert Fratto	ee67b9c707	Expose UserAgent and fix User-Agent version source (#2603) ... * Use prometheus/common/version for reporting build version in User Agent header Promtail wasn't using github.com/prometheus/common/version for the User-Agent header when obtaining the build version. Instead, it imported github.com/grafana/loki/pkg/build which modifies the Prometheus version package on init. This causes problems for importers that also import and modify github.com/prometheus/common/version with their own build information (i.e., grafana/agent). This PR changes Promtail to obtain the build info from the common dependency to obtain the correct build information regardless of how Promtail is being run. * expose UserAgent to allow it to be modifiable by importers	5 years ago
Sandeep Sukhani	6dd2e9ef34	Read, Write and operational dashboard improvements (#2584) ... 1. Add bigtable, boltdb-shipper to read/write dashboards. 2. Add boltdb-shipper to operational dashboard. 3. Record latency for write operation in boltdb-shipper.	5 years ago
Calle Pettersson	82a8cf425f	Add a counter for empty syslog messages (#2565)	5 years ago
Ben Ye	900f714e9d	add _total suffix to ruler counter metrics (#2582) ... Signed-off-by: Ben Ye <yb532204897@gmail.com>	5 years ago
Owen Diehl	164f5cd0ae	Ruler/loki rule validator (#2589) ... * revendors cortex * loki multitenantmanager * vendoring compat * go mod	5 years ago
Sandeep Sukhani	7566aa4950	fix closing of compressed file from boltdb-shipper compactor (#2574)	5 years ago
Aditya C S	c1f07f08a7	Validate max_query_length in Label API (#2572)	5 years ago
Ed Welch	b8b24f3129	clone labels when sending them to Handle function for stdin target (#2570)	5 years ago
Owen Diehl	f905e77259	exports grouploader (#2578)	5 years ago
Owen Diehl	4032c927cb	strict rule unmarshaling (#2580)	5 years ago
Owen Diehl	7b68293e26	logql rule loading (#2576)	5 years ago
Owen Diehl	8627af1199	Error on no schema configs (#2564) ... * ruler jsonnet * jsonnet fixes * errors instead of panics on no schema configs	5 years ago
Sandeep Sukhani	0a0ad68af2	fix dir setup based on which mode it is running (#2559)	5 years ago
Sandeep Sukhani	1bba2a89f8	add some metrics for monitoring compactor (#2548)	5 years ago
Sandeep Sukhani	4f6a4c2a93	cleanup boltdb files in queriers during startup/shutdown (#2558)	5 years ago
Sandeep Sukhani	2bb2eebace	register boltdb shipper compactor cli flags (#2546)	5 years ago
Owen Diehl	aea6c3a36d	Feature/ruler (take 2) (#2458) ... * begins speccing out ruler ruler memhistory encapsulates metricsHistory removes mutex from memhistory ruler module * upstream conflicts * implicit ast impls, parser for ruler * /api/prom ruler routes, ruler enabled in single binary * registers ruler flags, doesnt double instantiate metrics * cleanup for old samples in ruler * begins ruler tests * ForStateAppenderQuerier tests * memhistory stop * RestoreForState test * upstream querier ifc * introducing loki ruler metrics * removes rule granularity metric -- to be discussed in pr * validates ruler cfg * renames gauge metrics to not use total * removes unnecessary logs * logs synthetic restoreforstate * logs tenant in ruler * sets cortex to owen's unmerged fork * begins porting rules pkg * memstore work * work on queryable based in memory series store * removes unused pkgs, adds memstore test * MemStore must be started after construction * MemstoreTenantManager * ruler loading * ruler instantiation * better metrics & logging in ruler * grpc cortex compatibility in go.mod * cortex vendoring compat * increments memory cache hits only if cached * loki in memory metrics use prometheus default registerer * ruler only depends on ring * managerfactory rename * revendors cortex * ignore emacs stashing * adds comments * ruler /loki/api/v1 prefix * revendoring compat * comment	5 years ago
Sandeep Sukhani	dab393a62d	fix a flaky test in boltdb shipper (#2550)	5 years ago
Owen Diehl	750ab36bc1	fixes batch metrics help text & corrects bucketing (#2552)	5 years ago
Ed Welch	ee28dbff2d	implement shutdown for the no-op server (#2542) ... (cherry picked from commit b4c6065b97)	5 years ago
Ed Welch	62da8bb858	restart the tailer if we fail to read and upate current position (#2532) ... (cherry picked from commit b6d9fd5a00)	5 years ago
Sandeep Sukhani	b489130828	compactor for compacting boltdb files uploaded by shipper (#2526)	5 years ago
Sandeep Sukhani	97dfb297a0	compress boltdb files to gzip while uploading from shipper (#2507)	5 years ago
Sandeep Sukhani	501f93fa92	revendor cortex to latest master (#2543)	5 years ago
Sandeep Sukhani	9b0740c0ed	upload boltdb files from shipper only when they are not expected to be modified or during shutdown (#2487) ... * upload boltdb files from shipper only when they are not expected to be modified or during shutdown * updated comment * add one more test condition to make sure files are not re-uploaded Co-authored-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Ed Welch	b4c6065b97	implement shutdown for the no-op server (#2542)	5 years ago