apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Dylan Guedes	90c4821dd3	Docs: Document the common ring section (#4664 ) * Document the common ring section. - Create a new <ring_config> section that document the pkg/util/ring_config.go - Rewrite ruler and scheduler_ring docs to make usage of <ring_config> section - Document new ring under common section * Update docs/sources/configuration/_index.md Co-authored-by: Ed Welch <ed@oqqer.com>	4 years ago
Danny Kopping	5b30589acf	Documenting recording rules per-tenant WAL (#4566 ) * WIP Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Document ruler config options Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Self-review updates Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Clarifying WAL benefit in response to review Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding dashboard link Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Update docs/sources/configuration/_index.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Ensuring ruler and ingester default WAL paths do not conflict Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Trevor Whitney	62ee399e48	default ingester final sleep to 0 unless otherwise specified (#4608 ) * default ingester final sleep to 0 unless otherwise specified * add changelog and upgrading * update config docs	4 years ago
Ed Welch	0c9beadaaa	Loki: Add a ring to the compactor used to control concurrency when not running standalone (#4574 ) * add a ring to the compactor * wiring up the start/stop functionality * add some monitoring for multiple compactors running at the same time. * Add compactor to `read` target * fix tests/validation requirements * Update pkg/storage/stores/shipper/compactor/compactor_ring.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/storage/stores/shipper/compactor/compactor_ring.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/storage/stores/shipper/compactor/compactor_ring.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * used a shared ring_config between scheduler and compactor to avoid some duplication * fix lint * better handling of the tokens file with a way to easily tell all the rings to persist tokens * fix query scheduler replication factor * removing "scheduler" from log line which may be called by components other than a scheduler in the future. * update changelog Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Owen Diehl	45113cb375	Enable unordered_writes by default (#4594 ) * enable unordered_writes by default * removes out_of_orders from transfer test * update changelog	4 years ago
Dylan Guedes	331fa6d7b0	Docs: Add virtual targets docs (#4565 ) * Add CHANGELOG entry and updated modes diagram. * Update documentation with the new hybrid mode. - Update targets with new 'read' and 'write' options - Add new docker-compose yaml that exemplifies how to use new 'read' and 'write' targets * Add missing targets to configuration docs. - Missing ones: index-gateway, ingester-querier, query-scheduler	4 years ago
李国忠	6fcd02bad4	[querier] s3: add getObject retry (#4453 ) * [querier] s3: add getObject retry * [querier] s3: add getObject retry * [querier] s3: add getObject retry #4452 * [querier] s3: add getObject retry * [querier] s3: add getObject retry #4453 * [querier] s3: add getObject retry * [querier] s3: add getObject retry #4453 * [querier] s3: add getObject retry #4453 * [querier] s3: add getObject retry #4453 * [querier] s3: add getObject retry Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Dylan Guedes	e0b4e25962	Document the common config section (#4485 ) * Document the common_config section. * Apply suggestions from code review Fix subsection reference from `filesystem_storage_config` to `local_storage_config` Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Dylan Guedes	77ea11724b	Loki: Enable FIFO cache by default (#4519 ) * Set default FIFO cache with reasonable defaults. Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Remove FIFO cache feature flag. - We noticed that we always want that behavior, so the logic is better implemented in the `ApplyDinamicConfig` method - Update documentation * Move FIFO cache default behavior. - Move FIFO cache default behavior implementation to ApplyDinamicConfig * Add tests for the new FIFO cache default behavior. - The tests tests the behavior of the four existing caches (query results, query index, write dedupe, and chunk results) - Add changelog entry - Fix lint issues regarding import order - Revert moving of EnableFifoCache check * Apply suggestions from code review - Change `EnableFifoCache` description to mention that it is only applied for chunk and query results cache Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Mauro Stettler <mauro.stettler@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Dylan Guedes	f4cfd284a6	Docs: Move rule storages configs to their own sections (#4493 ) * Move rule storages configs to their own sections. - Create new docs sections `azure_storage_config`, `gcs_storage_config`, `s3_storage_config`, and `swift_storage_config` with the docs configs from `ruler_config["storage"]` - Reference the new sections in the existing `ruler_config` section - Add missing periods at `ruler_config` configs * Update storages to have a CLI flag prefix - This way we can reuse these configs in different places (different from `-ruler.config`)	4 years ago
Ed Welch	2b5f3000b6	Loki: Add a ring to the query scheduler to allow discovery via the ring as an alternative to DNS (#4424 ) * Fork the frontend and scheduler so we can add support for discovery via the ring. * Add a ring to the query scheduler which is then used by the frontend worker and querier workers to find the scheduler address as an alternative to using DNS * update some of the forked code to use dskit grpcclient and grpcutil since #4312 was merged after the original fork was created. * remove query scheduler dependency from querier Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * migrate logging package and make linter happy Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * add SafeReadRing for instances when scheduler is not enabled on same instance * Doc changes from code review A few doc fixes Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * go mod tidy on go-kit dep * update changelog * remove ReadRing in favor of SafeReadRing Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Dylan Guedes	6498c8cb13	Loki: Change default limits to common values (#4415 ) * Change some default limits to reasonable values. - These new limits protect users from overwhelming their cluster with ingestion load - The new limits are: * ingestion_rate_strategy: global <- currently defaults to local * max_global_streams_per_user: 5000 <- current default is no limit * max_query_length: 721h <- current default is no limit * max_query_parallelism: 32 <- current default is 14 * max_streams_per_user: 0 <- current default is 10000 * reject_old_samples: true <-- current default is false * reject_old_samples_max_age: 168h <-- current default is 336h * Remove parameters from examples - They are not necessary anymore because of the new default values * Add new entries to CHANGELOG and upgrading guide * Apply suggestions from code review Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com> * Change changelog mark entry from '-' to '*' Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	4 years ago
JordanRushing	4c5b084525	Add DroneCI pipeline stage to validate loki example configs; create example configuration files (#4358 ) * Add DroneCI pipeline stage to validate loki example configs; create example configuration files * Actually run loki in the validate-example-configurations pipeline stage * Change method for running loki binary * Add step dependency for config validation pipeline stage * Actually loop over example configs and verify them! * Push bad config to test failing builds * Create remaining example configs * Fix provided example loki configs * Use make target to validate example configs * Add make target to generate a new example configuration doc; add make target for CI to verify doc is re-generated upon changes to example configs * Update CI to check the generated example config doc * Clean up Makefile * Mark new example config validation make targets as phony; fix conditional for config example doc checking * Fix conditional * Remove redundant step in generate-example-config-doc phony make target * Add a correction to PR 4358 * Improve grammar and add newlines in example configuration descriptions Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Regenerate examples.md Co-authored-by: Karen Miller <karen.miller@grafana.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Trevor Whitney	b07920cb16	auto-apply memberlist ring config when join_members provided (#4400 ) * auto-apply memberlist ring config when join_members provided Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * move "memberlist" to a const Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * add comment for memberlist config auto-apply, update changelog Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * add docs Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com>	4 years ago
Karen Miller	04ab08dc89	Docs: remove empty section "Generic placeholders" (#4417 )	4 years ago
Karen Miller	24accf6a5b	Docs: correctly represent product name (#4416 ) The product name is Grafana Loki. The first use of the product name on each web page should be the full product name. After that, within each page, it is fine to use just Loki. This PR also set up some frontmatter with weights where it was missing on a few files, and correct where "Loki" was not capitalized.	4 years ago
Karen Miller	acde1799b6	Docs: clarify distinction between single binary and microservices. (#4374 ) * Docs: clarify distinction between single binary and microservices. * Update clarification of mode per review suggestions	4 years ago
lingenavd	d388472475	Update _index.md (#4324 )	4 years ago
Karen Miller	e8ac2214fb	Docs: inject newlines for configuration section readability (#4247 )	4 years ago
Owen Diehl	20515a2402	better per stream rate limits configuration options (#4236 ) * better per stream rate limits configuration options * clean up test config * pr feedback	4 years ago
Karen Miller	c97dc1b9eb	Docs: revise max_query_lookback knob definition (#4245 )	4 years ago
Jordan Rushing	a58aea1bf4	Update limits_config docs to include querier.max_query_lookback flag (#4244 )	4 years ago
Karen Miller	11a0d28b61	Docs: first draft, Loki accepts out-of-order writes (#4237 ) * Docs: first draft, Loki accepts out-of-order writes * Update docs/sources/architecture/_index.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Benoît Knecht	488a21bff7	pkg/storage/chunk/aws: Add s3.http.ca-file option (#4211 ) * pkg/storage/chunk/aws: Fix insecure-skip-verify documentation The documentation claimed that the `s3.http.insecure-skip-verify` option needed to be set to `false` in order to skip verifying certificates, but it needs to be set to `true`. Signed-off-by: Benoît Knecht <bknecht@protonmail.ch> * pkg/storage/chunk/aws: Add s3.http.ca-file option This option lets users set a custom CA file for connections to S3, in case they use a local S3 instance with an internal PKI. Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>	4 years ago
Owen Diehl	f17b189a3f	Promote/ruler api (#4132 ) * unmark the ruler api as experimental * deprecation notice for experimental prefix * removes vendor changes, adds -ruler.enable-api (without experimental prefix) on our ruler.Config	4 years ago
Owen Diehl	654b543e4e	updates unordered writes config docs (#4131 ) * updates unordered writes config docs * removes experimental flag * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Yashar Nesabian	4369998aea	update boltdb-shipper index period (#4122 )	4 years ago
Owen Diehl	92a4e57448	ingester.index-shards config (#4111 ) * consistent shard mapping test & better error msg * ingester.index-shards config * loki fails to start if the ingester & schema factors arent compatible. * lint * docs * update changelog	4 years ago
Karsten Jeschkies	c3aaebe509	Support frontend V2 with query scheduler. (#4071 ) * Support Cortex Frontend V2. * Format code. * Start query scheduler in Docker compose. * Model query flow in sequence diagram. * Model calls to downstreamer. * Move diagram to docs. * Embed generated SVG. * Descripe calls. * Launch Cortex query scheduler. * Disable sharding for now. * Log which framework is used. * Configure querier to use scheduler. * Remove unwanted docs. * Enable query splitting again. * Document scheduler address. * Document new scheduler options for frontend as well. * Remove unused querier option. * Use new query-scheduler module in Loki. * Remove unused option. * Document `max_queriers_per_tenant`.	4 years ago
Travis Patterson	8a2081a39e	Distributor: Truncate rather than drop log lines (#4051 ) * Truncate rather than drop logs - When log lines are longer than maxLineSize, truncate them to maxLineSize rather than dropping them Fixes: #3747 * Review feedback: - Rename 'max_line_size_should_truncate' to 'max_line_size_truncate' - Add configurable truncated indicator - Add Truncated Line metric - Changed truncated metrics to have the name mutated_{samples,bytes}_total * Linter * Review Feedback - Make metric variable names match metric names - Add benchmarks - reduce truncation allocs by 100x * Linter * Remove truncation indicator	4 years ago
Karen Miller	808c808714	Clarify prose about querier frontend min-sharding-lookback (#4050 ) * Clarify prose about querier frontend min-sharding-lookback Docs edit of prose from https://github.com/grafana/loki/pull/4047. * Update docs/sources/configuration/_index.md Co-authored-by: Danny Kopping <dannykopping@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	4 years ago
Karen Miller	1e85036409	Revise wording of query frontend service configuration (#4052 ) * Revise wording of query frontend service configuration * Add ClusterIP and publishNotReadyAddresses to Kubernetes YAML	4 years ago
Owen Diehl	a8a861b21a	adds query-frontend service hints for k8s, adapted from jsonnet (#4045 )	4 years ago
Cyril Tovena	3f3d10afcc	Add min_sharding_lookback limits to the frontends (#4047 ) * Add min_sharding_lookback limits to the frontends Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve doc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Sandeep Sukhani	58f9d0c085	add config and documentation about index-gateway (#3956 ) * add config and documentation about index-gateway * Update docs/sources/operations/storage/boltdb-shipper.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/storage/boltdb-shipper.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Vincent Miszczak	d6dc8b991c	Add ingester.autoforget-unhealthy-timeout opt-in feature (#3919 ) * Add ingester.autoforget-unhealthy-timeout opt-in feature * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Karen Miller	7eceea082b	Docs: Make identification of experimental items consistent and more obvious. (#3911 )	5 years ago
Paul Seiffert	2cee4e31d0	List 'compactor' as valid value for target option (#3938 )	5 years ago
Thomas Belian	5e5e638b66	Add memcached_client parameter option (#3799 ) Parameter addresses is tested with AWS ElastiCache memcached and works, this parameter is taken from cortex documentation	5 years ago
sanadhis	eab103fddf	docs: http_path_prefix as correct item of server_config (#3868 )	5 years ago
Kaviraj	9a64334f2a	Add missing `-querier.max-concurrent` config in the doc (#3875 )	5 years ago
Danny Kopping	8788ddb8d6	Ruler: documentation for recording rules (#3851 ) * Documenting recording rules Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Minor tweaks Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Correcting text Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Update docs/sources/rules/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	5 years ago
Sandeep Sukhani	12d418b7cc	docs for logs deletion feature (#3826 ) * docs for logs deletion feature * update doc Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	5 years ago
Nolan Crooks	6d026d211d	[Docs] Minor tweaks (#3846 )	5 years ago
Thomas Belian	ddbf04f64c	Change redis configuration value for enabling TLS to correct syntax (#3798 ) enable_tls: true doesn't work and this syntax is taken from cortex documentation, works, and is tested with AWS ElastiCache Redis cluster	5 years ago
Cyril Tovena	71677dbc8f	Retention doc (#3775 ) * wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finish docs. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Addresses Karen's feedback. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs/sources/operations/storage/retention.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/storage/retention.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/storage/retention.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/storage/retention.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	5 years ago
Karen Miller	3940f059f0	Docs: remove unnecessary lists of sections (#3790 ) Links to the sections are in the right side navigation panel.	5 years ago
Ed Welch	fa247353ed	Add a flag which allows the queriers to run completely standalone and only query stored data. (#3700 )	5 years ago
Cyril Tovena	58bf983518	Fixes doc w/r/t grpc compression. (#3551 ) * Fixes doc w/r/t grpc compression. Fixes #3537. Due to a change in upstream we didn't catch that breaking change in the config. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update _index.md * Update _index.md * Update _index.md	5 years ago
Owen Diehl	24f8f6a4de	compactor docs (#3543 )	5 years ago

1 2

81 Commits (bfc77118a161f9cb7afe95c7fa89fffd523bbabe)