5 Commits (46813867758729df3e18763ad7366f141291f3c9)

Author	SHA1	Message	Date
Charles Korn	bde65667f7	Upgrade dskit to use packages migrated from weaveworks/common (#10164) ... **What this PR does / why we need it**: This PR upgrades dskit and replaces use of packages from weaveworks/common with their migrated equivalents in dskit. See https://github.com/grafana/dskit/pull/342 for more details. Note that Loki still uses some packages from weaveworks/common that I haven't migrated (`aws` and `test`) - I'll migrate these separately. If this PR needs to be rebuilt, I used `rewrite.sh` ([source](https://gist.github.com/charleskorn/48efe62a09d6d70f3de30327003df5c5#file-rewrite-sh)) to generate most of these changes. **Which issue(s) this PR fixes**: (none) **Special notes for your reviewer**: (none) **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	2 years ago
Karsten Jeschkies	f24da4a194	Display replicas, cores and nodes in sizing tool. (#8088) ... **What this PR does / why we need it**: No every users wants to deploy Loki with Helm. That's why we are going to display the required replicas as well.  This change also fixes two bugs in the algorithm and introduces some tests for its invariant. **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> Co-authored-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> Co-authored-by: liam-howe-maersk <118169897+liam-howe-maersk@users.noreply.github.com> Co-authored-by: JordanRushing <rushing.jordan@gmail.com> Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Periklis Tsirakidis <periklis@redhat.com> Co-authored-by: Ashwanth Goli <iamashwanth@gmail.com>	2 years ago
Karsten Jeschkies	95ec710168	Introduce sizing tool to documentation page. (#7830) ... **What this PR does / why we need it**: The sizing tool should help to find a good cluster sizing when starting out with Loki. It's a dynamic form withing the documentation.  This pull request will be followed by help on hover. **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Karsten Jeschkies <karste.jeschkies@grafana.com>	3 years ago
Danny Kopping	6bf5b5d8c1	Use a line-buffered logger to deamplify write syscalls (#6954) ... We initialise a global logger in `pkg/util/log/log.go` and use it extensively throughout the Loki codebase. Every time we write a log message, a `write` syscall is invoked. Syscalls are problematic because they transition the process from userspace to kernelspace, which means: - a context-switch is incurred, which is inherently expensive ([1-2 microseconds](https://eli.thegreenplace.net/2018/measuring-context-switching-and-memory-overheads-for-linux-threads/)) - the goroutine executing the code is **blocked** - the underlying OS thread (_M_ in the go scheduler model) is **also blocked** - the goroutine has to be rescheduled once the syscall exits - the go scheduler may need to spawn additional OS threads if all are blocked in syscalls - which can also be expensive This change introduces the use of a line-buffered logger. It has a buffer of [256](https://gist.github.com/dannykopping/0704db32c0b08751d1d2494efaa734c2) entries, and once that buffer is filled it will flush to disk. However, a situation will arise in which that buffer remains somewhat empty for a period of time, so there is a periodic flush mechanism, configured to flush every 100ms. There is also a preallocated bytes slice of 10MB which is reused, to avoid excessive slice resizing & garbage collection. This does mean that we could lose up to 256 log messages in case of an ungraceful termination of the process, but this would need to be precisely timed within the 100ms flushes - in other words, the likelihood is low, and generally we shouldn't `kill -9` any Loki process.	3 years ago
Vladyslav Diachenko	615941234a	Logql Analyzer (#6171) ... * created logql-debugger page and API endpoint that returns information for each stage of the query * added docker-compose setup for local testing * refactored analyzer and added unit tests * removed docs page to release it in a separate PR when back-end API is published * changes structure of results object in response to return origin log line needed for case when only stream selector is sent in query * merged the latest main and resolved conflicts * added endpoint for readiness probe and configured CI * fixed drone.yml discrepancy * fixed method signature * fixed path to Dockerfile * added clean step	3 years ago