apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Paul Rogers	18cef217fc	chore: Linting updates in prep for Go 1.23 (#14071 )	9 months ago
Max Forasteiro	9bf08f7cc0	feat(lambda-promtail): build lambda with zip file (#13787 ) Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	9 months ago
Jay Clifford	ef03476f3d	fix(helm): removed helm test (#13651 )	10 months ago
Michel Hollands	7b7d3d4cd2	feat: add lokitool (#12166 ) Signed-off-by: Michel Hollands <michel.hollands@gmail.com>	1 year ago
Trevor Whitney	2ea36f7679	fix(nix): no systemd package on darwin systems (#12229 )	1 year ago
Kaviraj Kanagaraj	fac5997b18	feat: Support split align and caching for instant metric query results (#11814 ) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	1 year ago
Trevor Whitney	53949dc5cd	add scan-vulnerabilities make target (#10971 ) What this PR does / why we need it: Adds a make file target for running both trivy and snyk to scan our images for vulnerabilities during/before releases.	2 years ago
Trevor Whitney	8aaf5c10a8	add stats, volume, and volume_range commands to logcli (#9966 ) Add `stats`, `volume`, and `volume_range` commands to `logcli`. Does not implement the file client for now. I think it would be cool if in the future the file client could read from a downloaded TSDB index file for these commands.	2 years ago
Callum Styan	220cbdd4f1	Revert "Use centralized make-docs script from Writers' Toolkit (#9126 )" (#9146 ) This reverts commit `df1b34898b`. Sorry @jdbaldry but something in your PR made the helm-docs check fail, and only fail in CI, I tried a few things to figure out what was happening locally but the same make target always passed locally. cc @trevorwhitney	2 years ago
Jack Baldry	df1b34898b	Use centralized make-docs script from Writers' Toolkit (#9126 ) - Adds ability to build multiple projects simultaneously using, for example, `make docs PROJECTS='grafana grafana-cloud'`. - Adds `make doc-validator` which runs [`doc-validator`](https://github.com/grafana/technical-documentation/tree/main/tools/cmd/doc-validator) on all documentation. Using a centralized script will help ensure consistency in workflow across all projects. Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	2 years ago
Trevor Whitney	6c53113be1	[helm] Add a loki canary test to the helm chart (#7229 ) What this PR does / why we need it: Add a helm chart test that utilizes the Loki canary to confirm the Loki cluster is functioning correctly. Which issue(s) this PR fixes: Fixes #7228 Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz>	3 years ago
Enzo Nocera	4f9c85df5d	Guidelines: Update CONTRIBUTING.md (#6757 ) For a new contributor (which I am 🎉), the `CONTRIBUTING.md` file might seem a bit vague in regard to why we need to use the `GOPATH`-scoped Loki repository to contribute. The rationale is now given (If I've missed any other reasons, please tell me). * Fixed wrong level title, * Added information about local workflow when dealing with locally modified dependencies, * Added references to Go 1.18 workspaces. Signed-off-by: Raskyld <enzo@nocera.eu>	3 years ago
Trevor Whitney	a369189752	Add nix configuration (#7020 ) What this PR does / why we need it: This PR adds nix configuration to the Loki repo to make it easier for people who use nix to integrate building Loki into their existing workflows. For example, I have a `direnv` hook that uses nix shell to make sure I have the correct version of all dependent binaries (ie. `go`, `golangci-lint`, etc) on my `$PATH` whenever I change into the Loki directory. I find this especially useful for building with `CGO` enabled as it requires a C compiler with systemd headers to be available on the path. Furthermore, by having a `flake.nix` in the root of this repo, it will make it easier for users of the nix package manager to grab a specific version of Loki for their own systems / projects. This PR does _not_ attempt to replace any existing build processes, this is merely supplemental. All these nix configurations wrap existing `Makefile` targets, and the addition of these files will have no effect on anyone not using nix. Please see the README.md included in this PR for more information. Checklist - [X] Documentation added Co-authored-by: Christian Simon <simon@swine.de>	3 years ago
Jesus Vazquez	f657122ccd	Git ignore pkg/loki/wal (#6405 ) * Git ignore pkg/loki/wal I noticed that after running make test the directory pkg/loki/wal is created which a bunch of files and data. I think it would be a good idea to ignore the entire directory with git. * Trigger CLA sign	3 years ago
Travis Patterson	d97724e8dd	Introduce coverage to PR pipelines (#5357 ) * Add code coverage diff to CI * remove test package from coverage diff * review feedback * lint * Env variable test * faster experiment * drone vars * drone vars * how does the shell work? * add pr comment * escape json * report diff in comment * properly format json * quote post body * review feedback * add querier/queryrange to coverage	3 years ago
Owen Diehl	0c826b834a	Tsdb/compaction (#5783 ) * ChunkMetas.finalize uses pool * slight refactoring. Uses pools during chunkmetas.finalize and adds an InclusiveBounds helper * multi idx explicitly wraps tsdb files * TSDB Checksum helper + requires at least 2 indices for MultiIndex constructor * tsdb compaction impl * skips compacting a single file * compaction + testing * tsdb compaction tests * passes reference to &index.ChunkMetasPool for avoiding nocopy * linting * gitignore .vscode * s/Combine/FilePath/g	3 years ago
Christian Haudum	474adc0820	Fix path to binary in Dockerfile for querytee (#5537 ) * Fix path to binary in Dockerfile for querytee The Dockerfile referenced an incorrect path of the querytee binary causing the build of the querytee image to fail. ``` $ make loki-querytee-image docker build -t grafana/loki-querytee:main-84c97ce -f cmd/querytee/Dockerfile . Sending build context to Docker daemon 650.6MB ... Step 7/8 : COPY --from=build /src/loki/cmd/querytee/loki-querytee /usr/bin/querytee COPY failed: stat src/loki/cmd/querytee/loki-querytee: file does not exist make: *** [Makefile:526: loki-querytee-image] Error 1 ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add querytee binary to Docker ignore file Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Ed Welch	6e67d1ea5d	Loki/Promtail: Client Refactor (#3623 ) * move docker driver * move fluent-bit output plugin * move fluentd plugin * move logstash plugin * move promtail * match stage uses simple logql label matcher and filter expressions only * remove remaining logql imports * remove helpers package and move contents to util package * move validation and runtime out of util package * fix imports * move build and cfg packages into util * move ParseRequest function to util package * create a new package for logql models and move marshal/unmarshal code to util folder * move ErrorLabel to logqlmodel * move stats package to logqlmodel * decouple loghttp from logql by moving validation to logql package * move gcplog to tools * move PackedEntryKey constant to logqlmodel * fix stats proto import and rebuild protos * fix windows promtail tests * update drone to new dockerfile locations	4 years ago
Brett Jones	af23e36bc7	init	4 years ago
Ed Welch	b4eeddc710	Loki migrate-tool (#3151 ) * basics working, needs some more cleanup and more flags for config and a Dockerfile Signed-off-by: Edward Welch <edward.welch@grafana.com> * add label matchers start end time params and ability to remap the chunk to a different user Signed-off-by: Edward Welch <edward.welch@grafana.com> * adding dockerfile and makefile instructions Signed-off-by: Edward Welch <edward.welch@grafana.com> * push out some of the limits show errors from index query * for testing have docker container not run migrate command but instead stay running by tailing /dev/null * big overhaul to add parallelization * adding better stats * adding retry logic * tweaking retry logic * tweaking retry logic * work around changes in latest cortex Signed-off-by: Ed Welch <edward.welch@grafana.com> * fix up imports and changes since the last time this was merged added a readme fixed up the makefile added some comments * fixing some of my garbage * fixing more of my garbage * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * tabs not spaces Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Kaviraj	6856209093	Terraform script to automate GCP provisioning for gcplog (#3206 ) * Terraform script to automate GCP provisioning for gcplog * PR remarks	4 years ago
Reinhard Nägele	1bb4301f2e	Fix multiline docs (#3035 ) Signed-off-by: Reinhard Nägele <unguiculus@gmail.com>	5 years ago
Owen Diehl	4d9865acd4	Adds WAL support (experimental) (#2981 ) * marshalable chunks * wal record types custom serialization * proto types for wal checkpoints * byteswith output unaffected by buffer * wal & record pool ifcs * wal record can hold entries from multiple series * entry pool * ingester uses noopWal * removes duplicate argument passing in ingester code. adds ingester config validation & derives chunk encoding. * segment writing * [WIP] wal recovery from segments * replay uses sync.Maps & preserves WAL fingerprints * in memory wal recovery * wal segment recovery * ingester metrics struct * wal replay locks streamsMtx in instances, adds checkpoint codec * ingester metrics * checkpointer * WAL checkpoint writer * checkpointwriter can write multiple checkpoints * reorgs checkpointing * wires up checkpointwriter to wal * ingester SeriesIter impl * wires up ingesterRecoverer to consume checkpoints * generic recovery fn * generic recovery fn * recover from both wal types * cleans up old tmp checkpoints & allows aborting in flight checkpoints * wires up wal checkpointing * more granular wal logging * fixes off by 1 wal truncation & removes double logging * adds userID to wal records correctly * wire chunk encoding tests * more granular wal metrics * checkpoint encoding test * ignores debug bins * segment replay ignores out of orders * fixes bug between WAL reading []byte validity and proto unmarshalling refs * conf validations, removes comments * flush on shutdown config * POST /ingester/shutdown * renames flush on shutdown * wal & checkpoint use same segment size * writes entries to wal regardless of tailers * makes wal checkpoing duration default to 5m * recovery metrics * encodes headchunks separately for wal purposes * merge upstream * linting * addresses pr feedback uses entry pool in stream push/tailer removes unnecessary pool interaction checkpointbytes comment fillchunk helper, record resetting in tests via pool redundant comment defers wg done in recovery s/num/count/ checkpoint wal uses a logger encodeWithTypeHeader now creates its own []byte removes pool from decodeEntries wal stop can error * prevent shared access bug with tailers and entry pool * removes stream push entry pool optimization	5 years ago
Owen Diehl	aea6c3a36d	Feature/ruler (take 2) (#2458 ) * begins speccing out ruler ruler memhistory encapsulates metricsHistory removes mutex from memhistory ruler module * upstream conflicts * implicit ast impls, parser for ruler * /api/prom ruler routes, ruler enabled in single binary * registers ruler flags, doesnt double instantiate metrics * cleanup for old samples in ruler * begins ruler tests * ForStateAppenderQuerier tests * memhistory stop * RestoreForState test * upstream querier ifc * introducing loki ruler metrics * removes rule granularity metric -- to be discussed in pr * validates ruler cfg * renames gauge metrics to not use total * removes unnecessary logs * logs synthetic restoreforstate * logs tenant in ruler * sets cortex to owen's unmerged fork * begins porting rules pkg * memstore work * work on queryable based in memory series store * removes unused pkgs, adds memstore test * MemStore must be started after construction * MemstoreTenantManager * ruler loading * ruler instantiation * better metrics & logging in ruler * grpc cortex compatibility in go.mod * cortex vendoring compat * increments memory cache hits only if cached * loki in memory metrics use prometheus default registerer * ruler only depends on ring * managerfactory rename * revendors cortex * ignore emacs stashing * adds comments * ruler /loki/api/v1 prefix * revendoring compat * comment	5 years ago
simnv	766b08078e	Ignore .exe files build for Windows (#2549 ) When building Loki on Windows with `go build ./cmd/loki` command, it creates file `/loki.exe`, but .gitignore has only `/loki`. Same with `promtail.exe`, `logcli.exe` and `loki-canary.exe`. Also it is safe to assume that no `.exe` files should be tracked by git in Loki.	5 years ago
Owen Diehl	2a596a77d7	Lambda-Promtail (#2282 ) * specs out lambda-promtail * init fn * udpates readme/template * lambda-promtail docs * lambda promtail includes source timestamp and uses context * non markdown links * new doc structure	5 years ago
Robby Milo	5a8bc848db	Sync docs to website (#2378 ) * update docs location * add workflow * update docs content * update content * cleanup	5 years ago
Matt Mendick	0adad827a9	Codecov circleci test [WIP] (#1600 ) * Trying to get circle to run some tests/coverage * Trying again * Misspelling * Adding codecov.yml Adding codecov.yml with defaults, and adding in ignoring protobuf generated files * Adding yacc generated files to codecov ignore Adding yacc generated files to codecov ignore * Newline? Newline?	5 years ago
Joe Elliott	fc6c5c063f	dep => go mod (#1062 ) * go mod files added. dep removed Signed-off-by: Joe Elliott <number101010@gmail.com> * Magically got prometheus version to stick Signed-off-by: Joe Elliott <number101010@gmail.com> * Cortex updated and prometheus updated Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated prometheus dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added additional deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Added two replaces from Gopkg.toml. Tests passing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added deps Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated build image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golangci-lint to use vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod makefile step and referenced in drone and circle ci configs Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated loki-build-image to 1.12 Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed linting error. Force go111module on for linting Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod vendor Signed-off-by: Joe Elliott <number101010@gmail.com> * Forced the inclusion of modtimevfs Signed-off-by: Joe Elliott <number101010@gmail.com> * Pin client-go due to issue with v12 go.mod Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Added check-mod to drone Signed-off-by: Joe Elliott <number101010@gmail.com> * Re-readded correct golang client Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy Signed-off-by: Joe Elliott <number101010@gmail.com> * Pinned golang/x/net to avoid proxy errors Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed check-mod from all. Not necessary for tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go.sum to match new pin Signed-off-by: Joe Elliott <number101010@gmail.com> * Upgraded proto to match build image Signed-off-by: Joe Elliott <number101010@gmail.com> * Force check-mod to wait til after test and lint are successful Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to go builds to force usage of vendored dependencies Signed-off-by: Joe Elliott <number101010@gmail.com> * Turn on gomodules on all builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Added mod vendor to go builds to force usage of vendored dependencies" This reverts commit 65865a24c9a23133e0fa52942f2828ead7c22147. * Moved builds out of the gopath to enforce vendor usage Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "Turn on gomodules on all builds" This reverts commit b5847f0158e928e935e0b3c1b1d4eaba840ca3dc. * Explicitly choose build image for docker driver Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated golang.org/x/sys to match prometheus's version to fix promtail windows compilation Signed-off-by: Joe Elliott <number101010@gmail.com> * Added fluentbit dependency Signed-off-by: Joe Elliott <number101010@gmail.com> * Added dependency management clause Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated go version in contributing Signed-off-by: Joe Elliott <number101010@gmail.com> * Added phony makefile targets Signed-off-by: Joe Elliott <number101010@gmail.com> * Circle CI is increasingly failing linting Signed-off-by: Joe Elliott <number101010@gmail.com> * Force vendored deps on builds Signed-off-by: Joe Elliott <number101010@gmail.com> * Added logcli to gitignore Signed-off-by: Joe Elliott <number101010@gmail.com> * Reduced concurrency to help CircleCI Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved drone builds out of GOPATH to force use of go module Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod vendor to prevent go clean from redownloading all packages Signed-off-by: Joe Elliott <number101010@gmail.com> * Added to test as well Signed-off-by: Joe Elliott <number101010@gmail.com> * Added mod-vendor to final go commands Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved fluent-bit out of GO_PATH to force go modules Signed-off-by: Joe Elliott <number101010@gmail.com> * Pass mod vendor to the last holdout: go generate Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated ugorji/go to 1.17 to avoid version regression Signed-off-by: Joe Elliott <number101010@gmail.com> * go mod tidy and go mod vendor for ugorji/go Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Hiroshi Hatake	0cfa33d90d	fluent-bit shared object go plugin (#847 ) * Import fluent-bit-go-loki plugin sources * Add fluent-bit-go package constraint * Ensure dependencies with dep ``` $ dep ensure ``` * Update .gitignore * Move fluent-bit-go-loki to fluent-bit/fluent-bit-go-loki directory * Add documentation for fluent-bit-go-loki * Fix for lint * Fix for goimports errors * Display fluent-bit-go-loki plugin version * Use more descriptive description for fluent-bit Loki plugin * Remove needless newlines * Rectify fluent-bit-go-loki makefile Remove needless parts and rename meaningless target * Use more descriptive description in README * refactor(fluent-bit): move make targets into root Makefile * Organize imports * Reorder imports with goimports $ goimports -w out_loki.go * Remove needless LICENSE file Because it is the same for Loki repository. * Rely on Loki client retry mechanism * Make descriptions more clearly * Change default unit * BatchWait: msec -> sec * BatchSize: KiB -> Byte * Use logql to parse Loki native labels format * Use logger instead of fmt.Printf * Update documentation to reflect recent changes * Use prometheus version module to print version * Dump error message with logger * Use DYN_GO_FLAGS instead of custom FLAGS for fluent-bit-plugin * Support RemoveKeys parameter * use logql.ParseMatchers() * refactor fluent-bit go plugin * add ci step for docker image * fix lint issues * fix a failing url test * fix a failing url test * add build images for drone ci * Adding labelmap file to complex record to labels Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add helm chart and fix a bug with labelmap * New push path * dep sync * Move fluent-bit folder and use jsonnet for drone * fix build * Adds more documentation and include a basic config in the container Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * remove binaries * Some readme grammar fixes * docs: really nitpicky grammar change in clients README * Add helm documentation * fixes bad config * bump chart * fixes .gitignore	6 years ago
sh0rez	f504de8013	chore(packaging): Release pipeline (#840 ) This PR sets up an automated release pipeline to cross compile build artifacts and publish them to github. mitchellh/gox is used for building, ghr for publishing	6 years ago
Edward Welch	e6f431ad46	finishing up loki-canary move into loki repo	6 years ago
Edward Welch	cd5368a44d	preparing for move into loki repo	6 years ago
Cyril Tovena	2909fe54e1	Docker Logging Driver (#663 ) * adds first version of docker driver * without logrus and fixes some linter issue * fix the driver and start a build system * adds swarm label discovery * Add documentation and more targets * make the linter happy ❤️ * indent config.json * with circleci steps for master and branch * Review Feebacks	6 years ago
Edward Welch	5fe4ccbe1d	adding namespace to ksonnet config, moved images into a config file	6 years ago
Edward Welch	34bb9db758	using an environment variable passed into an arg instead of reading the pod name from a file and the downardApi added a makefile	6 years ago
Edward Welch	8340232b04	initial checkin mostly working, need to extract out the config needs a proper ksonnet or helm deployment or both needs a makefile	6 years ago
Edward Welch	916124b9ea	Adding `make debug` support to build debug binaries and debug containers which wrap the binary with delve and allow for remote debugging	6 years ago
Steven Sheehy	20d6c196b5	Separate loki & promtail charts Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	6 years ago
Steven Sheehy	625bf19ce4	Setup Helm chart repository Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	6 years ago
David Kaltschmidt	c4bddc73ab	gitignore logcli	6 years ago
Steven Sheehy	db60080253	Add readiness probe to Helm chart and cleanup Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	6 years ago
Ed	19260d4425	Adding a configurable timeout for calls to the loki server. (#368 ) Adding a configurable timeout for calls to the loki server. Updating promtail test to set sane defaults for all configurations and then override for testing. Removing .idea folder from .gitignore, we should encourage people (and myself) to use global ignore for IDE configs	6 years ago
Ed	f6d0857ddf	Adding a sync() function to filetarget (#293 ) * Adding a sync() function to filetarget which runs on a timer to catch any events missed by fsnotify * moving where we mark the final position to avoid a race condition exposed in testing, we want to stop the tailer before shutting down our tailer loop * Fix changes from PR and refactor to remove circular imports and make config consistent * Adding an "integration" type unit test which tests promtail as a whole * fixing a bunch of imports I munged moved tailer to its own file refactoring filetarget to make synclogic easier to follow and reused as much as possible	6 years ago
Tom Wilkie	5f73598389	Rename Tempo to Loki. (#36 ) * Rename Tempo to Loki. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Use new build image, don't delete generated files on clean. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	f6d5ebf513	Document how to get promtail running locally. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	1fef92bc17	Lint stuff (#27 ) * Check in generated files, to make getting started easier. Add gometalinter, and a lint check for changes to the generated files. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Make lint pass? Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	da18cb343a	Add option to disable auth when running locally. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	631b45d86f	Add commands for building and running tempo locally. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	0182f845e8	Make it a single binary. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago

1 2

59 Commits (8b4dda786b4d734f34cc8fe56543870a5f080a41)