mirror of https://github.com/grafana/loki
Tag:
Branch:
Tree:
bfa2a1df65
18060-enhance-log-level-detection-for-unstructured-logs
2.9.15RN
2.9.x-bump-golang
2005.12.08-limits
2023-03-16-new-query-limits
2025.08.04_metricsQuery
2025.12.02_troubleshooting-ingest
56quarters/vendor-updates
7139-json-properties-in-log-line-is-not-sorted
Alex3k-patch-1
Alex3k-patch-2
Alex3k-patch-3
Alex3k-patch-5
Alex3k-patch-6
JStickler-patch-1
RN-35
SE_16669
TR-query4
TR-query5
add-10055-to-release-notes
add-10193-to-release-notes
add-10213-to-release-notes
add-10281-to-release-notes
add-10417-to-release-notes
add-12403-to-release-notes
add-9063-to-release-notes
add-9484-to-release-notes
add-9568-to-release-notes
add-9704-to-release-notes
add-9857-to-release-notes
add-blooms-flag-to-index-lookup
add-bucket-name-to-objclient-metric
add-claude-github-actions-1755010590637
add-claude-github-actions-1755010669077
add-containerSecurityContext-to-statefulset-backend-sidecar
add-max-flushes-retries
add-page-count-to-dataobj-inspect
add-per-scope-limits
add-time-snap-middleware
add_metrics_namespace_setting
add_series_chunk_filter_test
add_vector_to_lokitool_tests
added-hints-to-try-explore-logs
adeverteuil-patch-1
aengusrooneygrafana-update-doc-pack-md
akhilanarayanan/dountilquorum
akhilanarayanan/query-escaping
akhilanarayanan/replace-do-with-dountilquorum2
alt-err-prop
andrewthomas92-patch-1
andrii/fix_default_value_for_sasl_auth
arrow-engine/stitch-store-and-engine
arve/remove_global_name_validation
ashwanth/remove-unordered-writes-config
ashwanth/restructure-query-section
ashwanth/skip-tsdb-load-on-err
attempt-count-streams-per-query
auto-remove-unhealthy-distributors
auto-triager
automated-helm-chart-update/2023-02-01-05-30-47
automated-helm-chart-update/2023-04-05-19-46-39
automated-helm-chart-update/2023-04-24-20-56-21
automated-helm-chart-update/2023-04-24-22-40-04
automated-helm-chart-update/2023-09-07-18-09-02
automated-helm-chart-update/2023-09-14-16-23-44
automated-helm-chart-update/2023-10-16-14-20-07
automated-helm-chart-update/2023-10-18-10-10-52
automated-helm-chart-update/2023-10-18-13-14-43
automated-helm-chart-update/2024-01-24-16-05-59
automated-helm-chart-update/2024-04-08-19-24-50
aws-bug
azuretaketwo
backport-10090-to-k160
backport-10101-to-release-2.9.x
backport-10221-to-release-2.8.x
backport-10318-to-k163
backport-10687-to-release-2.9.x
backport-11251-to-k175
backport-11827-to-k186
backport-13116-to-release-3.2.x
backport-13116-to-release-3.3.x
backport-13225-to-main
backport-14221-to-release-3.2.x
backport-14780-to-release-3.2.x
backport-15483-to-release-3.3.x
backport-16045-to-k239
backport-16203-to-k242
backport-16954-to-main
backport-17054-to-k249
backport-17129-to-k277
backport-18538-to-release-3.4.x
backport-8893-to-release-2.6.x
backport-8971-to-release-2.7.x
backport-9176-to-release-2.8.x
backport-9757-to-release-2.8.x
backport-9978-to-k158
backport-9978-to-k159
backport-b57d260dd
benclive/add-alternative-int64-encoder-dataobj
benclive/add-index-ptr-to-inspect
benclive/add-unique-parsed-keys-to-pointers
benclive/customize-client-params
benclive/dont-apply-limit-under-filter-exp
benclive/exp-query-comparator
benclive/fix-index-entries-for-single-tenant-objects
benclive/fix-some-data-races
benclive/hedge-requests-exp
benclive/implement-strict-logfmt-parsing-main
benclive/index-testing-wip
benclive/multi-tenant-toc
benclive/parquet-bench-experiment
benclive/query-fixes
benclive/respect-encoding-flags-compat
benclive/scan-kafka
benclive/serialize-stats-on-wire
benclive/sort-descriptions-before-return
benclive/sort-descriptors
benclive/test-fix-empty-matchers
benton/loki-mixin-updates
benton/loki-mixin-v2
better-batches
better-log-line-level-detection
blockbuilder-timespan
blockscheduler-track-commits
bound-parallelism-slicefor
buffered-kafka-reads
build-samples-based-on-num-chunks-size
callum-builder-basemap-lock
callum-explainer-hack
callum-hackathon-explainer
callum-iterator-arrow-record
callum-k136-jsonnet-fix
callum-lambda-promtail-test
callum-parallelize-first-last
callum-pipeline-sanitize-sm-values
callum-prob-step-eval
callum-quantile-inner-child
callum-query-limits-validation
callum-querylimit-pointers
callum-remove-epool
callum-ruler-local-warn
callum-s3-prefix-metric
callum-shard-last
callum-snappy-exp
callum-stream_limit-insights
callum-track-max-labels
chaudum/batch-log-enqueue-dequeue
chaudum/benchmark-reassign-queriers
chaudum/bloomfilter-e2e-parallel-requests
chaudum/bloomfilter-jsonnet
chaudum/bloomgateway-client-tracing
chaudum/bloomgateway-testing
chaudum/bloomstore-cache-test
chaudum/bloomstore-fetch-blocks
chaudum/bump-helm-4.4.3
chaudum/canary-actor
chaudum/chaudum/query-execution-pull-iterators
chaudum/chunk-compression-read-benchmark
chaudum/cleanup-ingester
chaudum/cmp-fix
chaudum/compactor-list-objects
chaudum/cri-config
chaudum/day-chunks-iter-test
chaudum/debug-compat
chaudum/debug-skipped
chaudum/distributor-healthcheck
chaudum/dockerfmt
chaudum/fix-flaky-multitenant-e2e-test
chaudum/fix-generate-benchmark-data-for-new-indexes
chaudum/fix-max-query-range-limit
chaudum/fix-predicate-from-matcher
chaudum/fixed-size-memory-ringbuffer
chaudum/generic-dataobj-section
chaudum/grpc-transport
chaudum/hackathon-analyze-pipelines
chaudum/hackathon-analyze-pipelines-v2
chaudum/hackathon-analyze-pipelines-v3
chaudum/hashwithoutlabels
chaudum/helm-remove-image-override-for-gel
chaudum/improve-git-fetch-makefile
chaudum/improve-timestamp-parsing
chaudum/index-gateway-instrumentation-k204
chaudum/instant-query-benchmarks
chaudum/integration-test-startup-timeout
chaudum/k204-index-gateway
chaudum/k259
chaudum/linked-map
chaudum/local-index-query
chaudum/logcli-load-multiple-schemaconfig
chaudum/loki-query-engine-ui
chaudum/make-bloomfilter-task-cancelable
chaudum/metastore-caching
chaudum/native-docker-builds
chaudum/new-engine-sharding
chaudum/page-cache
chaudum/physical-plan-optimizer-visitor-pattern
chaudum/querier-worker-cpu-affinity
chaudum/query-execution
chaudum/query-executor-4
chaudum/query-skip-factor
chaudum/remove-bigtable-backend
chaudum/remove-boltdb-backend
chaudum/remove-cassandra-backend
chaudum/remove-deprecated-storage-backends
chaudum/remove-dynamodb-backend
chaudum/rewrite-runtime-config
chaudum/run-query-engine-in-frontend
chaudum/seek-panic
chaudum/shard-by-section-k258
chaudum/shared-compressors
chaudum/sort-by-timestamp
chaudum/sort-dataobj-sections-by-timestamp-desc-parallelise
chaudum/syslog-udp-cleanup-idle-streams
chaudum/topk-popall
check-inverse-postings
cherrypick-9484-k151
chunk-inspect-read-corrupt
chunk-query
chunks-inspect-v4-read-corrupt
chunks_compaction_research
chunkv5
cle_updates
cleanup-campsite/removing-deprecations
cleanup-migrate
codeowners-mixins-20240925
context-cause-usage
correct-kafka-metric-names
correctly-propagates-ctx
crypto
crypto2
cursor/analyze-index-gateway-performance-changes-between-branches-4f9d
cursor/audit-license-change-commits-de87
cursor/compare-grafana-database-architectures-6434
cursor/compare-grafana-database-architectures-9478
cursor/compare-grafana-database-architectures-bc02
cursor/create-lightweight-loki-client-module-74c9
cursor/create-lightweight-loki-client-module-e1fe
cursor/extract-loki-client-utilities-to-new-module-2891
cursor/investigate-recent-api-response-changes-82fb
cursor/update-loki-opentelemetry-dependencies-efe4
custom-headers
dannykopping/groupcache-instrument
dannykopping/memcached-slab-allocator
dannykopping/remove-cache-stats
danstadler-pdx-patch-1
danstadler-pdx-patch-2
data-race-fix-01
dataobj
dataobj-compression-ratio-and-final-size
dataobj-comsumer-metastore-orig
dataobj-log-batches
dataobj-logs-sort
dataobj-logs-sortorder
dataobj-querier-logger
dataobj-reader-stats
dataobj-shard-debug
dataobj-store-sort-order
dataset-reader-fill-fail
debug-bloomgateway
dedup-only-partitions
del-ashwanth-custom
dependabot/go_modules/golang.org/x/crypto-0.45.0
dependabot/go_modules/operator/api/loki/golang.org/x/net-0.38.0
dependabot/go_modules/operator/golang.org/x/crypto-0.45.0
deprecatable-metrics-example
deps-update/main-alpine
deps-update/main-fluentfluent-bit
deps-update/main-github.comapachearrow-gov18
deps-update/main-github.comawsaws-sdk-go-v2config
deps-update/main-github.comawsaws-sdk-go-v2servicedynamodb
deps-update/main-github.comawsaws-sdk-go-v2services3
deps-update/main-github.combaidubcebce-sdk-go
deps-update/main-github.comgrafanalokiv3
deps-update/main-github.cominfluxdatatelegraf
deps-update/main-github.comparquet-goparquet-go
deps-update/main-github.compierreclz4v4
deps-update/main-go.opentelemetry.iocontribinstrumentationgoogle.golang.orggrpcotelgrpc
deps-update/main-go.opentelemetry.iocontribinstrumentationnethttphttptraceotelhttptrace
deps-update/main-go.opentelemetry.iocontribinstrumentationnethttpotelhttp
deps-update/main-go.opentelemetry.iootel
deps-update/main-go.opentelemetry.iootelsdk
deps-update/main-go.opentelemetry.iooteltrace
deps-update/main-golang.orgxcrypto
deps-update/main-golang.orgxnet
deps-update/main-golang.orgxtext
deps-update/main-google.golang.orgprotobuf
deps-update/main-hashicorpaws
deps-update/main-hashicorpgoogle
deps-update/main-https-github.comgrafanaloki.git
deps-update/main-rollout-operator
detected-labels-add-limits-param
detected-labels-from-store
detected-labels-minor-enhancements
dev-rel-workshop
dfinnegan-fgh-patch-1
digitalemil-patch-1
digitalemil-patch-2
digitalemil-patch-3
digitalemil-patch-4
dimitarvdimitrov-patch-1
dirlldown-config-endpoint__devenv
distributed-helm-chart
distributed-helm-demo
distributors-exp-avg
do-not-retry-enforced-labels-error
do-until-quorom-wip
doanbutar-patch-1
doanbutar-patch-2
docs-ipv6
docs-nvdh-log-queries
dodson/admonitions
dont-log-every-indexset-call-
dont-sync-old-tables
ej25a-patch-1
emit-events-without-debuggnig
enable-hedging-on-ingester-requests
enable-limitedpusherrorslogging-by-default
enable-stream-sharding
enforce-sharding-of-approx-topk-queries
engine-batchsize-metric
engine-track-task-timings
exceeds-rate-limit-check
executor-lazypipeline
explore-logs-fallback-query-path
faster-cleanupexpired
faster-truncate-log-lines
fastest-path-return
fcjack/image-workflows
fcjack/test-agent-lables
feat-multi-tenant-topics
feat/drain-format
feat/pattern-pattern-mining
feat/syslog-rfc3164-defaultyear
feat/usage-tracker
filter-node-pushdown
fix-2.8-references
fix-discarded-otlp-volume
fix-headers
fix-helm-bucket-issue
fix-helm-enterprise-values
fix-helmchart
fix-igw-job
fix-image-tag-script
fix-legacy-panels
fix-object-storage
fix-orphan-spans
fix-otlp-exporter-metric
fix-promtail-cves
fix-release-lib-shellcheck
fix-stream-generator-locks
fix/pattern-merge
fix_more_dashboards
fix_windowsserver_version
fmt-jsonnet-fix
force-loki-helm-publish
get-marked-for-deletions
gh-action-labeler-fix
gh-readonly-queue/main/pr-11793-215b5fd2fd71574e454529b1b620a295f1323dac
goldfish-rds-args-k279
goldfish-ui-improvements-2
goldfish-ui-k263
grafana-dylan-patch-1
grobinson/add-kafkav2
grobinson/fix-race-condition-in-tracker
grobinson/rate-metric-to-compare
grobinson/segmentation-key-demo
grobinson/track-consumption-lag-offsets
grobinson/use-local-distributor-rate-store-segmentation-keys
grobinson/wip-flush-in-background
groupcache
gtk-grafana/drilldown-config-endpoint__devenv
guard-againts-non-scheduler-request
guard-ingester-detected-field-errors
hackathon-2023-08-events-in-graphite-proxy
hackathon/demo
hackathon/hackathon-2023-12-arrow-engine
handle-errors-per-category
hedge-index-gateway
hedge-index-gateway-220
helm-5.47.3
helm-5.48
helm-chart-tagged-6.20.0
helm-chart-tagged-6.26.0
helm-chart-tagged-6.27.0
helm-chart-tagged-6.28.0
helm-chart-tagged-6.30.0
helm-chart-tagged-6.31.0
helm-chart-tagged-6.32.0
helm-chart-tagged-6.47.0
helm-chart-weekly-6.24.0-weekly.233
helm-chart-weekly-6.25.0-weekly.234
helm-chart-weekly-6.25.0-weekly.235
helm-chart-weekly-6.25.0-weekly.236
helm-chart-weekly-6.25.0-weekly.237
helm-chart-weekly-6.26.0
helm-chart-weekly-6.26.0-weekly.238
helm-chart-weekly-6.26.0-weekly.239
helm-chart-weekly-6.26.0-weekly.240
helm-chart-weekly-6.26.0-weekly.241
helm-chart-weekly-6.28.0-weekly.242
helm-chart-weekly-6.28.0-weekly.243
helm-chart-weekly-6.28.0-weekly.244
helm-chart-weekly-6.29.0-weekly.245
helm-chart-weekly-6.29.0-weekly.246
helm-chart-weekly-6.29.0-weekly.247
helm-chart-weekly-6.30.0
helm-chart-weekly-6.47.0
helm-chart-weekly-6.50.0
helm-loki-values-backend-target
high-load-gateway
hot-fix-breaking-change
ignore-yaml-errors
improve-benchtest-dataset
improve-cleanup-stats
improve-distributor-latency
index-gateways/reduce-goroutines
index-stats
ingest-limits-reader-client-optimization
ingest-pipelines
inline-tsdb-on-cache
integrate-laser
intentional-failure
is-this-qfs-cure
ivkalita/metastore-distributed-plus-scheduler
ivkalita/metastore-merge-nodes
jdb/2022-10-enterprise-logs-content-reuse
jdb/2023-03-update-doc.mk
jdb/2025-05/add-docs-license
jsonnet-update/2023-01-31-10-09-02
k100
k101
k102
k103
k104
k105
k106
k107
k108
k109
k110
k111
k112
k113
k114
k115
k116
k117
k118
k119
k12
k120
k121
k122
k123
k124
k125
k126
k127
k128
k129
k13
k130
k131
k131-no-validate-matchers-labels
k132
k133
k135
k135-sharding-hotfix
k136
k137
k138
k139
k14
k140
k141
k142
k143
k144
k145
k146
k146-with-chunk-logging
k147
k148
k149
k15
k150
k150-merge-itr-fix
k151
k152
k153
k154
k155
k156
k157
k158
k159
k16
k160
k161
k162
k163
k164
k165
k166
k167
k168
k168-ewelch-concurrency-limits
k169
k17
k170
k171
k171-with-retry
k172
k173
k174
k174-fixes2
k175
k176
k177
k178
k179
k18
k180
k181
k182
k183
k183-quantile-patch
k184
k185
k185-fix-previous-tsdb
k186
k187
k188
k189
k19
k190
k191
k192
k193
k194
k195
k195-backup
k196
k197
k198
k199
k199-debug
k20
k200
k201
k202
k203
k203-with-samples
k204
k204-separate-download
k205
k205-with-samples
k206
k207
k207-ingester-profiling-2
k208
k209
k209-ewelch-idx-gateway-hedging
k21
k210
k210-ewelch-idx-gateway-hedge
k210-ewelch-shard-limited
k211
k211-ewelch-congestion-control
k211-ewelch-datasample
k211-ewelch-test-frontend-changes
k212
k213
k213-ewelch
k214
k215
k216
k217
k217-alloy-v1.7-fork
k217-without-promlog
k218
k219
k22
k220
k220-index-sync
k220-move-detected-fields-logic-to-qf
k220-with-detected-fields-guard
k221
k221-index-sync-fixes
k221-with-stream-logging
k222
k222-shard-volume-queries
k228
k229
k23
k230
k231
k232
k233
k234
k235
k236
k236-with-agg-metric-payload-fix
k237
k238
k239
k24
k240
k241
k242
k243
k244
k245
k246
k246-with-per-tenant-ruler-wal-replay
k247
k248
k248-distributor-lvl-detection
k248-level-detection-debugging
k248-levels-as-index
k249
k25
k250
k251
k252
k253
k254
k255
k256
k256-ewelch-distributor
k257
k257-ewelch-max-query-series
k258
k259
k26
k260
k261
k262
k262-limit-patterns-2
k262-limit-patterns-test
k263
k263-log-labels-string
k264
k265
k266
k267
k268
k269
k27
k270
k271
k271-pattern-volume
k272
k272-ewelch
k273
k274
k274-7a1d4f729e
k275
k276
k276-dataobj-sort
k277
k278
k278-ewelch
k278-goldfish-storage
k279
k28
k280
k281
k282
k283
k284
k284-ewelch
k284-metastore-improv
k285
k286
k287
k288
k29
k30
k31
k32
k33
k34
k35
k36
k37
k38
k39
k40
k41
k42
k43
k44
k45
k46
k47
k48
k49
k50
k51
k52
k53
k54
k55
k56
k57
k58
k59
k60
k61
k62
k63
k64
k65
k66
k67
k68
k69
k70
k71
k72
k73
k74
k75
k76
k77
k78
k79
k80
k81
k82
k83
k84
k85
k86
k87
k88
k89
k90
k91
k92
k93
k94
k95
k96
k97
k98
k99
kadjoudi-patch-1
kafka-usage-wip
kafka-wal-block
karsten/dedup-overlapping-chunks
karsten/first-over-time
karsten/fix-grpc-error
karsten/protos-query-request
karsten/test-ops
kaviraj/changelog-logql-bug
kaviraj/memcached-backup-tmp
kaviraj/single-gomod
kavirajk/backport-10319-release-2.9.x
kavirajk/bug-fix-memcached-multi-fetch
kavirajk/cache-instant-queries
kavirajk/cache-test
kavirajk/experiment-instant-query-bug
kavirajk/fix-engine-literalevaluator
kavirajk/linefilte-path-on-top-of-k196
kavirajk/memcache-cancellation-bug-fix
kavirajk/metadata-cache-with-k183
kavirajk/promtail-use-inotify
kavirajk/script-to-update-example
kavirajk/update-go-version-gomod
kavirajk/upgrade-prometheus-0.46
kavirajk/url-encode-aws-url
label-filter-predicate-pushdown
lambda-promtail-generic-s3
leizor/latest-produce-ts
leizor/rate-limiter-hack
leizor/rate-limiter-hack-clean
limit-patterns-k263
limit-streams-chunks-subquery
logcli-copy-chunks
logcli_object_store_failure_logging
logfmt-unwrap-queries
logql-correctness-memlimit
loki-bench-tool
loki-config-json
loki-mixin-parallel-read-path
loki-streaming-query-api
lru-symbols-cache
lru-symbols-cache-w-conn-limits
main
make-section-filter-tenant-aware
map-streams-to-ingestion-scope
marinnedea-patch-1
mdc-loki-helm
mdsgrafana-patch-1
meher/handle-wal-corruption-on-startup
meher/log-partition-ring-cache-map-size
meher/parition-ring-manual-disable
meher/query-lab
meher/regexp-engine-v2
merge-pipeline-parallelism
mess-with-multiplegrpcconfigs
meta-monitoring-v2-p2
metadata-decoder-corrections
metastore-bootstrap
metastore-experiments
metric-query-testing
metricq-benchmark-delete-later
more-date-functions
more-details-tracing-for-distributors
more-release-testing
multi-zone-topology-support
new-index-spans
new-tests
no-extents-no-problem
operator-loki-v3
optimize-otlp-efficiency
otlp-severity-detection
owen-d/fix/nil-ptr-due-to-empty-resp
pablo/lambda-promtail-event-bridge-setup
pablo/promtail-wal-support
pablo/refactor-client-manager
pablo/refactor-http-targets
panic-if-builder-fails-to-init
panic_query_frontend_test
parallelise-merge-tables
parser-hints/bug
paul1r/ingester_startup
paul1r/poc_partition_inactive
paul1r/republish_lambda_promtail
periklis/k280-max-query-bytes-read
periklis/max-query-bytes-read
pooling-decode-buffers-dataobj
poyzannur/add-pdb-idx-gws
poyzannur/fix-blooms-checksum-bug
poyzannur/fix-compactor-starting-indexshipper-in-RW-mode
poyzannur/fix-errors-introduced-by-10748
poyzannur/fix-flaky-test
poyzans-query
pr-19328
pr-logfmt-1-foundation
pr-logfmt-2-planner
pr-logfmt-3-tokenizer
pr-logfmt-4-executor
pr_11086
pre-build-predicates
prepare-2.8-changelog
promtail-go-gelf
ptodev/reset-promtail-metrics-archive-23-april-2024
ptodev/update-win-eventlog
pub-sub-cancel
query-limits-validation
query-splitting-api
query-timestamp-validation
ratestore-standalone
rbrady/16330-fix-rolebinding-provisioner
read-corrupt-blocks
read-path-improvement-wal
reenable-ipv6-for-memberlist
refactor-extractors-multiple-samples-2
release-2.0.1
release-2.2
release-2.2.1
release-2.3
release-2.4
release-2.5.x
release-2.6.x
release-2.7.x
release-2.8.x
release-2.8.x-fix-failing-test
release-2.9.x
release-3.0.x
release-3.1.x
release-3.2.x
release-3.3.x
release-3.4.x
release-3.5.x
release-3.6.x
release-notes-appender
release-please--branches--add-major-release-workflow
release-please--branches--fix-vuln-scanning
release-please--branches--k195
release-please--branches--k196
release-please--branches--k197
release-please--branches--k198
release-please--branches--k199
release-please--branches--k200
release-please--branches--k201
release-please--branches--k202
release-please--branches--k203
release-please--branches--k204
release-please--branches--k205
release-please--branches--k206
release-please--branches--k208
release-please--branches--k209
release-please--branches--k210
release-please--branches--k211
release-please--branches--k212
release-please--branches--k215
release-please--branches--k216
release-please--branches--k221
release-please--branches--k222
release-please--branches--k228
release-please--branches--k234
release-please--branches--k235
release-please--branches--k236
release-please--branches--k237
release-please--branches--k238
release-please--branches--k239
release-please--branches--k240
release-please--branches--k241
release-please--branches--k242
release-please--branches--k243
release-please--branches--k244
release-please--branches--k246
release-please--branches--k247
release-please--branches--k249
release-please--branches--k250
release-please--branches--k251
release-please--branches--k253
release-please--branches--k254
release-please--branches--k255
release-please--branches--k256
release-please--branches--k257
release-please--branches--k260
release-please--branches--k261
release-please--branches--k262
release-please--branches--k263
release-please--branches--k264
release-please--branches--k265
release-please--branches--k266
release-please--branches--k267
release-please--branches--k268
release-please--branches--k270
release-please--branches--k271
release-please--branches--k272
release-please--branches--k273
release-please--branches--k278
release-please--branches--main
release-please--branches--main--components--operator
release-please--branches--release-2.9.x
release-please--branches--release-3.0.x
release-please--branches--release-3.1.x
release-please--branches--release-3.2.x
release-please--branches--release-3.3.x
release-please--branches--update-release-pipeline
remove-early-eof
remove-loki-ui
remove-override
remove_lokitool_binary
retry-limits-middleware
reuse-server-index
revert-7179-azure_service_principal_auth
revert-8662
revert-map-pooling
rfratto/protobuf-physical-plans
rgnvldr-patch-1
rk/update-helm-docs
salvacorts/2.9.12/fix-vulns
salvacorts/backport-3.4.x
salvacorts/compator-deletes-acache
salvacorts/debugging-forward-log-queries
salvacorts/log-small-results-cache
salvacorts/query-bucket-rate-limit
samu6851-patch-1
samu6851-patch-2
scheduler-assignlock-granular
scheduler_contentions
scope-usage
shantanu/add-to-release-notes
shantanu/dedupe-with-sm
shantanu/fix-scalar-timestamp
shantanu/otlp-push-optimizations
shantanu/otlp_dots_support
shantanu/prom-upgrade-and-fixes
shantanu/remove-ruler-configs
shantanu/utf8-queries-no-quotes
shard-parsing
shard-volume-queries
shipper/skip-notready-on-sync
simulate-retention-endpoint
singleflight
sjwlabelfmt
sjwnodeproto
sjwpoolmemory
skip-worktrees
snyk-monitor-workflow
sp/logged_trace_id
spiridonov-bifunc-alloc
spiridonov-blast-test
spiridonov-parity-hacks-1
spiridonov-style-chore
split-rules-into-more-groups
split-tests-by-package
split-with-header
steven_2_8_docs
stop-using-retry-flag
store-aggregated-metrics-in-loki
store-aggregated-metrics-in-loki-3
store-calls-stat
stream-limit-fixes
stripe-lock-ctx-cancelation
structured-metadata-indexing
structured-metadata-push-down-3
svennergr-patch-1
svennergr/structured-metadata-api
tch/bestBranchEvverrrrrrrrrr
temp-fluentbit-change
temp-proto-fix
test-docker-plugin-publish
test-failcheck
test-gateway
test-helm-release
test-query-limits-fixes
test-release
test-workflow
test_PR
test_branch
testing-drain-params
testing-drain-params-2
thor-ingestion
thor-query-parser-correctness
tpatterson/cache-json-label-values
tpatterson/chunk-iterator
tpatterson/expose-partition-ring
tpatterson/generate-drone-yaml
tpatterson/label-matcher-optimizations
tpatterson/reporder-filters
tpatterson/revert-async-store-change
tpatterson/size-based-compaction-with-latest
tpatterson/space-compaction
tpatterson/stats-estimate
trace-labels-in-distributor
transform_mixin
trevorwhitney/detect-only-no-parser
trevorwhitney/how-to-make-a-pr
trevorwhitney/index-stats-perf-improvement
trevorwhitney/logcli-client-test
trevorwhitney/refactor-nix-folder
trevorwhitney/respect-tsdb-version-in-compactor
trevorwhitney/series-volume-fix
trevorwhitney/structured-metadata-push-down
trevorwhitney/upgrade-dskit
trevorwhitney/use-tsdb-version-from-schema-config
trevorwhitney/volume-memory-fix-k160
trigger-ci
try-new-span-chagnes
try-reverting-pr9404
tsdb-benchmark-setup
tulmah-patch-1
twhitney/query-tee-enable-racing
twhitney/refactor-parse-2
twhitney/structured-metadata-push-down
twhitney/thor-unwrap-2
undelete
update-docs-Running-Promtail-on-AWS-EC2-tutorial
update-workflows
updateCHANGELOG
upgrade-golang-jwt-2.9
upgrade33
usage-poc-combined
use-worker-pool-for-kafka-push
use-worker-pool-kafka-push
use_constant_for_loki_prefix
use_go_120_6
validate-retention-api
wip-stringlabels
wrap-downloading-file-errors
wrapped-http-codes-propagate
x160-ewelch-cache
x161-ewelch-l2-cache
x162-ewelch-memcached-connect-timeout
xcap-coverage
xcap-pipeline-inject
yinkagr-patch-1
2.8.3
helm-loki-3.0.0
helm-loki-3.0.1
helm-loki-3.0.2
helm-loki-3.0.3
helm-loki-3.0.4
helm-loki-3.0.5
helm-loki-3.0.6
helm-loki-3.0.7
helm-loki-3.0.8
helm-loki-3.0.9
helm-loki-3.1.0
helm-loki-3.10.0
helm-loki-3.2.0
helm-loki-3.2.1
helm-loki-3.2.2
helm-loki-3.3.0
helm-loki-3.3.1
helm-loki-3.3.2
helm-loki-3.3.3
helm-loki-3.3.4
helm-loki-3.4.0
helm-loki-3.4.1
helm-loki-3.4.2
helm-loki-3.4.3
helm-loki-3.5.0
helm-loki-3.6.0
helm-loki-3.6.1
helm-loki-3.7.0
helm-loki-3.8.0
helm-loki-3.8.1
helm-loki-3.8.2
helm-loki-3.9.0
helm-loki-4.0.0
helm-loki-4.1.0
helm-loki-4.10.0
helm-loki-4.2.0
helm-loki-4.3.0
helm-loki-4.4.0
helm-loki-4.4.1
helm-loki-4.4.2
helm-loki-4.5.0
helm-loki-4.5.1
helm-loki-4.6.0
helm-loki-4.6.1
helm-loki-4.6.2
helm-loki-4.7.0
helm-loki-4.8.0
helm-loki-4.9.0
helm-loki-5.0.0
helm-loki-5.1.0
helm-loki-5.10.0
helm-loki-5.11.0
helm-loki-5.12.0
helm-loki-5.13.0
helm-loki-5.14.0
helm-loki-5.14.1
helm-loki-5.15.0
helm-loki-5.17.0
helm-loki-5.18.0
helm-loki-5.18.1
helm-loki-5.19.0
helm-loki-5.2.0
helm-loki-5.20.0
helm-loki-5.21.0
helm-loki-5.22.0
helm-loki-5.22.1
helm-loki-5.22.2
helm-loki-5.23.0
helm-loki-5.23.1
helm-loki-5.24.0
helm-loki-5.25.0
helm-loki-5.26.0
helm-loki-5.27.0
helm-loki-5.28.0
helm-loki-5.29.0
helm-loki-5.3.0
helm-loki-5.3.1
helm-loki-5.30.0
helm-loki-5.31.0
helm-loki-5.32.0
helm-loki-5.33.0
helm-loki-5.34.0
helm-loki-5.35.0
helm-loki-5.36.0
helm-loki-5.36.1
helm-loki-5.36.2
helm-loki-5.36.3
helm-loki-5.37.0
helm-loki-5.38.0
helm-loki-5.39.0
helm-loki-5.4.0
helm-loki-5.40.1
helm-loki-5.41.0
helm-loki-5.41.1
helm-loki-5.41.2
helm-loki-5.41.3
helm-loki-5.41.4
helm-loki-5.41.5
helm-loki-5.41.6
helm-loki-5.41.7
helm-loki-5.41.8
helm-loki-5.41.9-distributed
helm-loki-5.41.9-distributed-rc2
helm-loki-5.42.0
helm-loki-5.42.1
helm-loki-5.42.2
helm-loki-5.42.3
helm-loki-5.43.0
helm-loki-5.43.1
helm-loki-5.43.2
helm-loki-5.43.3
helm-loki-5.43.4
helm-loki-5.43.5
helm-loki-5.43.6
helm-loki-5.43.7
helm-loki-5.44.0
helm-loki-5.44.1
helm-loki-5.44.2
helm-loki-5.44.3
helm-loki-5.44.4
helm-loki-5.45.0
helm-loki-5.46.0
helm-loki-5.47.0
helm-loki-5.47.1
helm-loki-5.47.2
helm-loki-5.48.0
helm-loki-5.5.0
helm-loki-5.5.1
helm-loki-5.5.10
helm-loki-5.5.11
helm-loki-5.5.12
helm-loki-5.5.2
helm-loki-5.5.3
helm-loki-5.5.4
helm-loki-5.5.5
helm-loki-5.5.6
helm-loki-5.5.7
helm-loki-5.5.8
helm-loki-5.5.9
helm-loki-5.6.0
helm-loki-5.6.1
helm-loki-5.6.2
helm-loki-5.6.3
helm-loki-5.6.4
helm-loki-5.7.1
helm-loki-5.8.0
helm-loki-5.8.1
helm-loki-5.8.10
helm-loki-5.8.11
helm-loki-5.8.2
helm-loki-5.8.3
helm-loki-5.8.4
helm-loki-5.8.5
helm-loki-5.8.6
helm-loki-5.8.7
helm-loki-5.8.8
helm-loki-5.8.9
helm-loki-5.9.0
helm-loki-5.9.1
helm-loki-5.9.2
helm-loki-6.0.0
helm-loki-6.1.0
helm-loki-6.10.0
helm-loki-6.10.1
helm-loki-6.10.2
helm-loki-6.11.0
helm-loki-6.12.0
helm-loki-6.15.0
helm-loki-6.16.0
helm-loki-6.18.0
helm-loki-6.19.0
helm-loki-6.19.0-weekly.227
helm-loki-6.2.0
helm-loki-6.2.1
helm-loki-6.2.2
helm-loki-6.2.3
helm-loki-6.2.4
helm-loki-6.2.5
helm-loki-6.20.0
helm-loki-6.20.0-weekly.229
helm-loki-6.21.0
helm-loki-6.22.0
helm-loki-6.22.0-weekly.230
helm-loki-6.23.0
helm-loki-6.23.0-weekly.231
helm-loki-6.24.0
helm-loki-6.24.0-weekly.232
helm-loki-6.24.1
helm-loki-6.25.0
helm-loki-6.25.1
helm-loki-6.26.0
helm-loki-6.27.0
helm-loki-6.28.0
helm-loki-6.29.0
helm-loki-6.3.0
helm-loki-6.3.1
helm-loki-6.3.2
helm-loki-6.3.3
helm-loki-6.3.4
helm-loki-6.30.0
helm-loki-6.30.1
helm-loki-6.31.0
helm-loki-6.32.0
helm-loki-6.33.0
helm-loki-6.34.0
helm-loki-6.35.0
helm-loki-6.35.1
helm-loki-6.36.0
helm-loki-6.36.1
helm-loki-6.37.0
helm-loki-6.38.0
helm-loki-6.39.0
helm-loki-6.4.0
helm-loki-6.4.1
helm-loki-6.4.2
helm-loki-6.40.0
helm-loki-6.41.0
helm-loki-6.41.1
helm-loki-6.42.0
helm-loki-6.43.0
helm-loki-6.44.0
helm-loki-6.45.0
helm-loki-6.45.1
helm-loki-6.45.2
helm-loki-6.46.0
helm-loki-6.48.0
helm-loki-6.49.0
helm-loki-6.5.0
helm-loki-6.5.1
helm-loki-6.5.2
helm-loki-6.6.0
helm-loki-6.6.1
helm-loki-6.6.2
helm-loki-6.6.3
helm-loki-6.6.4
helm-loki-6.6.5
helm-loki-6.6.6
helm-loki-6.7.0
helm-loki-6.7.1
helm-loki-6.7.2
helm-loki-6.7.3
helm-loki-6.7.4
helm-loki-6.8.0
helm-loki-6.9.0
operator/v0.4.0
operator/v0.5.0
operator/v0.6.0
operator/v0.6.1
operator/v0.6.2
operator/v0.7.0
operator/v0.7.1
operator/v0.8.0
operator/v0.9.0
pkg/logql/syntax/v0.0.1
v0.1.0
v0.2.0
v0.3.0
v0.4.0
v1.0.0
v1.0.1
v1.0.2
v1.1.0
v1.2.0
v1.3.0
v1.4.0
v1.4.1
v1.5.0
v1.6.0
v1.6.1
v2.0.0
v2.0.1
v2.1.0
v2.2.0
v2.2.1
v2.3.0
v2.4.0
v2.4.1
v2.4.2
v2.5.0
v2.6.0
v2.6.1
v2.7.0
v2.7.1
v2.7.2
v2.7.3
v2.7.4
v2.7.5
v2.7.6
v2.7.7
v2.8.0
v2.8.1
v2.8.10
v2.8.11
v2.8.2
v2.8.3
v2.8.4
v2.8.5
v2.8.6
v2.8.7
v2.8.8
v2.8.9
v2.9.0
v2.9.1
v2.9.10
v2.9.11
v2.9.12
v2.9.13
v2.9.14
v2.9.15
v2.9.16
v2.9.17
v2.9.2
v2.9.3
v2.9.4
v2.9.5
v2.9.6
v2.9.7
v2.9.8
v2.9.9
v3.0.0
v3.0.1
v3.1.0
v3.1.1
v3.1.2
v3.2.0
v3.2.1
v3.2.2
v3.3.0
v3.3.1
v3.3.2
v3.3.3
v3.3.4
v3.4.0
v3.4.1
v3.4.2
v3.4.3
v3.4.4
v3.4.5
v3.4.6
v3.5.0
v3.5.1
v3.5.2
v3.5.3
v3.5.4
v3.5.5
v3.5.6
v3.5.7
v3.5.8
v3.5.9
v3.6.0
v3.6.1
v3.6.2
v3.6.3
${ noResults }
10 Commits (bfa2a1df65e233ce99fccd01df2130f440ebd166)
| Author | SHA1 | Message | Date |
|---|---|---|---|
|
|
3277321e21
|
feat: when using sqlite for storing delete requests, use the stored completion time to reduce the requests considered for querytime filtering (#16600)
|
10 months ago |
|
|
4c88be0ef2
|
chore: update loki modules for 3.0 release (#12433)
Signed-off-by: Edward Welch <edward.welch@grafana.com> |
2 years ago |
|
|
928e3f34f8
|
Blooms: Prepare compactor refactor (#11847)
Prepares `bloomcompactor` pkg for integration with new logic, primarily removes/thins out existing code. |
2 years ago |
|
|
3a7940af1c
|
compactor" track number of times table compactions were skipped consecutively (#11604)
**What this PR does / why we need it**: Earlier we had a counter for the number of times tables were skipped by compaction due to them being locked by retention. Counter makes it hard to reliably write an alert without making it noisy. I am changing the counter to gauge which would count the number of times tables were consecutively skipped by compaction. This would make it easier to write alerts. **Checklist** - [x] Tests updated --------- Co-authored-by: Ashwanth <iamashwanth@gmail.com> |
2 years ago |
|
|
6a62b8cf42
|
compaction: Separate metrics for tracking retention and compaction (#11263)
**What this PR does / why we need it**: In PR #9884, we separated the retention loop from compaction to avoid blocking compaction for too long due to some intensive delete requests. Currently, we track retention and compaction using the same metrics. This PR adds separate metrics for monitoring retention operation. I have also updated the Retention dashboard to use the new metrics. |
2 years ago |
|
|
c716e498ad
|
compactor: do not block compation when retention is taking too long (#9884)
**What this PR does / why we need it**: Currently, we perform compaction and apply retention in the same loop. Although we have a flag for not applying retention every time we perform compaction, we still see compaction getting blocked when processing some intensive delete requests(processed while applying retention). This PR separates out the compaction and retention to run in a separate loop. I have added a table-locking feature to avoid compaction and retention from processing the same tables at a time. However, compaction and retention would treat locked tables differently, as explained below: * When compaction sees a table is locked: It would skip the table and move on to the following table. However, before skipping, it would check if the table has any uncompacted files and increment the newly added counter called `loki_compactor_skipped_compacting_locked_tables_total` to track how often we are skipping compacting tables which have uncompacted files. * When retention sees a table is locked: It would wait for the lock to be released since we can't skip any tables while processing delete requests. **Special notes for your reviewer**: * The check for tables with uncompacted files looks for count > 1 because initially, we did not support per tenant index in `boltdb-shipper`, so a table can have a single compacted multi-tenant index file. In a rare case where we have a single file which was supposed to be compacted away, it is okay to have a single uncompacted file for a while. The aim here is to not block compaction for too long in a large cell with too many uncompacted files. * Retention only works on the compacted index, so we first compact down any uncompacted files while applying retention. **Checklist** - [x] Tests updated --------- Co-authored-by: Ashwanth <iamashwanth@gmail.com> |
2 years ago |
|
|
258a8b3108
|
Update nix configuration (#8452)
Fix nix configuration (mainly failing test), and break up various binaries (ie. logcli, promtail, loki, etc.) into their own pakcages. --------- Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com> |
2 years ago |
|
|
8628b15624
|
Use metrics namespace for more metrics (#11025)
**What this PR does / why we need it**:
Use the metrics namespace setting instead of hardcoding to `cortex`.
This is a follow up to (and based on)
https://github.com/grafana/loki/pull/11014.
**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [X] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](
|
2 years ago |
|
|
a118e994d8
|
Remove shared_store and shared_store_key_prefix from shipper and compactor (#10840)
**What this PR does / why we need it**:
#### Removes `shared_store` and `shared_store_key_prefix` from index
shipper and compactor configs and their corresponding CLI flags.
- `-tsdb.shipper.shared-store`
- `-boltdb.shipper.shared-store`
- `-tsdb.shipper.shared-store.key-prefix`
- `-boltdb.shipper.shared-store.key-prefix`
- `-boltdb.shipper.compactor.shared-store`
- `-boltdb.shipper.compactor.shared-store.key-prefix`
`shared_store` has been a confusing option allowing users to easily
misconfigure Loki.
Going forward `object_store` setting in the
[period_config](https://grafana.com/docs/loki/latest/configure/#period_config)
(which already configured the store for chunks) will be used to
configure store for the index.
And the newly added `path_prefix` option under the `index` key in
`period_config` will configure the path under which index tables are
stored.
This change enforces chunks and index files for a given period reside
together in the same storage bucket. More details in the upgrade guide.
---
`-compactor.delete-request-store` has to be **explicitly configured**
going forward. Without setting this, loki wouldn't know which object
store to use for storing delete requests. Path prefix for storing
deletes is decided by `-compactor.delete-request-store.key-prefix` which
defaults to `index/`.
**Checklist**
- [X] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [X] Documentation added
- [X] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [X] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](
|
2 years ago |
|
|
91cae03fe1
|
chore: restructure compactor package (#10775)
**What this PR does / why we need it**:
This pr moves the compactor pkg from
`pkg/storage/stores/shipper/indexshipper/compactor` to `pkg/compactor` -
same level as other high level components (querier, distributor...)
table compaction code however resides in the respective store pkg closer
to other store specific code.
`pkg/storage/stores/shipper/indexshipper/boltdb/compactor/table_compactor.go`
`pkg/storage/stores/shipper/indexshipper/tsdb/compactor.go`
**Special notes for your reviewer**:
**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](
|
2 years ago |
|
|
2e775f3f44
|
chore(storage): Restructure shipper packages (#10729)
**What this PR does / why we need it**:
This PR restructures the code for the shipper component of the Loki storage layer.
New package layout:
```console
$ tree -d pkg/storage/stores/shipper
pkg/storage/stores/shipper
└── indexshipper
├── boltdb
│ └── compactor
├── compactor
│ ├── client
│ │ └── grpc
│ ├── deletion
│ ├── deletionmode
│ ├── generationnumber
│ └── retention
├── downloads
├── gatewayclient
├── index
├── indexgateway
├── storage
├── testutil
├── tsdb
│ ├── index
│ ├── testdata
│ └── testutil
├── uploads
└── util
23 directories
```
* TSDB and BoltDB specific code is under `./pkg/storage/stores/shipper/indexshipper/tsdb` and `./pkg/storage/stores/shipper/indexshipper/boltdb` respectively.
* Common code for both TSDB and BoltDB is directly under `./pkg/storage/stores/shipper/indexshipper` and subdirectories, such as `uploads/`, `downloads/`, `compactor/`, ...
**Special notes for your reviewer**:
This PR is identical to https://github.com/grafana/loki/pull/10724 except of the package `pkg/storage/stores/indexshipper` the package is `pkg/storage/stores/shipper/indexshipper` (one level deeper).
Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
|
2 years ago |
|
|
52cd0a39b8
|
compactor: multi-store support (#7447)
**What this PR does / why we need it**:
This PR adds multi-store support for compactors. Since loki allows users
to configure mutiple stores using schema_config, compactor should be
able to operate on multiple object stores that contain index. Currently,
it can perform compaction on indexes in a single store.
To maintain backward compatibility: if
`boltdb.shipper.compactor.shared-store` is set, compactor will only
operate on that store, else compactor will be initialized to operate on
all the object store indexes (`boltdb, tsdb`) defined in the schema
config.
This PR also adds a new config option to define where delete requests
are to be stored - `boltdb.shipper.compactor.delete-request-store`. If
it's not set, `boltdb.shipper.compactor.shared-store` is used for
storing them, this is to ensure no config changes are required by the
users when upgrading. Refer to
[docs/sources/upgrading/_index.md](
|
3 years ago |
|
|
f5f1753851
|
Print duration in error messages with more readable. (#8816)
**What this PR does / why we need it**:
The old error messages would print only up to hours. E.g. `169h30s`.
This change will print it as `7d1h30s`. See
[model.Duration](
|
3 years ago |
|
|
b46e392adb
|
gofmt is needed for the build (#8678)
**What this PR does / why we need it**: `gofmt` is needed for the build, `make format` fails without it. |
3 years ago |
|
|
dfff86f1cd
|
skip compacting tables that do not fit in start and end date of desired schema config (#8127)
Before compacting tables, we look up the schema config for them. If we do not find a schema config for a table, we skip them. However, the schema lookup code was considering the start time of the schema and not the end time. This can cause the compactor to try compacting tables with the same prefix as one of the previous schemas, but the table interval falls into one of the subsequent schemas with a different config. This PR fixes the issue and simplifies the code by reusing existing schema lookup code. I have also updated test cases to verify the fix works. |
3 years ago |
|
|
e2842c69c5
|
Replace deprecated `io/ioutil` with `io` and `os` packages (#7212)
The `io/ioutil` package has been deprecated in Go 1.16 (See https://pkg.go.dev/io/ioutil). This PR replaces the existing `io/ioutil` functions with their new definitions in `io` and `os` packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com> |
3 years ago |
|
|
1e1c2470b5
|
compactor: prioritize newer tables when compacting. add operator flags to compac… (#6804)
* prioritize newer tables when compacting. add operator flags to compact time ranges of tables Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * reroll doc and PR changes after rebase Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> |
3 years ago |
|
|
882010a810
|
Index shipper move code (#6704)
* move compactor code to generic indexshipper * move table client to generic indexshipper * move storage package and relevant code for managing files on storage to generic indexshipper * add missing files * lint |
4 years ago |
|
|
25912ea02d
|
generic index compactor (#6537)
* make compactor generic to support compacting various index types * implement and register compaction for boltdb-shipper index type * lint and fix broken test * add more comments * fix some code and tests * remove duplicate user-id kv in logger * some improvements in code and test cases * lint |
4 years ago |
|
|
8f02495453
|
Refactoring of the Loki storage (#5833)
* Extract index interface from the series store * moving package around * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * moving package around * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve composite store for more generics index store * fixes first round of tests and ensure coverage is still good * implement series method * Fixes storage tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add test for GetSeries Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Make Loki compile Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint the whole project Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * tsdb util package usage Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more information of why SetChunkFilterer exists for the index Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes ruler/base package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes o2 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> |
4 years ago |
|
|
e4d6767412
|
Fix apply retention issue (#5342)
* call mark phase started/finished only when applying retention * correctly pass userID to IntervalMayHaveExpiredChunks call on retention and dletion expiry check |
4 years ago |
|
|
392fde1e27
|
refactor: use `T.TempDir` to create temporary test directory (#5253)
* refactor: use `T.TempDir` to create temporary test directory The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete. Reference: https://pkg.go.dev/testing#T.TempDir Signed-off-by: Eng Zer Jun <engzerjun@gmail.com> * docs: update CHANGELOG.md Signed-off-by: Eng Zer Jun <engzerjun@gmail.com> |
4 years ago |
|
|
bfef7bad44
|
Start refactor of storage (chunks) clients metrics into structs (#5057)
* Refactor azure metrics to be a struct (+ add an overall client metrics struct), add an azure client metric for total egress bytes. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add changelog entry for new azure metric. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fixes based on Kavi's review. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix one more merge conflict that I missed in the merge commit. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix failing tests. Signed-off-by: Callum Styan <callumstyan@gmail.com> |
4 years ago |
|
|
0426870f3f
|
fix broken boltdb-shipper test (#5222)
|
4 years ago |
|
|
88b5a767bc
|
changes on read path for supporting per user index with boltdb-shipper (#5073)
|
4 years ago |
|
|
ff47d7419a
|
compactor changes for building per user index files in boltdb shipper (#5026)
* compactor changes for building per user index files in boltdb shipper * update comments * lint and changes suggested from PR review * fix broken code after merging main |
4 years ago |
|
|
02d88462f3
|
allow applying retention at different interval than compaction with a config (#4736)
* allow applying retention at different interval than compaction with a config * add changelog entry |
4 years ago |
|
|
07ea1ed99c
|
Add common ring configuration (#4617)
* add ring section to common config Co-Authored-By: Dylan Guedes <djmgguedes@gmail.com> * fix compactor test to append loopback interface it ifnames in test if present * add loopback interface to compactor ring iface names * clean-up from PR review Co-authored-by: Dylan Guedes <djmgguedes@gmail.com> * only appy ingester config to rings with no non-default properties Co-authored-by: Dylan Guedes <djmgguedes@gmail.com> |
4 years ago |
|
|
0c9beadaaa
|
Loki: Add a ring to the compactor used to control concurrency when not running standalone (#4574)
* add a ring to the compactor * wiring up the start/stop functionality * add some monitoring for multiple compactors running at the same time. * Add compactor to `read` target * fix tests/validation requirements * Update pkg/storage/stores/shipper/compactor/compactor_ring.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/storage/stores/shipper/compactor/compactor_ring.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/storage/stores/shipper/compactor/compactor_ring.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * used a shared ring_config between scheduler and compactor to avoid some duplication * fix lint * better handling of the tokens file with a way to easily tell all the rings to persist tokens * fix query scheduler replication factor * removing "scheduler" from log line which may be called by components other than a scheduler in the future. * update changelog Co-authored-by: Owen Diehl <ow.diehl@gmail.com> |
4 years ago |
|
|
cfb4fc1f55
|
Use flagext from dskit (#4225)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> |
4 years ago |
|
|
29b07d190d
|
Retention speedup (#4172)
* drop the multi table chunk entries from index when the whole table is out of retention * speed up cleaning of deleted series from index * mark partially deleted chunk for deletion when all its tables are processed |
4 years ago |
|
|
32e55c3043
|
Fork cortex chunk storage into Loki. (#3842)
The chunk package has been forked from this commit for future reference and licensing.
|
5 years ago |
|
|
8a9abdfe16
|
Compactor improvements (#4018)
* use previously compacted file as seed file and copy index into it * allow configuring compactor to compact multiple tables at a time |
5 years ago |
|
|
6a36cf6745
|
add some tests in compactor and fix a bug in IntervalHasExpiredChunks check in retention with tests (#3969)
* add some tests in compactor and fix a bug in IntervalHasExpiredChunks check in retention with tests * check only for start time |
5 years ago |
|
|
7a7eeb0d63
|
skip applying retention when data in table is still within retention period and there are no overlapping delete requests (#3934)
|
5 years ago |
|
|
2b7a6f201c
|
Logs deletion support for boltdb-shipper store (#3688)
* add support for slicing the chunks for partial chunk deletion * add delete requests store and handler for requests * add delete requests manager and a way to check for chunk expiry based on delete requests * add chunk rewriter for partially deleted chunks * hook delete requests manager into compactor and update expiration checker * use chunk client in sweeper for deleting chunks * upload delete requests table during shutdown and some code refactoring * add tests and metrics * fixes post rebase * lint * changes suggested from PR review * avoid allocation when getting userid from chunkid * use bytes.IndexByte instead |
5 years ago |
|
|
43184298b2
|
Trigger compaction prior retention. (#3665)
* Trigger compaction prior retention. This should allow to process table where the amount of index files is below 4. Currently they are not processed and we should process them since they may never be compacted. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Hook retention into compaction. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove unused flags. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Better comment. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Better cancellation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Missing ctx in test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes flags that are not used. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add compactor retention tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> |
5 years ago |
|
|
806d6a503e
|
Custom Retention (#3642)
* Playing around with a POC based on the design doc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Getting series ID for label matchers is now working. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * wip/ Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * We're parsing the index label now. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Starting to extract interfaces to make the code testable. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Work in progress will try to add labels to chunk ref iterator Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Iterator for chunks ref with all labels !!!. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Optimize code away. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * with delete into the mix Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Cleaner but not yet working for v10 and v11. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes series cleaner. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * hooking into the compactor. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Hooking limit retention config. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finishing off the marker processor. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working on sweeper and fixing tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adding more tests and founding more bugs along the way. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Bug with path once boltdb is closed. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing more bug and more robust test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More test and cleanup getting close. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes moar bugs with regards to period schema Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix a flaky tests because of boltdb still open. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more metrics. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adding metrics. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve benchmark. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes issue. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint code. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * more logs. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Save files without using table key Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve logging and ability to use more goroutines. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Removes duplicate metrics since histogram contains total too. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more logs. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a deadlock bug when too many workers are trying to update the mark file. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a deadlock when reading and updating db at the same time. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes default config test of boltdb. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * PR Review feedbacks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove the user of assert to not fail a test if it's incorrect on the first shot. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add experimental notice to the flag documentation Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes empty index detection and table deletion. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Do not delete folder it's not necessary with object store. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Better working path cleanup Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> |
5 years ago |
|
|
835f710bc9
|
make prefix for keys of objects created by boltdb-shipper configurable (#3491)
* make prefix for keys of objects created by boltdb-shipper configurable * shared store key prefix should have a path separator |
5 years ago |
|
|
9249ebc022
|
Fix/validate compactor config (#2824)
* default compactor config throws error when compactor is a target or subtarget * linting * moves compactor config validation to NewCompactor fn |
5 years ago |