apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Michel Hollands	8868957783	Log delete request ID upon creation (#7307 ) Log the request ID when a new delete request is created. Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Bryan Boreham	9498ea2c02	chunks: decode varints directly from byte buffer; stop panicing on some corrupt inputs (#7264 ) What this PR does / why we need it: This is much faster, since we don't pay for byte-at-a-time reads from the buffer. For best performance with gzip and lz4, we still want a bufio.Reader. We don't need the pool of buffered readers now. Other compression formats have their own internal buffers so don't need extra buffering. Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - NA Documentation added - NA Tests updated - [x] `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` (using changes from https://github.com/grafana/loki/pull/7246, ignoring lines that didn't change much) ``` name old time/op new time/op delta Write/ordered-lz4-256k-4 24.1ms ± 7% 22.1ms ± 2% -8.52% (p=0.008 n=5+5) Write/unordered-lz4-4 54.6ms ±29% 49.1ms ± 1% -10.04% (p=0.016 n=5+4) Read/none_66_kB-4 4.16ms ± 6% 2.54ms ± 2% -38.87% (p=0.008 n=5+5) Read/gzip_66_kB-4 177ms ± 4% 161ms ± 4% -9.09% (p=0.008 n=5+5) Read/lz4-64k_66_kB-4 59.5ms ± 2% 49.5ms ± 3% -16.80% (p=0.008 n=5+5) Read/lz4-256k_66_kB-4 62.9ms ± 0% 51.2ms ± 1% -18.58% (p=0.016 n=4+5) Read/lz4-1M_66_kB-4 62.9ms ± 2% 51.5ms ± 1% -18.03% (p=0.008 n=5+5) Read/lz4_66_kB-4 63.7ms ± 4% 51.2ms ± 1% -19.68% (p=0.008 n=5+5) Read/snappy_66_kB-4 51.1ms ± 2% 41.9ms ± 1% -17.95% (p=0.016 n=5+4) Read/flate_66_kB-4 168ms ± 4% 150ms ± 2% -10.49% (p=0.008 n=5+5) Read/zstd_66_kB-4 334ms ±16% 272ms ± 6% -18.52% (p=0.008 n=5+5) Read/none_262_kB-4 3.93ms ± 1% 2.48ms ± 2% -36.91% (p=0.008 n=5+5) Read/gzip_262_kB-4 159ms ± 5% 143ms ± 2% -10.12% (p=0.008 n=5+5) Read/lz4-64k_262_kB-4 53.5ms ± 1% 43.2ms ± 3% -19.35% (p=0.008 n=5+5) Read/lz4-256k_262_kB-4 55.8ms ± 1% 45.3ms ± 2% -18.75% (p=0.008 n=5+5) Read/lz4-1M_262_kB-4 57.5ms ± 2% 45.5ms ± 2% -20.91% (p=0.008 n=5+5) Read/lz4_262_kB-4 56.8ms ± 2% 45.3ms ± 2% -20.26% (p=0.008 n=5+5) Read/snappy_262_kB-4 46.0ms ± 2% 37.5ms ± 1% -18.57% (p=0.008 n=5+5) Read/flate_262_kB-4 153ms ± 3% 134ms ± 2% -12.33% (p=0.008 n=5+5) Read/none_524_kB-4 3.85ms ± 2% 2.46ms ± 2% -36.04% (p=0.008 n=5+5) Read/gzip_524_kB-4 128ms ± 2% 119ms ± 2% -7.64% (p=0.008 n=5+5) Read/lz4-64k_524_kB-4 44.6ms ± 3% 34.7ms ± 1% -22.19% (p=0.008 n=5+5) Read/lz4-256k_524_kB-4 45.4ms ± 1% 36.7ms ± 5% -19.07% (p=0.008 n=5+5) Read/lz4-1M_524_kB-4 47.3ms ± 1% 38.6ms ± 2% -18.36% (p=0.008 n=5+5) Read/lz4_524_kB-4 47.4ms ± 1% 38.0ms ± 1% -19.70% (p=0.008 n=5+5) Read/snappy_524_kB-4 37.3ms ± 1% 31.1ms ± 3% -16.47% (p=0.008 n=5+5) Read/flate_524_kB-4 122ms ± 1% 111ms ± 3% -9.04% (p=0.008 n=5+5) Read/sample_none_66_kB-4 7.81ms ± 1% 6.62ms ± 6% -15.24% (p=0.008 n=5+5) Read/sample_gzip_66_kB-4 237ms ± 2% 219ms ± 1% -7.57% (p=0.008 n=5+5) Read/sample_lz4-64k_66_kB-4 105ms ± 2% 92ms ± 1% -12.03% (p=0.008 n=5+5) Read/sample_lz4-256k_66_kB-4 109ms ± 1% 98ms ± 2% -9.65% (p=0.008 n=5+5) Read/sample_lz4-1M_66_kB-4 113ms ± 8% 98ms ± 2% -13.66% (p=0.008 n=5+5) Read/sample_lz4_66_kB-4 109ms ± 1% 99ms ± 2% -9.29% (p=0.008 n=5+5) Read/sample_snappy_66_kB-4 86.6ms ± 1% 77.2ms ± 6% -10.92% (p=0.008 n=5+5) Read/sample_flate_66_kB-4 236ms ± 8% 214ms ± 3% -9.11% (p=0.008 n=5+5) Read/sample_none_262_kB-4 7.70ms ± 2% 6.34ms ± 2% -17.67% (p=0.008 n=5+5) Read/sample_lz4-64k_262_kB-4 94.7ms ± 1% 84.4ms ± 2% -10.90% (p=0.008 n=5+5) Read/sample_lz4-256k_262_kB-4 98.4ms ± 1% 87.7ms ± 2% -10.93% (p=0.008 n=5+5) Read/sample_lz4-1M_262_kB-4 101ms ± 2% 90ms ± 2% -11.19% (p=0.008 n=5+5) Read/sample_lz4_262_kB-4 100ms ± 2% 90ms ± 3% -10.78% (p=0.008 n=5+5) Read/sample_snappy_262_kB-4 77.9ms ± 2% 68.7ms ± 2% -11.84% (p=0.008 n=5+5) Read/sample_flate_262_kB-4 211ms ± 2% 190ms ± 2% -9.86% (p=0.008 n=5+5) Read/sample_none_524_kB-4 7.82ms ± 2% 6.43ms ± 2% -17.80% (p=0.008 n=5+5) Read/sample_gzip_524_kB-4 176ms ± 1% 164ms ± 2% -6.80% (p=0.008 n=5+5) Read/sample_lz4-64k_524_kB-4 77.7ms ± 1% 69.8ms ± 4% -10.10% (p=0.008 n=5+5) Read/sample_lz4-256k_524_kB-4 81.7ms ± 2% 72.1ms ± 3% -11.79% (p=0.008 n=5+5) Read/sample_lz4-1M_524_kB-4 83.8ms ± 1% 74.4ms ± 1% -11.21% (p=0.008 n=5+5) Read/sample_lz4_524_kB-4 83.2ms ± 0% 76.1ms ± 8% -8.53% (p=0.016 n=4+5) Read/sample_snappy_524_kB-4 63.6ms ± 2% 56.5ms ± 2% -11.23% (p=0.008 n=5+5) Read/sample_flate_524_kB-4 172ms ± 2% 157ms ± 1% -8.88% (p=0.008 n=5+5) name old speed new speed delta Write/ordered-lz4-256k-4 687MB/s ± 6% 749MB/s ± 2% +9.09% (p=0.008 n=5+5) Write/unordered-lz4-4 313MB/s ±24% 341MB/s ± 1% +9.23% (p=0.016 n=5+4) Read/none_66_kB-4 1.82GB/s ± 6% 2.97GB/s ± 2% +63.42% (p=0.008 n=5+5) Read/gzip_66_kB-4 669MB/s ± 4% 736MB/s ± 4% +10.00% (p=0.008 n=5+5) Read/lz4-64k_66_kB-4 1.42GB/s ± 2% 1.71GB/s ± 3% +20.21% (p=0.008 n=5+5) Read/lz4-256k_66_kB-4 1.43GB/s ± 0% 1.76GB/s ± 1% +22.83% (p=0.016 n=4+5) Read/lz4-1M_66_kB-4 1.43GB/s ± 2% 1.74GB/s ± 1% +21.99% (p=0.008 n=5+5) Read/lz4_66_kB-4 1.41GB/s ± 4% 1.76GB/s ± 1% +24.44% (p=0.008 n=5+5) Read/snappy_66_kB-4 1.29GB/s ± 2% 1.57GB/s ± 1% +21.86% (p=0.016 n=5+4) Read/flate_66_kB-4 710MB/s ± 4% 793MB/s ± 2% +11.70% (p=0.008 n=5+5) Read/zstd_66_kB-4 425MB/s ±14% 517MB/s ± 6% +21.74% (p=0.008 n=5+5) Read/none_262_kB-4 1.93GB/s ± 1% 3.06GB/s ± 2% +58.54% (p=0.008 n=5+5) Read/gzip_262_kB-4 701MB/s ± 5% 779MB/s ± 2% +11.20% (p=0.008 n=5+5) Read/lz4-64k_262_kB-4 1.47GB/s ± 1% 1.82GB/s ± 3% +24.03% (p=0.008 n=5+5) Read/lz4-256k_262_kB-4 1.48GB/s ± 1% 1.82GB/s ± 2% +23.09% (p=0.008 n=5+5) Read/lz4-1M_262_kB-4 1.46GB/s ± 2% 1.84GB/s ± 2% +26.44% (p=0.008 n=5+5) Read/lz4_262_kB-4 1.48GB/s ± 2% 1.85GB/s ± 2% +25.42% (p=0.008 n=5+5) Read/snappy_262_kB-4 1.31GB/s ± 2% 1.61GB/s ± 1% +22.80% (p=0.008 n=5+5) Read/flate_262_kB-4 729MB/s ± 3% 831MB/s ± 2% +14.05% (p=0.008 n=5+5) Read/none_524_kB-4 1.97GB/s ± 2% 3.08GB/s ± 2% +56.33% (p=0.008 n=5+5) Read/gzip_524_kB-4 714MB/s ± 2% 773MB/s ± 2% +8.27% (p=0.008 n=5+5) Read/lz4-64k_524_kB-4 1.47GB/s ± 3% 1.88GB/s ± 1% +28.48% (p=0.008 n=5+5) Read/lz4-256k_524_kB-4 1.50GB/s ± 1% 1.86GB/s ± 5% +23.63% (p=0.008 n=5+5) Read/lz4-1M_524_kB-4 1.49GB/s ± 1% 1.83GB/s ± 2% +22.51% (p=0.008 n=5+5) Read/lz4_524_kB-4 1.49GB/s ± 1% 1.86GB/s ± 1% +24.53% (p=0.008 n=5+5) Read/snappy_524_kB-4 1.34GB/s ± 1% 1.60GB/s ± 3% +19.75% (p=0.008 n=5+5) Read/flate_524_kB-4 750MB/s ± 1% 824MB/s ± 3% +9.97% (p=0.008 n=5+5) Read/sample_none_66_kB-4 967MB/s ± 1% 1142MB/s ± 6% +18.13% (p=0.008 n=5+5) Read/sample_gzip_66_kB-4 501MB/s ± 2% 542MB/s ± 1% +8.19% (p=0.008 n=5+5) Read/sample_lz4-64k_66_kB-4 811MB/s ± 2% 922MB/s ± 1% +13.66% (p=0.008 n=5+5) Read/sample_lz4-256k_66_kB-4 825MB/s ± 1% 913MB/s ± 2% +10.68% (p=0.008 n=5+5) Read/sample_lz4-1M_66_kB-4 795MB/s ± 7% 920MB/s ± 1% +15.67% (p=0.008 n=5+5) Read/sample_lz4_66_kB-4 824MB/s ± 1% 909MB/s ± 2% +10.25% (p=0.008 n=5+5) Read/sample_snappy_66_kB-4 759MB/s ± 1% 852MB/s ± 6% +12.37% (p=0.008 n=5+5) Read/sample_flate_66_kB-4 506MB/s ± 7% 556MB/s ± 4% +9.88% (p=0.008 n=5+5) Read/sample_none_262_kB-4 983MB/s ± 2% 1194MB/s ± 2% +21.47% (p=0.008 n=5+5) Read/sample_lz4-64k_262_kB-4 830MB/s ± 1% 932MB/s ± 2% +12.24% (p=0.008 n=5+5) Read/sample_lz4-256k_262_kB-4 839MB/s ± 1% 942MB/s ± 2% +12.28% (p=0.008 n=5+5) Read/sample_lz4-1M_262_kB-4 829MB/s ± 2% 934MB/s ± 2% +12.60% (p=0.008 n=5+5) Read/sample_lz4_262_kB-4 836MB/s ± 2% 937MB/s ± 3% +12.11% (p=0.008 n=5+5) Read/sample_snappy_262_kB-4 774MB/s ± 2% 878MB/s ± 2% +13.44% (p=0.008 n=5+5) Read/sample_flate_262_kB-4 529MB/s ± 2% 587MB/s ± 2% +10.94% (p=0.008 n=5+5) Read/sample_none_524_kB-4 970MB/s ± 2% 1180MB/s ± 2% +21.64% (p=0.008 n=5+5) Read/sample_gzip_524_kB-4 521MB/s ± 1% 559MB/s ± 2% +7.31% (p=0.008 n=5+5) Read/sample_lz4-64k_524_kB-4 842MB/s ± 1% 937MB/s ± 4% +11.30% (p=0.008 n=5+5) Read/sample_lz4-256k_524_kB-4 834MB/s ± 2% 945MB/s ± 3% +13.37% (p=0.008 n=5+5) Read/sample_lz4-1M_524_kB-4 842MB/s ± 1% 949MB/s ± 1% +12.63% (p=0.008 n=5+5) Read/sample_lz4_524_kB-4 849MB/s ± 0% 929MB/s ± 7% +9.53% (p=0.016 n=4+5) Read/sample_snappy_524_kB-4 783MB/s ± 2% 882MB/s ± 2% +12.65% (p=0.008 n=5+5) Read/sample_flate_524_kB-4 534MB/s ± 2% 585MB/s ± 1% +9.73% (p=0.008 n=5+5) name old alloc/op new alloc/op delta Write/unordered-lz4-4 18.8MB ± 4% 17.2MB ± 7% -8.28% (p=0.008 n=5+5) Read/none_66_kB-4 41.9kB ± 0% 43.7kB ± 0% +4.41% (p=0.008 n=5+5) Read/gzip_66_kB-4 713kB ± 0% 741kB ± 0% +3.85% (p=0.016 n=5+4) Read/lz4-64k_66_kB-4 730kB ± 1% 748kB ± 0% +2.53% (p=0.016 n=5+4) Read/lz4-256k_66_kB-4 770kB ± 0% 806kB ± 3% +4.70% (p=0.016 n=4+5) Read/lz4-1M_66_kB-4 770kB ± 0% 812kB ± 4% +5.43% (p=0.016 n=4+5) Read/lz4_66_kB-4 770kB ± 0% 792kB ± 0% +2.83% (p=0.029 n=4+4) Read/snappy_66_kB-4 361kB ± 0% 376kB ± 0% +4.02% (p=0.008 n=5+5) Read/flate_66_kB-4 715kB ± 0% 743kB ± 0% +3.88% (p=0.016 n=5+4) Read/none_262_kB-4 11.2kB ± 0% 11.7kB ± 0% +4.12% (p=0.016 n=5+4) Read/gzip_262_kB-4 198kB ± 0% 203kB ± 0% +2.93% (p=0.016 n=4+5) Read/lz4-64k_262_kB-4 169kB ± 0% 174kB ± 0% +2.75% (p=0.008 n=5+5) Read/lz4-256k_262_kB-4 177kB ± 0% 182kB ± 0% +2.79% (p=0.008 n=5+5) Read/lz4-1M_262_kB-4 181kB ± 0% 186kB ± 0% +2.77% (p=0.008 n=5+5) Read/lz4_262_kB-4 181kB ± 0% 186kB ± 0% +2.73% (p=0.016 n=5+4) Read/snappy_262_kB-4 87.8kB ± 0% 90.1kB ± 0% +2.58% (p=0.008 n=5+5) Read/flate_262_kB-4 197kB ± 0% 203kB ± 0% +2.63% (p=0.016 n=5+4) Read/zstd_262_kB-4 553MB ± 0% 552MB ± 0% -0.15% (p=0.032 n=5+5) Read/none_524_kB-4 5.94kB ± 0% 6.16kB ± 0% +3.76% (p=0.016 n=5+4) Read/gzip_524_kB-4 96.2kB ± 0% 98.3kB ± 0% +2.16% (p=0.016 n=4+5) Read/lz4-64k_524_kB-4 70.9kB ± 0% 72.8kB ± 0% +2.71% (p=0.016 n=4+5) Read/lz4-256k_524_kB-4 73.8kB ± 0% 75.8kB ± 0% +2.68% (p=0.008 n=5+5) Read/lz4-1M_524_kB-4 76.5kB ± 0% 78.7kB ± 0% +2.77% (p=0.016 n=4+5) Read/lz4_524_kB-4 76.5kB ± 0% 78.6kB ± 0% +2.74% (p=0.016 n=5+4) Read/snappy_524_kB-4 39.1kB ± 0% 39.5kB ± 0% +1.19% (p=0.008 n=5+5) Read/flate_524_kB-4 95.4kB ± 0% 97.3kB ± 0% +2.06% (p=0.008 n=5+5) Read/sample_none_66_kB-4 39.9kB ± 0% 41.7kB ± 0% +4.62% (p=0.008 n=5+5) Read/sample_gzip_66_kB-4 686kB ± 0% 715kB ± 0% +4.23% (p=0.016 n=4+5) Read/sample_lz4-64k_66_kB-4 707kB ± 0% 728kB ± 0% +2.88% (p=0.016 n=5+4) Read/sample_lz4-256k_66_kB-4 749kB ± 0% 783kB ± 4% +4.65% (p=0.008 n=5+5) Read/sample_lz4_66_kB-4 748kB ± 0% 770kB ± 0% +2.91% (p=0.016 n=5+4) Read/sample_snappy_66_kB-4 350kB ± 0% 364kB ± 0% +4.04% (p=0.029 n=4+4) Read/sample_flate_66_kB-4 688kB ± 0% 716kB ± 0% +4.08% (p=0.029 n=4+4) Read/sample_none_262_kB-4 10.6kB ± 0% 11.0kB ± 0% +4.23% (p=0.016 n=5+4) Read/sample_gzip_262_kB-4 194kB ± 0% 199kB ± 0% +2.48% (p=0.016 n=4+5) Read/sample_lz4-64k_262_kB-4 165kB ± 0% 169kB ± 0% +2.86% (p=0.016 n=4+5) Read/sample_lz4-256k_262_kB-4 172kB ± 0% 177kB ± 0% +2.92% (p=0.016 n=4+5) Read/sample_lz4-1M_262_kB-4 176kB ± 0% 181kB ± 0% +2.91% (p=0.029 n=4+4) Read/sample_lz4_262_kB-4 176kB ± 0% 181kB ± 0% +2.81% (p=0.016 n=5+4) Read/sample_snappy_262_kB-4 88.1kB ± 0% 90.8kB ± 0% +3.11% (p=0.029 n=4+4) Read/sample_flate_262_kB-4 194kB ± 0% 198kB ± 0% +2.13% (p=0.029 n=4+4) Read/sample_none_524_kB-4 5.56kB ± 0% 5.76kB ± 0% +3.65% (p=0.016 n=5+4) Read/sample_gzip_524_kB-4 95.4kB ± 0% 96.9kB ± 0% +1.56% (p=0.008 n=5+5) Read/sample_lz4-64k_524_kB-4 69.0kB ± 0% 71.0kB ± 0% +2.87% (p=0.008 n=5+5) Read/sample_lz4-256k_524_kB-4 71.8kB ± 0% 73.8kB ± 0% +2.79% (p=0.016 n=4+5) Read/sample_lz4-1M_524_kB-4 74.5kB ± 0% 76.6kB ± 0% +2.75% (p=0.008 n=5+5) Read/sample_lz4_524_kB-4 74.5kB ± 0% 188.4kB ±89% +152.98% (p=0.008 n=5+5) Read/sample_snappy_524_kB-4 40.7kB ± 1% 41.3kB ± 1% +1.58% (p=0.008 n=5+5) Read/sample_flate_524_kB-4 95.2kB ± 0% 96.2kB ± 0% +0.95% (p=0.008 n=5+5) name old allocs/op new allocs/op delta Read/gzip_66_kB-4 13.2k ± 0% 13.2k ± 0% -0.04% (p=0.000 n=5+4) Read/zstd_66_kB-4 18.0k ± 1% 17.8k ± 1% -1.06% (p=0.032 n=5+5) Read/gzip_262_kB-4 4.61k ± 0% 4.61k ± 0% -0.11% (p=0.029 n=4+4) Read/flate_262_kB-4 4.61k ± 0% 4.60k ± 0% -0.11% (p=0.008 n=5+5) Read/zstd_262_kB-4 7.10k ± 2% 6.88k ± 3% -3.01% (p=0.008 n=5+5) Read/gzip_524_kB-4 2.62k ± 0% 2.62k ± 0% -0.06% (p=0.000 n=4+5) Read/zstd_524_kB-4 6.13k ± 2% 5.94k ± 2% -3.03% (p=0.024 n=5+5) Read/sample_flate_66_kB-4 13.2k ± 0% 13.2k ± 0% -0.01% (p=0.029 n=4+4) Read/sample_flate_262_kB-4 4.62k ± 0% 4.61k ± 0% -0.22% (p=0.029 n=4+4) Read/sample_zstd_262_kB-4 7.12k ± 1% 6.97k ± 1% -2.11% (p=0.008 n=5+5) Read/sample_gzip_524_kB-4 2.63k ± 0% 2.63k ± 0% -0.11% (p=0.008 n=5+5) Read/sample_flate_524_kB-4 2.63k ± 0% 2.62k ± 0% -0.15% (p=0.008 n=5+5) Read/sample_zstd_524_kB-4 6.11k ± 2% 5.89k ± 1% -3.68% (p=0.016 n=5+4) ``` Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Susana Ferreira	10e9695b21	Fix rangemapper with VectorExpr support (#7288 ) What this PR does / why we need it: The new expression type VectorExpr introduced recently in [PR](https://github.com/grafana/loki/pull/7007), is not taken into account in the query mapping in the AST mapper rangemapper. The AST mapper shardmapper was updated in [PR](https://github.com/grafana/loki/pull/7045). Special notes for your reviewer: Checklist - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Gerard Vanloo	bcdf17b0a5	Configurable TLS MinVersion and CipherSuites (#7227 )	3 years ago
Michel Hollands	77c10da81d	Remove deletion filter debug log messages (#7277 ) These debug messages are a slightly misleading and appear too frequent. Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Michel Hollands	391b9cea10	Logcli: load remote schema before validation (#7258 ) The --remote-schema option didn't work as validation would be called before the remote schema was loaded. An error about a missing schema section is the result. This was tested by running against a remote bucket containing a schemaconfig.yaml file with a local config file that does not contain a schema config section. Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Dylan Guedes	dcfba366de	Loki: Shard streams based on desired rate (#7199 ) What this PR does / why we need it: Modify our sharding mechanism to be dictated by the desired rate and ingestion rate. Once the number of shards is calculated, the stream entries will be distributed across the shards. If the number of entries is lower than number of shards, we report a metric and use number of entries as the number of shards.	3 years ago
Owen Diehl	dab6a24541	correct heartbeat timeout to distributor lifecycler for ring page (#7253 ) Passes the correct `heartbeat_timeout` to the created lifecycler config in distributor ring construction. When we didn't include this, it would create a visual bug incorrectly marking many distributors as unhealthy when in fact they were in fine condition. Including The _lifecycler_ `heartbeat_timeout` is _only_ referenced in the status page [here](`74c8cf03ba/vendor/github.com/grafana/dskit/ring/lifecycler.go (L892-L894)`). The _ring_ `heartbeat_timeout` continues to be updated correctly.	3 years ago
李国忠	4089b73bc3	[Optimization] logql client: When logql fails to execute, return the failure information to the caller (#6671 ) …ail msg to the client caller. When logql fails to execute, return the failure information to the caller. Solve the problem that grafana's loki alarm cannot see the cause of failure. Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Eng Zer Jun	e2842c69c5	Replace deprecated `io/ioutil` with `io` and `os` packages (#7212 ) The `io/ioutil` package has been deprecated in Go 1.16 (See https://pkg.go.dev/io/ioutil). This PR replaces the existing `io/ioutil` functions with their new definitions in `io` and `os` packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	3 years ago
李国忠	557fdf2206	[bugfix] logql: vectorExpr run fail when query_range: parallelise_shardable_queries: true (#7045 ) …hardable_queries: true <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: A grafana error occurred when I deployed the vector(0) function to our online loki cluster. It was finally found because online loki has a configuration `unexpected expr type (syntax.VectorExpr) for ASTMapper type (logql.ShardMapper) ` ```yaml query_range: parallelise_shardable_queries: true ``` Which issue(s) this PR fixes: Fixes # Special notes for your reviewer: error snapssnapshot ![image](https://user-images.githubusercontent.com/9583245/188357957-e573e589-a789-4e5e-a1c3-b91c019ceb62.png) with this pr，success snapshot ![image](https://user-images.githubusercontent.com/9583245/188358036-5ca0d9be-c2b6-4516-807c-c027445ee62d.png) <!-- Note about CHANGELOG entries, if a change adds: an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Sandeep Sukhani	2af3d680cb	tsdb supports data deletion, this PR takes care of todo to enable delete request client for tsdb (#7223 ) What this PR does / why we need it: `tsdb` index store supports deletion of data, but we had missed enabling the creation of delete requests client, which is required for query time filtering of data requested for deletion. This PR takes care of it. I have also removed some unused code and adjusted the tests accordingly. Checklist - [x] Tests updated	3 years ago
Sandeep Sukhani	7f298ff72f	avoid using bloomfilters for chunks in stats calls by avoiding duplicates (#7209 ) What this PR does / why we need it: Avoid using bloomfilters for chunks deduplication in tsdb `Stats` calls by avoiding fetching duplicate entries. The idea is to split and align queries by [ObjectStorageIndexRequiredPeriod](`61794710a7/pkg/storage/config/schema_config.go (L47)`) and make each split process chunks with a start time >= start time of the table interval. In other terms, table interval that contains start time of the chunk, owns it. For e.g. if the table interval is 10s, and we have chunks 5-7, 8-12, 11-13. Query with range 6-15 would be split into 6-10, 10-15. query1 would process chunks 5-7, 8-12 and query2 would process chunks 11-13. This check is not applied for the first split so that we do not eliminate any chunks that overlaps the original query intervals but starts at the previous table. For e.g. if the table interval is 10s, and we have chunks 5-7, 8-13, 14-13. Query with range 11-12 should process chunk 8-13 even though its start time <= start time of table we will query for index. The caveat here is that we will overestimate the data we will be processing if the index is not compacted yet since it could have duplicate chunks when RF > 1. I think it is okay since the Stats call is just an estimation and need not be accurate. Removing all the extra processing saves us quite a bit of CPU and memory, as seen from the benchmark comparison between the two implementations: ``` name old time/op new time/op delta IndexClient_Stats-10 187µs ± 0% 34µs ± 1% -82.00% (p=0.008 n=5+5) name old alloc/op new alloc/op delta IndexClient_Stats-10 61.5kB ± 4% 12.5kB ± 2% -79.69% (p=0.008 n=5+5) name old allocs/op new allocs/op delta IndexClient_Stats-10 1.46k ± 0% 0.48k ± 0% -67.28% (p=0.008 n=5+5) ``` Checklist - [x] Tests updated	3 years ago
Owen Diehl	bac52165d3	highly sharded subqueries are ellided in expr.String (#7203 ) This is mostly a quality of life improvement. Highly sharded subqueries can be burdensome to view and they're logged on every request. This PR will only display the first four subqueries in a `Concat{Log,Sample}Expr`, which is easier on the eyes when viewing logs and is more efficient.	3 years ago
Sandeep Sukhani	4e7105d018	add index requests panel to reads/writes dashboard (#7163 ) What this PR does / why we need it: In PR #7154, I refactored the code also to measure the latency on index write requests. This PR adds the index requests panel to the Loki Reads/Writes dashboard.	3 years ago
Christian Haudum	67b12f2357	Update distributor push tests (#6641 ) Adds a test case for concurrent pushes (which would have spotted a bug in https://github.com/grafana/loki/pull/5248) and a test case for tolerance regarding replication factor. The PR also replaces the mock ring with an actual in-memory kv store ring. And variables and function names were changed to meet Loki/logs terminology. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Ashwanth	3245625c40	ingester_querier: early exit queries on reaching quorum (#7172 ) Currently `forGivenIngesters` and its derivatives don't allow passing down context to the work function as a result of which we are not making use of the tracking functionality provided by `replicationSet.Do` to exit early and relieve pressure on the ingesters. This change ensures that Unary rpc calls receive ctx populated by the replication set tracker to exit early on reaching quorum or max failures. The same cannot be applied to stream rpc calls given how the code is structured at the moment where we are processing the streams outside of the work func	3 years ago
Owen Diehl	12ebd33f94	TSDB indexBuckets helper logs instead of errors (#7187 ) indexBuckets will return the list of overlapping TSDB buckets for a time range. Historically, this errored when there was no overlap, which is generally impossible. While mutating period configurations as a test (which should never be done in production), I realized this would preemptively error us when there were old TSDB WALs from a bucket that no longer belonged to a TSDB period (i.e. by replacing a `tsdb` index with a `boltdb-shipper` one). This PR changes this to a log line instead as we don't need to error and stop computation; it's better to return an empty overlap instead. Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	3 years ago
Owen Diehl	f1114fd221	[Bugfix] TSDB index bucket calculation (#7178 ) When calculating the index buckets a chunk belongs to, we shouldn't error when a bucket falls outside the current TSDB store's periodConfig range. This can happen when a chunk is written across the bounds between a previous index type (i.e. boltdb-shipper) and a new tsdb type or vice versa. Instead of erroring when this happens, we'll now correctly return only the buckets matching the TSDB store (the other buckets are handled by their respective stores). I think we can also remove the error from this function signature, although I left it in as it's conceivably helpful in the case of a chunk range that doesn't overlap at all with the store (which shouldn't happen). Dependent on https://github.com/grafana/loki/pull/7177	3 years ago
Owen Diehl	85e8f00958	fixes azure client secret duplicate CLI registration (#7185 ) Fixes duplicate key registration from https://github.com/grafana/loki/pull/7179/files#diff-665c7ce7ef736bdf9f9da76a8e389d5b6fcdc90099db913ce36b4c0da2a97ef2R121-R122	3 years ago
Owen Diehl	ab24efe1bf	allow backup store metrics registration by including backend type in labels (#7177 ) The refactor [here](https://github.com/grafana/loki/blob/main/pkg/storage/store.go#L237) introduced in https://github.com/grafana/loki/pull/7154 will cause a panic due to duplicate registration via the same store prefix when using a backup index store (for use during tsdb migrations). This PR adds more granularity to the label value to differentiate these and avoid the problem.	3 years ago
Vladyslav Diachenko	7f629c8849	[loki] Azure auth using Service Principal (#7179 ) This PR adds the ability to use Service Principal credentials to authenticate through Azure OAuth. It might be needed when we want to access Azure Blob storage from another tenant. Previously we could use only StorageAccount key that can not be used cross-tenant.	3 years ago
Owen Diehl	2766e0a8c7	fixes nil interface value with non-nil type in cache construction (#7175 ) Fixes a bug introduced in https://github.com/grafana/loki/pull/7013 This will panic when the interface's _value_ is nil because it's been assigned a concrete type already and then passed as an interface. Instead, we treat it as a nil interface with no type bound to it: ``` panic: runtime error: invalid memory address or nil pointer dereference goroutine 40570 [running]: github.com/grafana/loki/pkg/util/server.onPanic({0x2209320, 0x3dfae20}) /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/util/server/recovery.go:41 +0x53 github.com/grafana/loki/pkg/util/server.glob..func1.1.1() /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/util/server/recovery.go:29 +0x3a panic({0x2209320, 0x3dfae20}) /usr/local/go/src/runtime/panic.go:838 +0x207 github.com/grafana/loki/pkg/storage/stores/indexshipper/compactor/generationnumber.(GenNumberLoader).getCacheGenNumber(0x0, {0xc0081e14d7, 0x2}) /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/storage/stores/indexshipper/compactor/generationnumber/gennumber_loader.go:120 +0x60 github.com/grafana/loki/pkg/storage/stores/indexshipper/compactor/generationnumber.(GenNumberLoader).getCacheGenNumbersPerTenants(0x2?, {0xc000e8c4d0?, 0x1, 0x0?}) /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/storage/stores/indexshipper/compactor/generationnumber/gennumber_loader.go:98 +0xe7 github.com/grafana/loki/pkg/storage/stores/indexshipper/compactor/generationnumber.(*GenNumberLoader).GetResultsCacheGenNumber(0x2b26298?, {0xc000e8c4d0?, 0xc0050ad080?, 0x2dcdeeb9bcc040e4?}) ```	3 years ago
Travis Patterson	071118a090	revert stream push error response (#7160 ) We've changed approaches to how we want to do per-stream rate limiting. The new approach will rely on creating an api on the ingester so the distributors can decide how streams should be sharded. As a result, the work to return rate-limited streams in the error body is no longer needed.	3 years ago
Sandeep Sukhani	c8c8303a6a	refactor index storage code to index reader/writer interfaces for better monitoring (#7154 ) What this PR does / why we need it: Currently, it is hard to monitor just the index latencies since the chunks PUT code is coupled with both object storage and index writer. This PR refactors the code to have separate interfaces defined for index readers/writers. I have also added a wrapper called `monitoredReaderWriter`, which measures the latency on the index operations. Special notes for your reviewer: Cortex had added `PutChunksAndIndex` for DynamoDB, which would write both chunks and index in a single request when configured for storing both. I have removed the use of it since it would be too complex to support it with this refactoring. Loki, anyways does not recommend using chunks in the index store, and using a hosted object storage would be much cheaper. Also, this is not a breaking change, it might somewhat impact the performance of flushing the chunks. Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Ashwanth	88503ae2e1	tsdb: clean-up metrics (#7150 ) What this PR does / why we need it: clean-up unused tsdb metrics and refactor others to use a `CounterVec` Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	3 years ago
Kaviraj Kanagaraj	6f0690e7bf	fix(ruler): Initialize multi remote clients config (#7151 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: To be backward compatible with single client config. ``` ts=2022-09-14T08:24:04.549961234Z caller=memberlist_logger.go:74 level=debug msg="Initiating push/pull sync with: 10.132.18.167:7946" │ │ panic: assignment to entry in nil map │ │ goroutine 1 [running]: │ │ github.com/grafana/loki/pkg/ruler.NewRuler({{{0xc00002a120}, {0x0, 0x0, 0x0}, {0x6400000, 0x6400000, {0x0, 0x0}, 0x0, 0x0, ...}, ...}, ...}, ...) │ │ /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/ruler/ruler.go:21 +0x1b7 ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Karol Bakunowski	4e7e51ceb5	Remove duplicated `loki_boltdb_shipper` prefix from `tables_upload_operation_total` metric (#7040 ) This PR removes the duplicated `loki_boltdb_shipper` prefix from the `tables_upload_operation_total` metric.	3 years ago
Travis Patterson	3f0ecd3c7b	Implement all-or-nothing stream ingestion (#7094 ) Per-stream sharding will introduce automatic stream sharding on rate-limited streams. The ingester's current line-by-line behavior may result in duplicates when a partially-accepted stream is resharded and sent to other ingesters. This PR makes the ingester either rate limit the entire stream before attempting to write lines to a chunk. The PR 1. Separates validation, storage, and wal replay into their own functions 2. Implements a flag to rate limit the whole stream or not 3. Checks rate limits for the whole stream rather than line-by-line 4. Returns an error with information on the whole stream when rate limited	3 years ago
李国忠	461f844044	[bugfix] fix query_frontend init fail error (#7013 ) Co-authored-by: Ashwanth <iamashwanth@gmail.com>	3 years ago
Sandeep Sukhani	61794710a7	tsdb: boltdb shipper backup store tsdb (#7130 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: Since we are planning to start deploying `tsdb` to critical cells, we want to have a disaster recovery without any data loss. This PR adds support to `tsdb` store itself to configure `boltdb-shipper` as a backup chunk writer. When the flag is enabled, `tsdb` store would also index chunks in `boltdb-shipper` index store with table names set to `<configured-tsdb-table-prefix>`+`backup_`+`<table-number>`. This is the simplest option that I can come up with without maintaining a lot of code and we can simply drop the support before marking `tsdb` as production ready. The rollback/recovery plan would just be to change the schema config to use `boltdb-shipper` as index store with `<configured-tsdb-table-prefix>`+`backup_` as prefix like below: tsdb enabled: ``` schema_config: configs: - from: 2022-01-01 store: boltdb-shipper index: prefix: index_ - from: 2022-09-10 store: tsdb index: prefix: index_ ``` rollback: ``` schema_config: configs: - from: 2022-01-01 store: boltdb-shipper index: prefix: index_ - from: 2022-09-10 store: boltdb-shipper index: prefix: index_backup_ ``` <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [x] Tests updated	3 years ago
Periklis Tsirakidis	630a491580	loki: Attach the panic recovery handler on all HTTP handlers (#6780 )	3 years ago
Travis Patterson	ef47e0ee8a	Fix errors loading deletes after force-removing partially-completed delete (#7125 ) When sharded deletes that are partially completed are force-removed, everything is removed except the shards that are currently being deleted. The shards being deleted aren't necessarily in numerical order so there's no way to know assume which shards have and haven't been deleted. The Delete store assumed it could look up requests in sequential order and returned empty requests where there were gaps in the stored data. This change takes the `sequence number` from the stored key of the delete requests it finds and uses that to look up details.	3 years ago
Michel Hollands	accb5e1ca2	Log the interval of a delete request (#7120 ) What this PR does / why we need it: An interval is generated automatically for a delete request: this is the max size of a shard of a deletion request. This value is now logged. Additionally a setting name that should have been renamed was fixed. Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Kaviraj Kanagaraj	2c9fa05c29	feat(loki-canary): Add support to push logs directly to Loki. (#7063 ) Add additional `push` mode to loki-canary, which pushes the logs directly to given Loki URL as it generates logs. The real function of Loki Canary is to act like a tenant and help us know the whether Loki installation is working as perceived by a real tenant. Main rationale for this additional push mode is to make canary more standalone without needing for `promtail` (or `grafana-agent`) to scrape it's logs and send to loki, with this change, Loki canary happily tests Loki behavior without needing any other dependencies. NOTES: 1. If you run Loki behind any proxy that has different authorization policies to READ or WRITE to Loki, then important change that canary operator need to be aware of it, now the user credentials they pass via `-user` and `-pass` to access loki endpoints need to have both `read' and `write` permissions (previously canary just query the logs where as promtail is the one pushes the logs, so just READ permissions was sufficient). 2. There will be follow up PR(s) to cleanup, particularly `reader` and `comparitor` component in terms of logging with proper logger. Rationale is to keep the changes small per PR to make it easy to review. 3. This changes were tested it in one of the internal Loki dev cell. 4. This PR is a no-op if this new `push` flag is disabled (it's disabled by default) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Mohamed-Amine Bouqsimi	3d4788fb51	Add ability for Ruler to remote write to multiple clients (#6823 )	3 years ago
Michel Hollands	08cf2153d5	Add sequence num to log output for delete requests (#7087 ) Signed-off-by: Michel Hollands <michel.hollands@grafana.com> What this PR does / why we need it: Print the sequence number of the delete request so the messages are not all the same. Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Periklis Tsirakidis	1a41682bb0	Fix internal server initialization (#7085 )	3 years ago
Periklis Tsirakidis	c30fb9450b	Add support for custom internal server (#7069 ) Co-authored-by: Christian Simon <simon@swine.de>	3 years ago
Adam	eaf7f34f49	bugfix:add cases.NoLower option (#7052 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: see #7048 if we don't apply cases.NoLower, it will upper first alphabet lower the others ```go package main import ( "fmt" "golang.org/x/text/cases" "golang.org/x/text/language" "strings" ) func main() { source := "testAPI" fmt.Println(cases.Title(language.Und).String(source)) // out: Testapi fmt.Println(cases.Title(language.Und, cases.NoLower).String(source)) // out: TestAPI fmt.Println(strings.Title(source)) // out: TestAPI } ``` Which issue(s) this PR fixes: Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: wujw39640 <wujw39640@hundsun.com> Signed-off-by: wujunwei <wjw3323@live.com> Co-authored-by: wujw39640 <wujw39640@hundsun.com>	3 years ago
Ashwanth	53e01a795a	tsdb: build tsdb from tenantheads (#7036 ) What this PR does / why we need it: refactors tsdb manager code to expose a new method `BuildFromHead` which comes in handy for building tsdb from tenant heads after rotation. Currently we build it by recovering head from WAL even when we have access to the in-mem tenant heads. Special notes for your reviewer: using `loki_tsdb_head_rotations_` metric to track loop failures which could be caused either by rotation failures or if rotation is blocked by prev head failing to build Checklist* - [ ] Documentation added - [x] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	3 years ago
Michel Hollands	34c2c0c4a9	Update metrics descriptions (#6982 ) What this PR does / why we need it: The docs team reviewed and fixed the descriptions of the deletion metrics added to the documentation in https://github.com/grafana/loki/pull/6971. The fix is in a separate PR https://github.com/grafana/loki/pull/6976. This PR fixes that same description in the code. Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Adam	d69d3b664c	Deprecation Fix: Replace strings.Title with cases.Title().String() (#7048 ) Signed-off-by: wujunwei <wjw3323@live.com> <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: [Golang 1.18 deprecates the `strings.title function`](https://github.com/golang/go/issues/48367) use `cases.Title(language.Und, cases.NoLower).String()` instead. Which issue(s) this PR fixes: Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: wujw39640 <wujw39640@hundsun.com>	3 years ago
Trevor Whitney	c0eb73c34a	Enable build in env w/o internet or GCP creds (#7029 ) What this PR does / why we need it: The build is currently dependent upon you building in an environment with internet connectivity and GCP credentials. This PR fixes the few parts of our code and test that rely on these external dependencies so it can be built in an environment without internet or GCP credentials (such as when building a [nix flake](https://github.com/grafana/loki/pull/7020)) Which issue(s) this PR fixes: Fixes #7028	3 years ago
Ashwanth	5ea38fbd51	tsdb: build rotated heads asynchronously (#6956 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: Currently `Append` call to the head has to wait for block creation to complete when rotating head. And since we rotate head every 15m (default period), this would result in latency spikes at these times. This PR moves the TSDB build step to an async routine. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
李国忠	3296c6dbe8	[new feature] logql engine support exec `vector(0)` grama (#7023 ) What this PR does / why we need it: logql engine support exec `vector(0)` grama. new PR of :https://github.com/grafana/loki/pull/7007 Which issue(s) this PR fixes: Fixes #6946 Special notes for your reviewer: test passed. test1: vectorExpr only ![image](https://user-images.githubusercontent.com/9583245/187871060-2ce63f01-5613-4a1e-9a52-59712f32cc7c.png) test2: binExpr and vectorExpr ![image](https://user-images.githubusercontent.com/9583245/188090201-702c1ec4-7c37-41cb-9f2f-8f137a39dadf.png) Checklist - [ ] Documentation added - [x] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Travis Patterson	bc16450349	Revert "Hide duplicates caused by stream sharding (#7005 )" (#7031 ) @owen-d had a suggestion that was dramatically dimpler. Revert this whole thing and the simplified PR will be forthcoming	3 years ago
李国忠	25cc9402b7	fix test increase timeout (#7022 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: increase timeout, fix test fail. ``` --- FAIL: TestInstanceFor (0.01s) downstreamer_test.go:205: Error Trace: downstreamer_test.go:205 downstreamer_test.go:256 Error: lock couldn't be acquired Test: TestInstanceFor FAIL ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
李国忠	6f4671a572	[new feature] syntax: support "vector" grama (#7007 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: Implement vector() function. The "vector function" feature is divided into multiple PRs to avoid excessively large PRs resulting in too long review time. Which issue(s) this PR fixes: Fixes #6946 Special notes for your reviewer: The test for syntax has passed ![image](https://user-images.githubusercontent.com/9583245/187588436-7c2c401f-1d88-4bff-a41a-965d36454ebe.png) <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
sardismart	821301b95f	azblob: Make azblob support private cloud endpoint suffix (#6842 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: The endpoint suffix of private cloud(azure stack hub) azblob is different from Azure. The loki configuration has already support customize endpoint suffix but not use when select blob/container URL. This PR introduce following change: * Consider endpoint suffix from configuration when select blob or container URL format Which issue(s) this PR fixes: Fixes #6863 Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [x] Documentation added - [x] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago

1 2 3 4 5 ...

1726 Commits (8868957783b7fb5aa1384d57b4584f19ee256999)