apitech
/
loki
mirror of https://github.com/grafana/loki
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Like Prometheus, but for logs.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
10873 Commits
1073 Branches
354 Tags
737 MiB
 
 
 
 
 
 
Tag: Branch: Tree: e08721487c
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'e08721487c'
${ noResults }
loki/pkg/chunkenc
History
Ed Welch f7da6e2869
chore: optimize with a new pipeline specifically when reordering chunks as we write them (#19211) 		3 months ago
..
testdata
…
README.md chore: Update ASCII diagram of chunk binary encoding (#15386) 1 year ago
dumb_chunk.go chore(linter): Fix issues flagged by updated golangci-lint version (#19206) 4 months ago
encoding_helpers.go
…
facade.go chore: optimize chunk rewrite while filtering out deleted data (#19140) 4 months ago
hash_test.go chore: update loki modules for 3.0 release (#12433) 2 years ago
interface.go chore: optimize chunk rewrite while filtering out deleted data (#19140) 4 months ago
interface_test.go chore: Move compression utilities into separate package (#14167) 1 year ago
memchunk.go chore: optimize with a new pipeline specifically when reordering chunks as we write them (#19211) 3 months ago
memchunk_test.go chore(linter): Fix issues flagged by updated golangci-lint version (#19206) 4 months ago
pool.go chore: Move compression utilities into separate package (#14167) 1 year ago
symbols.go chore: upgrade Prometheus to 208187eaa19b (#18756) 5 months ago
symbols_test.go chore: upgrade Prometheus to 208187eaa19b (#18756) 5 months ago
unordered.go chore: upgrade Prometheus to 208187eaa19b (#18756) 5 months ago
unordered_test.go chore: consistently use new labels.Labels API (#18490) 6 months ago
util_test.go chore: Linting update for new golangci (#16572) 10 months ago
variants.go refactor(stringlabels): Support stringlabels in `logql/log` package (#17838) 7 months ago

README.md

Chunk v4 format

// Header
+-----------------------------------+
| Magic Number (uint32, 4 bytes)    |
+-----------------------------------+
| Version (1 byte)                  |
+-----------------------------------+
| Encoding (1 byte)                 |
+-----------------------------------+

// Blocks
+--------------------+----------------------------+
| block 1 (n bytes)  | checksum (uint32, 4 bytes) |
+--------------------+----------------------------+
| block 1 (n bytes)  | checksum (uint32, 4 bytes) |
+--------------------+----------------------------+
| ...                                             |
+--------------------+----------------------------+
| block N (n bytes)  | checksum (uint32, 4 bytes) |
+--------------------+----------------------------+

// Metas
+------------------------------------------------------------------------------------------------------------------------+
| #blocks (uvarint)                                                                                                      |
+--------------------+-----------------+-----------------+------------------+---------------+----------------------------+
| #entries (uvarint) | minTs (uvarint) | maxTs (uvarint) | offset (uvarint) | len (uvarint) | uncompressedSize (uvarint) |
+--------------------+-----------------+-----------------+------------------+---------------+----------------------------+
| #entries (uvarint) | minTs (uvarint) | maxTs (uvarint) | offset (uvarint) | len (uvarint) | uncompressedSize (uvarint) |
+--------------------+-----------------+-----------------+------------------+---------------+----------------------------+
| ...                                                                                                                    |
+--------------------+-----------------+-----------------+------------------+---------------+----------------------------+
| #entries (uvarint) | minTs (uvarint) | maxTs (uvarint) | offset (uvarint) | len (uvarint) | uncompressedSize (uvarint) |
+--------------------+-----------------+-----------------+------------------+---------------+----------------------------+
| checksum (uint32, 4 bytes)                                                                                             | 
+------------------------------------------------------------------------------------------------------------------------+

// Structured Metadata
+---------------------------------+
| #labels (uvarint)               |
+---------------+-----------------+
| len (uvarint) | value (n bytes) |
+---------------+-----------------+
| ...                             |
+---------------+-----------------+
| checksum (uint32, 4 bytes)      |
+---------------------------------+

// Footer
+-----------------------+--------------------------+
| len (uint64, 8 bytes) | offset (uint64, 8 bytes) |   // offset to Structured Metadata
+-----------------------+--------------------------+
| len (uint64, 8 bytes) | offset (uint64, 8 bytes) |   // offset to Metas
+-----------------------+--------------------------+