prometheus

Commit Graph

Author	SHA1	Message	Date
beorn7	0e3f35324b	scrape: Enable ingestion of multiple exemplars per sample This has become a requirement for native histograms, as a single histogram sample commonly has many buckets, so that providing many exemplars makes sense. Since OM text doesn't support native histograms yet, the test had to be expanded to also support protobuf test cases. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Jonatan Ivanov	9225d118dc	Add Exemplars support for all time series This change removes restrictions to allow adding exemplars to all time series. It also contains some improvements in test values so that it is easier to track what is tested. The advantage of doing this is having a little less error-prone tests: "yy" is not really descriptive but "counter-test" can give people a better idea about what is tested so it is harder to make mistakes. Closes gh-11982 Signed-off-by: Jonatan Ivanov <jonatan.ivanov@gmail.com>	3 years ago
Damien Grisonnet	0fee615719	model/textparse: improve openmetrics error outputs Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>	3 years ago
Damien Grisonnet	e3b5416d05	model/textparse: improve error outputs Parsing errors in the Prometheus HTTP format parser are very hard to investigate since they only approximately indicate what is going wrong in the parser and don't provide any information about the incorrect input. As such it is very hard to tell what is wrong in the format exposed by the application. Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>	3 years ago
beorn7	3e5ad99c33	textparse: Minor doc comment improvement Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Bryan Boreham	1f04899ae3	Update package model/textparse for new labels.Labels type Parse metrics using labels.ScratchBuilder, so we reduce assumptions about internals of Labels. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Marc Tudurí	f7df3b86ba	histograms: parse float histograms from proto definition (#11149 ) * histograms: parse float histograms from proto definition Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Improve comment Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Ignore float buckets Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Refactor Histogram() function Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Fix test_float_histogram Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Update model/textparse/protobufparse.go Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update protobufparse.go Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update scrape.go Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update scrape/scrape.go Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> Signed-off-by: Marc Tuduri <marctc@protonmail.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>	3 years ago
Matthieu MOREL	c2b4de3611	refactor (package model): move from github.com/pkg/errors to 'errors' and 'fmt' packages (#10747 ) Signed-off-by: Matthieu MOREL <mmorel-35@users.noreply.github.com> Co-Authored-By: Julien Pivotto <roidelapluie@gmail.com> Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>	4 years ago
Peter Štibraný	cd18da3605	Fix OpenMetrics parser to sort uppercase labels correctly. (#10510 ) Signed-off-by: Peter Štibraný <pstibrany@gmail.com>	4 years ago
Levi Harrison	a9e2ffb717	Specify type in metadata parser errors (#10269 )	4 years ago
beorn7	c954cd9d1d	Move packages out of deprecated pkg directory This creates a new `model` directory and moves all data-model related packages over there: exemplar labels relabel rulefmt textparse timestamp value All the others are more or less utilities and have been moved to `util`: gate logging modetimevfs pool runtime Signed-off-by: beorn7 <beorn@grafana.com>	4 years ago
beorn7	7a8bb8222c	Style cleanup of all the changes in sparsehistogram so far A lot of this code was hacked together, literally during a hackathon. This commit intends not to change the code substantially, but just make the code obey the usual style practices. A (possibly incomplete) list of areas: * Generally address linter warnings. * The `pgk` directory is deprecated as per dev-summit. No new packages should be added to it. I moved the new `pkg/histogram` package to `model` anticipating what's proposed in #9478. * Make the naming of the Sparse Histogram more consistent. Including abbreviations, there were just too many names for it: SparseHistogram, Histogram, Histo, hist, his, shs, h. The idea is to call it "Histogram" in general. Only add "Sparse" if it is needed to avoid confusion with conventional Histograms (which is rare because the TSDB really has no notion of conventional Histograms). Use abbreviations only in local scope, and then really abbreviate (not just removing three out of seven letters like in "Histo"). This is in the spirit of https://github.com/golang/go/wiki/CodeReviewComments#variable-names * Several other minor name changes. * A lot of formatting of doc comments. For one, following https://github.com/golang/go/wiki/CodeReviewComments#comment-sentences , but also layout question, anticipating how things will look like when rendered by `godoc` (even where `godoc` doesn't render them right now because they are for unexported types or not a doc comment at all but just a normal code comment - consistency is queen!). * Re-enabled `TestQueryLog` and `TestEndopints` (they pass now, leaving them disabled was presumably an oversight). * Bucket iterator for histogram.Histogram is now created with a method. * HistogramChunk.iterator now allows iterator recycling. (I think @dieterbe only commented it out because he was confused by the question in the comment.) * HistogramAppender.Append panics now because we decided to treat staleness marker differently. Signed-off-by: beorn7 <beorn@grafana.com>	4 years ago
Bryan Boreham	92a3eeac55	Create less garbage when parsing metrics (#9299 ) * Refactor: extract function to make scrapeLoop for testing Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Add benchmarks for ScrapeLoopAppend For Prometheus and OpenMetrics Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Create less garbage when parsing metrics Exemplar escapes to heap due to being passed through text-parser interface, but we can reduce the impact by hoisting it out of the loop and resetting it after every use. (Note the cost was paid on every line even when exemplars were disabled) Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Create less garbage when parsing OpenMetrics After calling parseLVals() we always append the return value, so pass in what we want to append it to and save garbage. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	4 years ago
beorn7	5de2df752f	Hacky implementation of protobuf parsing This "brings back" protobuf parsing, with the only goal to play with the new sparse histograms. The Prom-2.x style parser is highly adapted to the structure of the Prometheus text format (and later OpenMetrics). Some jumping through hoops is required to feed protobuf into it. This is not meant to be a model for the final implementation. It should just enable sparse histogram ingestion at a reasonable efficiency. Following known shortcomings and flaws: - No tests yet. - Summaries and legacy histograms, i.e. without sparse buckets, are ignored. - Staleness doesn't work (but this could be fixed in the appender, to be discussed). - No tricks have been tried that would be similar to the tricks the text parsers do (like direct pointers into the HTTP response body). That makes things weird here. Tricky optimizations only make sense once the final format is specified, which will almost certainly not be the old protobuf format. (Interestingly, I expect this implementation to be in fact much more efficient than the original protobuf ingestion in Prom-1.x.) - This is using a proto3 version of metrics.proto (mostly to be consistent with the other protobuf uses). However, proto3 sees no difference between an unset field. We depend on that to distinguish between an unset timestamp and the timestamp 0 (1970-01-01, 00:00:00 UTC). In this experimental code, we just assume that timestamp is never specified and therefore a timestamp of 0 always is interpreted as "not set". Signed-off-by: beorn7 <beorn@grafana.com>	5 years ago
cw9	13cd50c52e	Return error on NaN and Inf timestamps for OpenMetrics parser Signed-off-by: cw9 <chaow929@gmail.com>	5 years ago
Ben Kochie	269e7c8091	Fix golint issues. Signed-off-by: Ben Kochie <superq@gmail.com>	6 years ago
Julien Pivotto	40b66d29ff	OpenMetrics parse: avoid setting prev token (#6781 ) We can avoid setting a prev token in the OM parser. The previous coundition that checked for prev was unreacheable. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	6 years ago
Julien Pivotto	9398b9ac1b	OpenMetrics: force input to end with # EOF (#6505 ) Fixes #1169 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	6 years ago
shreyassrivatsan	e825282dd1	Add exemplar support to the openmetrics parser (#6292 ) * Add exemplar support to the openmetrics parser Signed-off-by: Shreyas Srivatsan <shreyas@chronosphere.io>	6 years ago
zhulongcheng	970ef41bb6	Fix exported function comments (#6002 ) Signed-off-by: zhulongcheng <zhulongcheng.dev@gmail.com>	6 years ago
Brian Brazil	94b1af1471	Update to disallow new Go 1.13 float formats. (#5984 ) Don't inadvertantly expand the acceptable ways to specify a float. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	c66bfce8cf	Don't panic on short input. (#5939 ) Fixes #5935 Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Tariq Ibrahim	8fdfa8abea	refine error handling in prometheus (#5388 ) i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors. ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives. iii) Does away with the use of fmt package for errors in favour of pkg/errors Signed-off-by: tariqibrahim <tariq181290@gmail.com>	7 years ago
AixesHunter	9f903fb3f7	github.com/cznic/golex has moved to modernc.org/golex (#5034 ) Signed-off-by: aixeshunter <aixeshunter@gmail.com>	7 years ago
Brian Brazil	3fdb92010e	Expand OM to OpenMetrics Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	7 years ago
Brian Brazil	c271ef456a	Add an OpenMetrics parser. This is based on the existing parser, and has slightly better performance (few %) than it. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	7 years ago
Brian Brazil	ffe7efb411	Prepare for multiple text formats Pass content type down to text parser. Add layer of indirection in front of text parser, and rename to avoid future clashes. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	7 years ago
Brian Brazil	70c98a06f1	Handle empty HELP in text parser. (#4444 ) Fixes #4427 Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	7 years ago
Fabian Reinartz	0ff42e754e	pkg/textparse: unescape help string Signed-off-by: Fabian Reinartz <freinartz@google.com>	8 years ago
Fabian Reinartz	5fcc5b6028	pkg/textparse: support null bytes in comments and help Signed-off-by: Fabian Reinartz <freinartz@google.com>	8 years ago
Fabian Reinartz	76a4a46cb0	pkg/textparse: refactor and add metadata handling Extends the parser to allow retrieving metadata. The lexer now yields proper tokens that are fed into a hand-written parser on top. Signed-off-by: Fabian Reinartz <freinartz@google.com>	8 years ago
Goutham Veeramachaneni	11ee713ced	Handle common escape characters properly. (#2995 ) Fixes #2974 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	b7eddbcd98	textparse: Add fuzzing and fix bug caught See https://github.com/cznic/golex/issues/11 for info on the bug Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	0e78b7e7c8	parser: support spaces everywhere Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Fabian Reinartz	bc7aff8cef	retrieval: extract scrape cache	9 years ago
Fabian Reinartz	a83014f53c	retrieval: fix memory leak and consumption for caches	9 years ago
Fabian Reinartz	377886b371	pkg/textparse: implement timestamp parsing	9 years ago
Brian Brazil	5c9a6ce747	Add license to files. This should fix CI for dev-2.0.	9 years ago
Fabian Reinartz	d9fb57cde4	*: Simplify []byte to string unsafe conversion	9 years ago
Fabian Reinartz	d80a3de235	pkg/textparse: add documentation	9 years ago
Fabian Reinartz	db48726a6b	pkg/textparse: allocate single string per metric	9 years ago
Fabian Reinartz	e44d80314d	pkg/textparse: add tests and method to retrieve full labels	9 years ago
Fabian Reinartz	091a7f2395	pkg/textparse: add initial text parser	9 years ago

15 Commits (c58e20ad0eb4d2fbce202ae537b2bc81509ec420)