prometheus

Commit Graph

Author	SHA1	Message	Date
Ondrej Kokes	2e30f1231b	docs: textparse.Parser return type mismatch The docs suggest the Next method returns a bool, but that's not the case (`Entry` is an int). ``` // Next advances the parser to the next sample. It returns false if no // more samples were read or an error occurred. Next() (Entry, error) ``` The docs were first added in `d80a3de235` in 2017. Back then the signature was indeed `func (p *Parser) Next() bool`. But then it got refactored in `76a4a46cb0` and the signature changed with it, yet docs stayed the same - and eventually made their way into the `Parser` interface. However, the Protobuf parser does have the right wording: `5de2df752f` ``` // Next advances the parser to the next "sample" (emulating the behavior of a // text format parser). It returns (EntryInvalid, io.EOF) if no samples were // read. ``` Changing all other implementations (and the interface itself) to match this doc. Signed-off-by: Ondrej Kokes <ondrej.kokes@gmail.com>	2 years ago
Bryan Boreham	eff3a13e19	model/textparse: parsers take a labels SymbolTable This allows strings to be interned to save memory. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Owen Williams	a28d7865ad	UTF-8: Add support for parsing UTF8 metric and label names This adds support for the new grammar of `{"metric_name", "l1"="val"}` to promql and some of the exposition formats. This grammar will also be valid for non-UTF-8 names. UTF-8 names will not be considered valid unless model.NameValidationScheme is changed. This does not update the go expfmt parser in text_parse.go, which will be addressed by https://github.com/prometheus/common/issues/554/. Part of https://github.com/prometheus/prometheus/issues/13095 Signed-off-by: Owen Williams <owen.williams@grafana.com>	2 years ago
Bryan Boreham	c83e1fc574	textparse: remove MetricType alias No backwards-compatibility; make a clean break. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	8065bef172	Move metric type definitions to common/model They are used in multiple repos, so common is a better place for them. Several packages now don't depend on `model/textparse`, e.g. `storage/remote`. Also remove `metadata` struct from `api.go`, since it was identical to a struct in the `metadata` package. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Arthur Silva Sens	5082655392	Append Created Timestamps (#12733 ) * Append created timestamps. Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Log when created timestamps are ignored Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Proposed changes to Append CT PR. Changes: * Changed textparse Parser interface for consistency and robustness. * Changed CT interface to be more explicit and handle validation. * Simplified test, change scrapeManager to allow testability. * Added TODOs. Signed-off-by: bwplotka <bwplotka@gmail.com> * Updates. Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * Refactor head_appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Fix linter issues Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Use model.Sample in head appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> --------- Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: bwplotka <bwplotka@gmail.com>	2 years ago
Arthur Silva Sens	ef8e6ae780	Parse created timestamps from Prometheus Protobuf (#12973 ) Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>	2 years ago
Matthieu MOREL	0a513f827d	ci(lint): enable errorlint linter on model Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
beorn7	c3c7d44d84	lint: Adjust to the lint warnings raised by current versions of golint-ci We haven't updated golint-ci in our CI yet, but this commit prepares for that. There are a lot of new warnings, and it is mostly because the "revive" linter got updated. I agree with most of the new warnings, mostly around not naming unused function parameters (although it is justified in some cases for documentation purposes – while things like mocks are a good example where not naming the parameter is clearer). I'm pretty upset about the "empty block" warning to include `for` loops. It's such a common pattern to do something in the head of the `for` loop and then have an empty block. There is still an open issue about this: https://github.com/mgechev/revive/issues/810 I have disabled "revive" altogether in files where empty blocks are used excessively, and I have made the effort to add individual `// nolint:revive` where empty blocks are used just once or twice. It's borderline noisy, though, but let's go with it for now. I should mention that none of the "empty block" warnings for `for` loop bodies were legitimate. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Damien Grisonnet	0fee615719	model/textparse: improve openmetrics error outputs Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>	3 years ago
Damien Grisonnet	e3b5416d05	model/textparse: improve error outputs Parsing errors in the Prometheus HTTP format parser are very hard to investigate since they only approximately indicate what is going wrong in the parser and don't provide any information about the incorrect input. As such it is very hard to tell what is wrong in the format exposed by the application. Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>	3 years ago
beorn7	3e5ad99c33	textparse: Minor doc comment improvement Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Damien Grisonnet	96b9d8cebb	model/textparse: associate correct token to errors In some cases, the Prometheus HTTP format parser was not returning the right token in the error output which made debugging impossible. Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>	3 years ago
Bryan Boreham	1f04899ae3	Update package model/textparse for new labels.Labels type Parse metrics using labels.ScratchBuilder, so we reduce assumptions about internals of Labels. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Marc Tudurí	f7df3b86ba	histograms: parse float histograms from proto definition (#11149 ) * histograms: parse float histograms from proto definition Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Improve comment Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Ignore float buckets Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Refactor Histogram() function Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Fix test_float_histogram Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Update model/textparse/protobufparse.go Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update protobufparse.go Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update scrape.go Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update scrape/scrape.go Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> Signed-off-by: Marc Tuduri <marctc@protonmail.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>	3 years ago
Matthieu MOREL	c2b4de3611	refactor (package model): move from github.com/pkg/errors to 'errors' and 'fmt' packages (#10747 ) Signed-off-by: Matthieu MOREL <mmorel-35@users.noreply.github.com> Co-Authored-By: Julien Pivotto <roidelapluie@gmail.com> Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>	4 years ago
Levi Harrison	a9e2ffb717	Specify type in metadata parser errors (#10269 )	4 years ago
beorn7	c954cd9d1d	Move packages out of deprecated pkg directory This creates a new `model` directory and moves all data-model related packages over there: exemplar labels relabel rulefmt textparse timestamp value All the others are more or less utilities and have been moved to `util`: gate logging modetimevfs pool runtime Signed-off-by: beorn7 <beorn@grafana.com>	4 years ago
beorn7	7a8bb8222c	Style cleanup of all the changes in sparsehistogram so far A lot of this code was hacked together, literally during a hackathon. This commit intends not to change the code substantially, but just make the code obey the usual style practices. A (possibly incomplete) list of areas: * Generally address linter warnings. * The `pgk` directory is deprecated as per dev-summit. No new packages should be added to it. I moved the new `pkg/histogram` package to `model` anticipating what's proposed in #9478. * Make the naming of the Sparse Histogram more consistent. Including abbreviations, there were just too many names for it: SparseHistogram, Histogram, Histo, hist, his, shs, h. The idea is to call it "Histogram" in general. Only add "Sparse" if it is needed to avoid confusion with conventional Histograms (which is rare because the TSDB really has no notion of conventional Histograms). Use abbreviations only in local scope, and then really abbreviate (not just removing three out of seven letters like in "Histo"). This is in the spirit of https://github.com/golang/go/wiki/CodeReviewComments#variable-names * Several other minor name changes. * A lot of formatting of doc comments. For one, following https://github.com/golang/go/wiki/CodeReviewComments#comment-sentences , but also layout question, anticipating how things will look like when rendered by `godoc` (even where `godoc` doesn't render them right now because they are for unexported types or not a doc comment at all but just a normal code comment - consistency is queen!). * Re-enabled `TestQueryLog` and `TestEndopints` (they pass now, leaving them disabled was presumably an oversight). * Bucket iterator for histogram.Histogram is now created with a method. * HistogramChunk.iterator now allows iterator recycling. (I think @dieterbe only commented it out because he was confused by the question in the comment.) * HistogramAppender.Append panics now because we decided to treat staleness marker differently. Signed-off-by: beorn7 <beorn@grafana.com>	4 years ago
beorn7	5de2df752f	Hacky implementation of protobuf parsing This "brings back" protobuf parsing, with the only goal to play with the new sparse histograms. The Prom-2.x style parser is highly adapted to the structure of the Prometheus text format (and later OpenMetrics). Some jumping through hoops is required to feed protobuf into it. This is not meant to be a model for the final implementation. It should just enable sparse histogram ingestion at a reasonable efficiency. Following known shortcomings and flaws: - No tests yet. - Summaries and legacy histograms, i.e. without sparse buckets, are ignored. - Staleness doesn't work (but this could be fixed in the appender, to be discussed). - No tricks have been tried that would be similar to the tricks the text parsers do (like direct pointers into the HTTP response body). That makes things weird here. Tricky optimizations only make sense once the final format is specified, which will almost certainly not be the old protobuf format. (Interestingly, I expect this implementation to be in fact much more efficient than the original protobuf ingestion in Prom-1.x.) - This is using a proto3 version of metrics.proto (mostly to be consistent with the other protobuf uses). However, proto3 sees no difference between an unset field. We depend on that to distinguish between an unset timestamp and the timestamp 0 (1970-01-01, 00:00:00 UTC). In this experimental code, we just assume that timestamp is never specified and therefore a timestamp of 0 always is interpreted as "not set". Signed-off-by: beorn7 <beorn@grafana.com>	5 years ago
Ben Kochie	269e7c8091	Fix golint issues. Signed-off-by: Ben Kochie <superq@gmail.com>	6 years ago
shreyassrivatsan	e825282dd1	Add exemplar support to the openmetrics parser (#6292 ) * Add exemplar support to the openmetrics parser Signed-off-by: Shreyas Srivatsan <shreyas@chronosphere.io>	6 years ago
zhulongcheng	970ef41bb6	Fix exported function comments (#6002 ) Signed-off-by: zhulongcheng <zhulongcheng.dev@gmail.com>	6 years ago
Brian Brazil	94b1af1471	Update to disallow new Go 1.13 float formats. (#5984 ) Don't inadvertantly expand the acceptable ways to specify a float. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Tariq Ibrahim	8fdfa8abea	refine error handling in prometheus (#5388 ) i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors. ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives. iii) Does away with the use of fmt package for errors in favour of pkg/errors Signed-off-by: tariqibrahim <tariq181290@gmail.com>	7 years ago
Giedrius Statkevičius	10ec2552af	promparse: sort all labels when parsing (#5372 ) * promparse: sort all labels when parsing Some label names might start with an uppercase letter in which case it needs to come before __name__. Without this it means that we are not maintaining the same invariant that all label names should be sorted in such cases. Amend the tests to check this problem automatically. Without the change the tests do not pass. Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>	7 years ago
AixesHunter	9f903fb3f7	github.com/cznic/golex has moved to modernc.org/golex (#5034 ) Signed-off-by: aixeshunter <aixeshunter@gmail.com>	7 years ago
Brian Brazil	c271ef456a	Add an OpenMetrics parser. This is based on the existing parser, and has slightly better performance (few %) than it. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	7 years ago
Brian Brazil	ffe7efb411	Prepare for multiple text formats Pass content type down to text parser. Add layer of indirection in front of text parser, and rename to avoid future clashes. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	7 years ago
Brian Brazil	70c98a06f1	Handle empty HELP in text parser. (#4444 ) Fixes #4427 Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	7 years ago
Fabian Reinartz	0ff42e754e	pkg/textparse: unescape help string Signed-off-by: Fabian Reinartz <freinartz@google.com>	8 years ago
Fabian Reinartz	5fcc5b6028	pkg/textparse: support null bytes in comments and help Signed-off-by: Fabian Reinartz <freinartz@google.com>	8 years ago
Fabian Reinartz	76a4a46cb0	pkg/textparse: refactor and add metadata handling Extends the parser to allow retrieving metadata. The lexer now yields proper tokens that are fed into a hand-written parser on top. Signed-off-by: Fabian Reinartz <freinartz@google.com>	8 years ago
Goutham Veeramachaneni	11ee713ced	Handle common escape characters properly. (#2995 ) Fixes #2974 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	b7eddbcd98	textparse: Add fuzzing and fix bug caught See https://github.com/cznic/golex/issues/11 for info on the bug Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	0e78b7e7c8	parser: support spaces everywhere Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Fabian Reinartz	bc7aff8cef	retrieval: extract scrape cache	9 years ago
Fabian Reinartz	a83014f53c	retrieval: fix memory leak and consumption for caches	9 years ago
Fabian Reinartz	377886b371	pkg/textparse: implement timestamp parsing	9 years ago
Brian Brazil	5c9a6ce747	Add license to files. This should fix CI for dev-2.0.	9 years ago
Fabian Reinartz	d9fb57cde4	*: Simplify []byte to string unsafe conversion	9 years ago
Fabian Reinartz	d80a3de235	pkg/textparse: add documentation	9 years ago
Fabian Reinartz	db48726a6b	pkg/textparse: allocate single string per metric	9 years ago
Fabian Reinartz	e44d80314d	pkg/textparse: add tests and method to retrieve full labels	9 years ago
Fabian Reinartz	091a7f2395	pkg/textparse: add initial text parser	9 years ago

21 Commits (5aa3d8260a7335a0da6946dea82f608cafa46c20)