prometheus

Commit Graph

Author	SHA1	Message	Date
Tom Wilkie	d479151f1f	Various enhancements and refactorings for remote write receiver: - Remove unrelated changes - Refactor code out of the API module - that is already getting pretty crowded. - Don't track reference for AddFast in remote write. This has the potential to consume unlimited server-side memory if a malicious client pushes a different label set for every series. For now, its easier and safer to always use the 'slow' path. - Return 400 on out of order samples. - Use remote.DecodeWriteRequest in the remote write adapters. - Put this behing the 'remote-write-server' feature flag - Add some (very) basic docs. - Used named return & add test for commit error propagation Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	5 years ago
Julien Pivotto	4e5b1722b3	Move away from testutil, refactor imports (#8087 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Marek Slabicki	8224ddec23	Capitalizing first letter of all log lines (#7043 ) Signed-off-by: Marek Slabicki <thaniri@gmail.com>	6 years ago
Vadym Martsynovskyy	a9970a47ef	Fix incorrect examples in docs Signed-off-by: Vadym Martsynovskyy <vmartsynovskyy@gmail.com>	6 years ago
Romain Baugue	f2ca36e56a	Exhaust every request body before closing it (#5166 ) (#5479 ) From the documentation: > The default HTTP client's Transport may not > reuse HTTP/1.x "keep-alive" TCP connections if the Body is > not read to completion and closed. This effectively enable keep-alive for the fixed requests. Signed-off-by: Romain Baugue <romain.baugue@elwinar.com>	7 years ago
Romain Baugue	95193fa027	Exhaust every request body before closing it (#5166 ) (#5479 ) From the documentation: > The default HTTP client's Transport may not > reuse HTTP/1.x "keep-alive" TCP connections if the Body is > not read to completion and closed. This effectively enable keep-alive for the fixed requests. Signed-off-by: Romain Baugue <romain.baugue@elwinar.com>	7 years ago
qinng	cc75c27580	Fix multiple response.WriteHeader calls error in remote read adapter (#5159 ) * fix multiple response.WriteHeader calls in remote read adapter * remove useless return Signed-off-by: qinng <guoruyi1@xiaomi.com>	7 years ago
Tariq Ibrahim	8fdfa8abea	refine error handling in prometheus (#5388 ) i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors. ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives. iii) Does away with the use of fmt package for errors in favour of pkg/errors Signed-off-by: tariqibrahim <tariq181290@gmail.com>	7 years ago
LongKB	23480bef43	Remove the duplicated words (#5251 ) Although it is spelling mistakes, it might make an affects while reading. Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com> Signed-off-by: Kim Bao Long <longkb@vn.fujitsu.com>	7 years ago
Callum Styan	5358f76c5c	update remote write path proto so that Labels/Timeseries can't be nil (#4957 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	7 years ago
Alex Yu	5dcce32ef8	update promlog to latest version (#4876 ) * update promlog to latest version Signed-off-by: Alex Yu <yu.alex96@gmail.com> * Update api tests, fix main setup Signed-off-by: Alex Yu <yu.alex96@gmail.com> * tidy go.sum Signed-off-by: Alex Yu <yu.alex96@gmail.com> * revendor prometheus/common Signed-off-by: Alex Yu <yu.alex96@gmail.com> * only initialize config; use kingpin for remote_storage_adapter Signed-off-by: Alex Yu <yu.alex96@gmail.com> * actually parse the flags Signed-off-by: Alex Yu <yu.alex96@gmail.com> * clean up imports Signed-off-by: Alex Yu <yu.alex96@gmail.com>	7 years ago
Simon Pasquier	ed19373a78	: remove use of golang.org/x/net/context (#4869 ) : remove use of golang.org/x/net/context Signed-off-by: Simon Pasquier <spasquie@redhat.com> scrape: fix TestTargetScrapeScrapeCancel Signed-off-by: Simon Pasquier <spasquie@redhat.com>	7 years ago
Tom Wilkie	d3a1ff1abf	Reduce memory usage of remote read by reducing pointer usage. (#4655 ) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
beorn7	4fb59d1e61	Remove use of deprecated prometheus.Handler Signed-off-by: beorn7 <beorn@soundcloud.com>	7 years ago
Julius Volz	8fbe1b5133	Handle a bunch of unchecked errors (#4461 ) There are many more (mostly finalizers like Close/Stop/etc.), but most of the others seemed like one couldn't do much about them anyway. Signed-off-by: Julius Volz <julius.volz@gmail.com>	7 years ago
Harsh Agarwal	6a464ae174	expose log.level for promlog for remote_storage_adapter (#4195 ) * expose log.level for promlog for remote_storage_adapter Signed-off-by: sipian <cs15btech11019@iith.ac.in> * replace flag description Signed-off-by: Harsh Agarwal <cs15btech11019@iith.ac.in>	8 years ago
Peter Gallerani	a9d5034add	Fix missing 'msg' in remote storage adapter main.go .Log info message (#4377 ) Signed-off-by: Peter Gallerani <peter.gallerani@gmail.com>	8 years ago
ferhat elmas	ffa673f7d8	General simplifications (#3887 ) Another try as in #1516	8 years ago
Matthew Pound	e6dcc72f9a	Fix instructions on updating prometheus.yml for Remote Write Adapter Example (#3422 )	8 years ago
Krasi Georgiev	5d8f93a22a	now using only github.com/gogo/protobuf bumped all grpc-gateway packages to v1.2.2 updated and run the denproto.sh script	8 years ago
Julius Volz	099df0c5f0	Migrate "golang.org/x/net/context" -> "context" (#3333 ) In some places, where ctxhttp or gRPC are concerned, we still need to use the old contexts.	8 years ago
Jack Neely	128b31d058	Log failure to send NaN values to remote store as Debug (#3235 ) This was a warning and can be a frequent occurrence. Let's not fill up logs unless we are asked to.	8 years ago
Goutham Veeramachaneni	f5aed810f9	logging: Port to common/promlog Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	d21f149745	*: migrate to go-kit/log	8 years ago
Julius Volz	aa5cdcb11e	Remove extra space in log output	8 years ago
gdmello	35c952e344	Added logging for remote storage adapter (#3106 ) * Added logging for remote storage adapter on startup and on any error condition during /read or /write. * CR feedback.	8 years ago
Tom Wilkie	cf105f9d57	Update example remote adapters for change in proto location.	9 years ago
Julius Volz	e0f046396a	Fix InfluxDB retention policy usage in read adapter (#2781 )	9 years ago
Tom Wilkie	3141a6b36b	Compress remote storage requests and responses with unframed/raw snappy. (#2696 ) * Compress remote storage requests and responses with unframed/raw snappy, for compatibility with other languages. * Remove backwards compatibility code from remote_storage_adapter, update example_write_adapter * Add /documentation/examples/remote_storage/example_write_adapter/example_writer_adapter to .gitignore	9 years ago
Brian Brazil	0e0fc5a7f4	Correct example name to adapter. (#2590 )	9 years ago
Brian Brazil	c813c824d4	Separate out remote read responses. Fixes #2574	9 years ago
Julius Volz	3581057ea4	Update remote storage bridge README.md	9 years ago
Julius Volz	b391cbb808	Add InfluxDB read-back support to remote storage bridge	9 years ago
Julius Volz	815762a4ad	Move retrieval.NewHTTPClient -> httputil.NewClientFromConfig	9 years ago
Julius Volz	beb3c4b389	Remove legacy remote storage implementations This removes legacy support for specific remote storage systems in favor of only offering the generic remote write protocol. An example bridge application that translates from the generic protocol to each of those legacy backends is still provided at: documentation/examples/remote_storage/remote_storage_bridge See also https://github.com/prometheus/prometheus/issues/10 The next step in the plan is to re-add support for multiple remote storages.	9 years ago
Julius Volz	b16371595d	Add standalone remote storage bridge example In preparation for removing specific remote storage implementations, this offers an example of how to achieve the same in a separate process. Rather than having three separate bridges for OpenTSDB, InfluxDB, and Graphite, I decided to support all in one binary. For now, this is in the example documenation directory, but perhaps we will want to make a first-class project / repository out of it.	9 years ago
Julius Volz	b5163351bf	Simplify and fix remote write example After removing gRPC, this can be simplified again. Also, the configuration for the remote storage moved from flags to the config file.	9 years ago
Tom Wilkie	d83879210c	Switch back to protos over HTTP, instead of GRPC. My aim is to support the new grpc generic write path in Frankenstein. On the surface this seems easy - however I've hit a number of problems that make me think it might be better to not use grpc just yet. The explanation of the problems requires a little background. At weave, traffic to frankenstein need to go through a couple of services first, for SSL and to be authenticated. So traffic goes: internet -> frontend -> authfe -> frankenstein - The frontend is Nginx, and adds/removes SSL. Its done this way for legacy reasons, so the certs can be managed in one place, although eventually we imagine we'll merge it with authfe. All traffic from frontend is sent to authfe. - Authfe checks the auth tokens / cookie etc and then picks the service to forward the RPC to. - Frankenstein accepts the reads and does the right thing with them. First problem I hit was Nginx won't proxy http2 requests - it can accept them, but all calls downstream are http1 (see https://trac.nginx.org/nginx/ticket/923). This wasn't such a big deal, so it now looks like: internet --(grpc/http2)--> frontend --(grpc/http1)--> authfe --(grpc/http1)--> frankenstein Next problem was golang grpc server won't accept http1 requests (see https://groups.google.com/forum/#!topic/grpc-io/JnjCYGPMUms). It is possible to link a grpc server in with a normal go http mux, as long as the mux server is serving over SSL, as the golang http client & server won't do http2 over anything other than an SSL connection. This would require making all our service to service comms SSL. So I had a go a writing a grpc http1 server, and got pretty far. But is was a bit of a mess. So finally I thought I'd make a separate grpc frontend for this, running in parallel with the frontend/authfe combo on a different port - and first up I'd need a grpc reverse proxy. Ideally we'd have some nice, generic reverse proxy that only knew about a map from service names -> downstream service, and didn't need to decode & re-encode every request as it went through. It seems like this can't be done with golang's grpc library - see https://github.com/mwitkow/grpc-proxy/issues/1. And then I was surprised to find you can't do grpc from browsers! See http://www.grpc.io/faq/ - not important to us, but I'm starting to question why we decided to use grpc in the first place? It would seem we could have most of the benefits of grpc with protos over HTTP, and this wouldn't preclude moving to grpc when its a bit more mature? In fact, the grcp FAQ even admits as much: > Why is gRPC better than any binary blob over HTTP/2? > This is largely what gRPC is on the wire.	9 years ago
Julius Volz	aa3f2b7216	Generic write cleanups and changes. - fold metric name into labels - return initialization errors back to main - add snappy compression - better context handling - pre-allocation of labels - remove generic naming - other cleanups	9 years ago

41 Commits (2d9f3e34ddce2d3323b1e7ea69d1cbe9d2e140dc)