55 Commits (0f01f07a88e64fd53efdc9bbf03c82449b52a363)

Author	SHA1	Message	Date
Paul Rogers	f2c2a22bdc	chore: Preparation for incoming static code analysis CI check (#15164) ... Co-authored-by: Danny Cooper <danny.cooper@grafana.com>	6 months ago
Emad Mohamadi	9267ee3561	fix(canary): Reconnect immediately upon tail max duration (#14287)	8 months ago
Paul Rogers	18cef217fc	chore: Linting updates in prep for Go 1.23 (#14071)	9 months ago
renovate[bot]	d43b2de1b4	fix(deps): update module github.com/fsouza/fake-gcs-server to v1.47.7 (#13935) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	9 months ago
Trevor Whitney	c6ab6b31e8	chore: remove initial metric aggregation experiment (#13729)	9 months ago
Marco Pracucci	b88583da7d	feat: upgrade prometheus (#13671) ... Signed-off-by: Marco Pracucci <marco@pracucci.com> Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	10 months ago
Trevor Whitney	467eb1bb1b	feat: collect and serve pre-aggregated bytes and counts (#13020) ... Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	11 months ago
Kaviraj Kanagaraj	71507a2b64	feat(canary): Add test to check query results with and without cache. (#13104) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	12 months ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Kevin Burke	9dad6a1350	clients,pkg: use http.NewRequestWithContext (#10616) ... **What this PR does / why we need it**: It saves one line, and the new function (vs. http.NewRequest) performs additional validation of the request. **Which issue(s) this PR fixes**: None **Special notes for your reviewer**: None **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) Not necessary: - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Charles Korn	8e6a5433ac	Upgrade dskit and remove dependency on weaveworks/common (#10220) ... **What this PR does / why we need it**: This PR upgrades `github.com/grafana/dskit` removes Loki's dependency on `github.com/weaveworks/common`. The changes in dskit, apart from the migration (https://github.com/grafana/dskit/pull/356), are: * https://github.com/grafana/dskit/pull/347 * https://github.com/grafana/dskit/pull/349, which required small changes in `pkg/canary/reader/reader.go` and `pkg/canary/writer/push.go` * https://github.com/grafana/dskit/pull/352 * https://github.com/grafana/dskit/pull/354 **Which issue(s) this PR fixes**: (none) **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Charles Korn	bde65667f7	Upgrade dskit to use packages migrated from weaveworks/common (#10164) ... **What this PR does / why we need it**: This PR upgrades dskit and replaces use of packages from weaveworks/common with their migrated equivalents in dskit. See https://github.com/grafana/dskit/pull/342 for more details. Note that Loki still uses some packages from weaveworks/common that I haven't migrated (`aws` and `test`) - I'll migrate these separately. If this PR needs to be rebuilt, I used `rewrite.sh` ([source](https://gist.github.com/charleskorn/48efe62a09d6d70f3de30327003df5c5#file-rewrite-sh)) to generate most of these changes. **Which issue(s) this PR fixes**: (none) **Special notes for your reviewer**: (none) **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	2 years ago
Trevor Whitney	4a56445686	Upgrade `golangci-lint` and fix linting errors (#9601) ... **What this PR does / why we need it**: Upgrade `golangci-lint` and fixes all the errors. The upgrade includes some stricter linting.	2 years ago
Ed Welch	79c5fdd9c0	Canary: Refactor how the direct Canary push works (#9569) ... **What this PR does / why we need it**: When doing some testing a while back I ran into several cases were we didn't handle errors/retries properly in the code we added for the canaries to send directly to Loki. The changes here effectively duplicate the way promtail's client does sends to Loki which is code we've been using/relying on for years now. I also had a hard time following the previous implementation which used the io.Writer interface for swapping out writing to stdout vs pushing to Loki and refactored this to use a new interface `EntryWriter` which has implementations that write to stdout and to Loki Unfortunately the changes in the diff don't read super well with this refactoring so hopefully the explanation helps. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Paul Rogers <129207811+paul1r@users.noreply.github.com>	2 years ago
Bill Franklin	b88aa08d89	Loki Canary: Enable insecure-skip-verify for all requests (#8869) ... **What this PR does / why we need it**: This supports cases where the cert cannot be trusted but we are not using mTLS, i.e. `loki-canary -tls=true -insecure=true` **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Bill Franklin	3ed9f0c9ef	Canary: support filtering / parsing logs with LogQL (#8871) ... **What this PR does / why we need it**: The canary logs can be shipped to Loki in various formats. This allows users of the canary to parse out the canaries logs from any format e.g. nested json. The alternative is to do something different for the canary logs to prevent the logs being wrapped / manipulated (e.g. write to Loki directly, or use a different log pipeline) which limits the benefits of the canary. I've confirmed this works locally. **Which issue(s) this PR fixes**: Fixes #7775 **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Michel Hollands	4f8d3240f6	Use 0.28.1 build image and update go and alpine versions (#8583) ... **What this PR does / why we need it**: - Use the 0.28.1 build image - Update Go version to 1.20.1 - Use Alpine 3.16.4 - Fix linter issues due to updated govet in 1.20.1 - Modify images to not create `/etc/nsswitch.conf` file anymore (available by default on Alpine 3.16.4) - Remove impossible test cases for `TestMemChunk_IteratorBounds`(see https://github.com/grafana/loki/pull/8591#discussion_r1115020111 for context) Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: DylanGuedes <djmgguedes@gmail.com>	2 years ago
Travis Patterson	4a3e6f9d95	Bump go modules GitHub.com prometheus alertmanager 0.25.0 (#8100) ... This PR required some code changes: https://github.com/grafana/loki/pull/8095 We have all the information, it's just plumbed in a little different now. Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Kaviraj Kanagaraj	30ddd77150	loki-canary: respect `useTLS` flag when `push` mode is enabled. (#7701) ... **What this PR does / why we need it**: Currently `useTLS` flag is ignored on the push writer. This PR consider that flag and adjust the schema (http -> https) when that flag is enabled. **Which issue(s) this PR fixes**: Fixes #7692 **Special notes for your reviewer**: **Checklist**	3 years ago
Kaviraj Kanagaraj	ceb09efdf8	fix(loki-canary): Send to Loki after updating `totalEntries`. (#7211) ... **What this PR does / why we need it**: Fixes: #7142 (take a look for more details about the problem there) After trying few things to avoid this race and discussin it with Ed, This small changes seems to be more accurate and seems to fix the issue. This is unblocking rolling out new canary in our internal cells Update: For some reason I thought the fix didn't fix the original issue when I tested it with internal loki dev cell, almost a month ago. But after testing it for more than 2hr on same cell. I realized it is working fine (will leave it for a day like this just to confirm) Verified following: ``` {.... container="loki-canary"} |= "websocket missing" ``` and noticed it's happening no longer after the fix. Also verified the metrics ``` sum(increase(loki_canary_websocket_missing_entries_total{...}[$__range])) ``` It dropped to zero.   Update 2: After running it for whole day, looks there was two log entries missed on the websocket. My guess is since websocket is long-live connection and something can interrupt the connection loosing this message.  I also quickly checked how often this happens on other bigger envs (say ops) and looks like it's not that uncommon.  **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/issues/7142 **Special notes for your reviewer**: Tested it on internal loki dev cells **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Danny Kopping	c3e92b876c	Restoring lint checks that were previously disabled (#7340) ... I removed these checks in https://github.com/grafana/loki/pull/7337 to not make https://github.com/grafana/loki/pull/7243 overly complex to review. Refactored code to fix valid lint issues, ignored the irrelevant/dangerous ones. I also removed `unused` and `unparam` as these produce low-quality results that are largely useless.	3 years ago
Eng Zer Jun	e2842c69c5	Replace deprecated `io/ioutil` with `io` and `os` packages (#7212) ... The `io/ioutil` package has been deprecated in Go 1.16 (See https://pkg.go.dev/io/ioutil). This PR replaces the existing `io/ioutil` functions with their new definitions in `io` and `os` packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	3 years ago
Kaviraj Kanagaraj	2c9fa05c29	feat(loki-canary): Add support to push logs directly to Loki. (#7063) ... Add additional `push` mode to loki-canary, which pushes the logs directly to given Loki URL as it generates logs. The real function of Loki Canary is to act like a tenant and help us know the whether Loki installation is working as perceived by a real tenant. Main rationale for this additional push mode is to make canary more standalone without needing for `promtail` (or `grafana-agent`) to scrape it's logs and send to loki, with this change, Loki canary happily tests Loki behavior without needing any other dependencies. **NOTES**: 1. If you run Loki behind any proxy that has different authorization policies to READ or WRITE to Loki, then important change that canary operator need to be aware of it, now the user credentials they pass via `-user` and `-pass` to access loki endpoints need to have both `read' and `write` permissions (previously canary just query the logs where as promtail is the one pushes the logs, so just READ permissions was sufficient). 2. There will be follow up PR(s) to cleanup, particularly `reader` and `comparitor` component in terms of logging with proper logger. Rationale is to keep the changes small per PR to make it easy to review. 3. This changes were tested it in one of the internal Loki dev cell. 4. **This PR is a no-op if this new `push` flag is disabled (it's disabled by default)** Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Chris Hodges	c582c5a727	loki-canary: Fix resource leak when using client-side certs (#6579) ... * loki-canary: Fix resource leak when using client-side certs * swap panic for os.Exit when failing to create httpClient	3 years ago
Ed Welch	b9301045cb	introduce randomness to the first call for the metric test query to reduce the likelyhood that large numbers of canaries will all make this request at the same time. (#6562) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Chris Hodges	6cbcd6aa76	loki-canary: Add support for client-side TLS certs for Loki connection (#6310) ... * Add support for client-side TLS certs to loki-canary * fix lint errors, make the case of using TLS but not client certs more clear, check that -tls is set if client certs are used * move changelog to canary section instead of main * wrap httpClient errors and simplify dialer creation for websocket	3 years ago
Daniel Kovacs	e9d2a30733	Fixed reader to support both authentication and tenant-id at the same time. (#5719)	3 years ago
Cyril Tovena	8f02495453	Refactoring of the Loki storage (#5833) ... * Extract index interface from the series store * moving package around * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * moving package around * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve composite store for more generics index store * fixes first round of tests and ensure coverage is still good * implement series method * Fixes storage tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add test for GetSeries Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Make Loki compile Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint the whole project Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * tsdb util package usage Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more information of why SetChunkFilterer exists for the index Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes ruler/base package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes o2 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
afayngelerindbx	e49c360c36	canary: Adds locking to prevent multiple concurrent invocations of `confirmMissing` from clobbering each other (#5568)	3 years ago
Salva Corts	ece671c146	Enhance logging on timeout tailing logs on canary (#4825) ... * Enhance logging on timeout tailing logs * Apply CR changes	4 years ago
Callum Styan	3ee8ed3b1d	Allow for loki-canary to generate a percentage of out of order log lines (#4126) ... * Allow for loki-canary to generate a percentage of log lines with out of order timestamps, percentage and out of order time range is all configurable. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Switch OoO min/max flags to time duration and properly calculate OoO percentage check. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Change some variable names + add comments to make it more obvious what various parts of the canary code are actually doing. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update pkg/canary/writer/writer.go * Add notes about usage of the out-of-order-percentage flag and the requirement to use pipeline stages. Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Arve Knudsen	10c92a901e	Chore: Upgrade to latest Cortex (#4188) ... * Chore: Upgrade to latest Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix formatting Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Sandeep Sukhani	6b775ac31c	allow setting tenant id for querying logs from loki (#4185)	4 years ago
Ed Welch	6e67d1ea5d	Loki/Promtail: Client Refactor (#3623) ... * move docker driver * move fluent-bit output plugin * move fluentd plugin * move logstash plugin * move promtail * match stage uses simple logql label matcher and filter expressions only * remove remaining logql imports * remove helpers package and move contents to util package * move validation and runtime out of util package * fix imports * move build and cfg packages into util * move ParseRequest function to util package * create a new package for logql models and move marshal/unmarshal code to util folder * move ErrorLabel to logqlmodel * move stats package to logqlmodel * decouple loghttp from logql by moving validation to logql package * move gcplog to tools * move PackedEntryKey constant to logqlmodel * fix stats proto import and rebuild protos * fix windows promtail tests * update drone to new dockerfile locations	4 years ago
Cyril Tovena	a5cb38d54c	Fixes unmarshalling of tailing responses. (#3237) ... * Fixes unmarshalling of tailing responses. Some websocket were still using old json package. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes missing update of generated files. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	62272cdaa1	Fixes all lint errors. (#2768) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Sandeep Sukhani	76e73d7c20	set timestamp in instant query done by canaries which otherwise gets set to now by loki (#2732)	5 years ago
Sandeep Sukhani	501f93fa92	revendor cortex to latest master (#2543)	5 years ago
Ed Welch	204e61afe3	Loki-Canary: Backoff retries on query failures, add histograms for query performance. (#2413) ... * Canary needs to wait before spot-checking entries some reasonable amount of time else the nature of the async thread doing the testing can ask for them from Loki only a few ms after they are written and they would have no chance of being present. Adding two histograms for tracking the performance of the spot check and metric test queries * Adding backoff to queries if they fail specifically waiting 5mins when a 429 is received from Loki * fix order of parameters	5 years ago
Ed Welch	76ceec45d9	If an entry is missing on the websocket, start checking after `wait` but continue checking until `max-wait` (#2369) ... Refactored out a useful pruneList function	5 years ago
Ed Welch	2b145da7d7	Canary tweaks (#2350) ... * tweak the canary a bit, add a metric for websocket pings, fix some logging, improve the mutex locking a bit, add a read timeout on the websocket, add a lastmessage time checker to restart websocket * fix constructor * fix counter name change metric count reporting to two separate metrics, expected vs actual * add note to upgrade guide for metric name changes * fixed tests.... * Update docs/operations/upgrade.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	5 years ago
Ed Welch	81b9decfb5	Loki-Canary: Add query spot checking and metric count checking (#2344) ... * Add spot check capability to canary to query certain values over time to make sure they are still in Loki. Add a context timeout to Loki queries * Updating canary to use /loki/api/v1 endpoints adding a metric test query * fixing metric names to have _total for counters * fix parsing of timestamp from log line and don't use entry timestamp * Tweak the metric test query to adjust the range to the time the canary has been running to get accurate results until the requested range has been reached. Add a total spot check entry counter to get a ratio of success * Add some debugging logs to troubleshoot missing entries * add more tests, fix some log lines * update docs, make the spot-check timer separate from prune entries * just how many paramaters can I add to the constructor.... * run prune entries on a separate thread, only run one at a time.	5 years ago
Lukas Grimm	17390fd45d	Canary: make stream configurable (#2259) ... * tried to fix canary in non kubernetes env * update doc	5 years ago
Calle Pettersson	baa264392b	Set user agent on outgoing http requests (#2242) ... Signed-off-by: Calle Pettersson <calle@cape.nu>	5 years ago
Joe Elliott	383eb6e70b	Addition of a `/suspend` endpoint to Loki Canary (#1891) ... * First pass loki canary pause Signed-off-by: Joe Elliott <number101010@gmail.com> * Write suspending msg to stderr Signed-off-by: Joe Elliott <number101010@gmail.com> * Added resume endpoint Signed-off-by: Joe Elliott <number101010@gmail.com> * Added documentation on new endpoints Signed-off-by: Joe Elliott <number101010@gmail.com> * Pass both signals to the same channel Signed-off-by: Joe Elliott <number101010@gmail.com> * lint Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed return on startCanary Signed-off-by: Joe Elliott <number101010@gmail.com>	5 years ago
Aditya C S	f9ff2a4b02	Add goimport local-prefixes configuration to .golangci.yml (#1710)	5 years ago
Ed Welch	790ba403b1	Add another sleep to the canary reconnects this time if connection succeeds but there is an error receiving the stream. (#1581) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Cyril Tovena	b34c2b0bd3	Move to jsoniter instead of default json package (#1249) ... * Move to jsoniter instead of default json package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Removes jsoniter on the logcli output Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Ed Welch	8a95972f73	Add some additional logging to the canary on queries (#1137) ... * log what was missing from the websocket connection and what we received from a query to Loki to troubleshoot missing logs * removing unnecessary unused return variables from print statements * update test to handle the new debug output	6 years ago
Robert Fratto	f1c1097bbe	pkg/canary: use default HTTP client when reading from Loki ... The documentation[1] for net/http.Client says the following: A Client is an HTTP client. Its zero value (DefaultClient) is a usable client that uses DefaultTransport. The Client's Transport typically has internal state (cached TCP connections), so Clients should be reused instead of created as needed. Clients are safe for concurrent use by multiple goroutines. A Client is higher-level than a RoundTripper (such as Transport) and additionally handles HTTP details such as cookies and redirects. Canary was creating a new http.Client every time a query request was made to Loki, causing a new TCP connection to be established each time. If Loki has an outage, it's possible to get into a situation where all websocket requests are failing and for a high volume of requests to go through the normal query route. This can possibly lead to networking issues like socket starvation. [1]: https://golang.org/pkg/net/http/#Client	6 years ago