How to customize http.Client or http.Transport in Go to retry after timeout?
Answer #1 100 %Note that the Timeout field of http.Client is more or less obsolete. Best practice now is to use http.Request.Context() for timeouts. ? Flimzy
Thanks for the inspiration from @Flimzy! I attempted to use context for timeout control instead of http.Client way. Here's the code:
func (ct *CustomTransport) RoundTrip(req *http.Request) (resp *http.Response, err error) {
req.Header.Set("Secret", "Blah blah blah")
// ... other customizations for each request
for i := 1; i <= 5; i++ {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
//reqT := req.WithContext(ctx)
resp, err = ct.RoundTripper.RoundTrip(req.WithContext(ctx))
if errors.Is(err, context.DeadlineExceeded) {
log.Warnf("#%d got timeout will retry - %v", i, err)
//time.Sleep(time.Duration(100*i) * time.Millisecond)
continue
} else {
break
}
}
As per the log, it works (note the timestamp in the logs, it actually retried):
2020-07-16T00:06:12.788+0800 DEBUG begin to get "https://httpbin.org/delay/10"
2020-07-16T00:06:20.794+0800 WARN #1 got timeout will retry - context deadline exceeded
2020-07-16T00:06:28.794+0800 WARN #2 got timeout will retry - context deadline exceeded
2020-07-16T00:06:36.799+0800 WARN #3 got timeout will retry - context deadline exceeded
2020-07-16T00:06:44.803+0800 WARN #4 got timeout will retry - context deadline exceeded
2020-07-16T00:06:52.809+0800 WARN #5 got timeout will retry - context deadline exceeded
2020-07-16T00:06:52.809+0800 DEBUG got final result: context deadline exceeded
2020-07-16T00:06:52.809+0800 WARN client got error: Get "https://httpbin.org/delay/10": context deadline exceeded
2020-07-16T00:06:52.809+0800 DEBUG end to get "https://httpbin.org/delay/10", time cost: 40.019334668s