RubyFlow The Ruby and Rails community linklog

×

The Ruby and Rails community linklog

Made a library? Written a blog post? Found a useful tutorial? Share it with the Ruby community here or just enjoy what everyone else has found!

[ANN] httpx 0.21.0: circuit breaker and webdav plugins, more timeouts, xml support

httpx 0.21.0 has been released.

https://honeyryderchuck.gitlab.io/httpx/rdoc/files/doc/release_notes/0_21_0_md.html

HTTPX.get("https://gitlab.com/honeyryderchuck/httpx")

HTTPX is an HTTP client library for the Ruby programming language.

Among its features, it supports:

  • HTTP/2 and HTTP/1.x protocol versions
  • Concurrent requests by default
  • Simple and chainable API
  • Proxy Support (HTTP(S), CONNECT tunnel, Socks4/4a/5)
  • Simple Timeout System
  • Lightweight by default (require what you need)

And also:

  • Compression (gzip, deflate, brotli)
  • Streaming Requests
  • Authentication (Basic Auth, Digest Auth, AWS Sigv4)
  • Expect 100-continue
  • Multipart Requests
  • Cookies
  • HTTP/2 Server Push
  • H2C Upgrade
  • Automatic follow redirects
  • International Domain Names
  • GRPC
  • Circuit breaker
  • WebDAV
  • Datadog integration
  • Faraday integration
  • Webmock integration
  • Sentry integration

Here are the updates since the last release:

0.21.0 Features :write_timeout, :read_timeout and :request_timeout

https://gitlab.com/honeyryderchuck/httpx/-/wikis/Timeouts

The following timeouts are now supported:

  • :write_timeout: total time (in seconds) to write a request to the server;
  • :read_timeout: total time (in seconds) to read a response from the server;
  • :request_timeout: tracks both of the above (time to write the request and read a response);

ruby HTTPX.with(timeout: { request_timeout: 60}).get(...

Just like :connect_timeout, the new timeouts are deadline-oriented, rather than op-oriented, meaning that they do not reset on each socket operation (as most ruby HTTP clients do).

None of them has a default value, in order not to break integrations, but that’ll change in a future v1, where they’ll become the default timeouts.

Circuit Breaker plugin

https://gitlab.com/honeyryderchuck/httpx/-/wikis/Circuit-Breaker

The :circuit_breaker plugin wraps around errors happening when performing HTTP requests, and support options for setting maximum number of attempts before circuit opens (:circuit_breaker_max_attempts), period after which attempts should be reset (:circuit_breaker_reset_attempts_in), timespan until circuit half-opens (circuit_breaker_break_in), respective half-open drip rate (:circuit_breaker_half_open_drip_rate), and a callback to do your own check on whether a response has failed, in case you want HTTP level errors to be marked as failed attempts (:circuit_breaker_break_on).

Read the wiki for more info about the defaults.

ruby http = HTTPX.plugin(:circuit_breaker) # that's it! http.get(...

WebDAV plugin

https://gitlab.com/honeyryderchuck/httpx/-/wikis/WebDav

The :webdav introduces some “convenience” methods to perform common WebDAV operations.

```ruby webdav = HTTPX.plugin(:webdav, origin: “http://webdav-server”) .plugin(:digest_authentication).digest_auth(“user”, “pass”)

res = webdav.put(“/file.html”, body: “this is the file body”) res = webdav.copy(“/file.html”, “/newdir/copy.html”) # … ```

XML transcoder, :xml option and response.xml

A new transcoder was added fot the XML mime type, which requires "nokogiri" to be installed. It can both serialize Nokogiri nodes in a request, and parse response content into nokogiri nodes:

ruby response = HTTPX.post("https://xml-server.com", xml: Nokogiri::XML("<xml ...")) response.xml #=> #(Document:0x16e4 { name = "document", children = ...

Improvements :proxy plugin: :no_proxy option

Support was added, in the :proxy plugin, to declare domains, either via regexp patterns, or strings, for which requests should bypass the proxy.

ruby http = HTTPX.plugin(:proxy).with_proxy( uri: "http://10.10.0.1:51432", no_proxy: ["gitlab.local", /*.google.com/] ) http.get("https://duckduckgo.com/?q=httpx") #=> proxied http.get("https://google.com/?q=httpx") #=> not proxied http.get("https://gitlab.com") #=> proxied http.get("https://gitlab.local") #=> not proxied

OOTB support for other JSON libraries

If one of multi_json, oj or yajl is available, all httpx operations doing JSON parsing or dumping will use it (the json standard library will be used otherwise).

```ruby require “oj” require “httpx”

response = HTTPX.post(“https://somedomain.json”, json: { “foo” => “bar” }) # will use “oj” puts response.json # will use “oj” ```

Bugfixes
  • :expect plugin: :expect_timeout can accept floats (not just integers).
Chore
  • DoH :https resolver: support was removed for the “application/dns-json” mime-type (it was only supported in practice by the Google DoH resolver, which has since added support for the standardized “application/dns-message”).
0.20.5

The intersect? refinement introduced in the previous version had a wrong variable name.

0.20.4 Improvements

The :response_cache plugin is now more compliant with how the RFC 2616 defines which behaviour caches shall have:

  • it caches only responses with one of the following status codes: 200, 203, 300, 301, 410.
  • it discards cached responses which become stale.
  • it supports “cache-control” header directives to decided when to cache, to store, what the response “age” is.
  • it can cache more than one response for the same request, provided that the request presents different header values for the headers declared in the “vary” response header (previously, it was only caching the first response, and discarding the remainder).
Bugfixes
  • fixed DNS resolution bug which caused a loop when a failed connection attempt would cause a new DNS request to be triggered for the same domain, filling up and giving preference to the very IP which failed the attempt.
  • response_cache: request verb is now taken into account, not causing HEAD/GET confusion for the same URL.
0.20.3 Bugfixes
  • DoH resolver wasn’t working for non-absolute (the large majority) of domains since v0.19.
  • Allowing a single IP string to be passed to the resolver option :nameserver (just like the resolv library does), besides the already supported list of IPs.
0.20.2 Bugfixes
  • fix for selector timeout errors closing all connections and ignoring resolvers.

Timeout errors on select were being propagated to all pooled connections, although not all of them were being selected on, and not all of them having timed out. plus, resolver timeouts were doing the same, making connections fail with connection timeout error, rather than resolve timeout error. A patch was implemented, where the selector now yields an error to the selected connections, rather than plain raising exception.

0.20.1 Bugfixes
  • bugfix for unregistering connections when timing out on DNS resolving; this wasn’t happening, leaving a few cases where requests to the same domain timing out on resolution would hang on the second request.

Post a comment

You can use basic HTML markup (e.g. <a>) or Markdown.

As you are not logged in, you will be
directed via GitHub to signup or sign in