httpx 1.5.0 has been released.

HTTPX.get("https://gitlab.com/honeyryderchuck/httpx")

HTTPX is an HTTP client library for the Ruby programming language.

Among its features, it supports:

• HTTP/2 and HTTP/1.x protocol versions
• Concurrent requests by default
• Simple and chainable API
• Proxy Support (HTTP(S), CONNECT tunnel, Socks4/4a/5)
• Simple Timeout System
• Lightweight by default (require what you need)

And also:

• Compression (gzip, deflate, brotli)
• Streaming Requests
• Authentication (Basic Auth, Digest Auth, AWS Sigv4)
• Expect 100-continue
• Multipart Requests
• Cookies
• HTTP/2 Server Push
• H2C Upgrade
• Automatic follow redirects
• International Domain Names
• GRPC
• Circuit breaker
• WebDAV
• SSRF Filter
• Response caching
• HTTP/2 bidirectional streaming
• QUERY HTTP verb
• Datadog integration
• Faraday integration
• Webmock integration
• Sentry integration

Here are the updates since the last release:

1.5.0 Features :stream_bidi plugin

The :stream_bidi plugin enables bidirectional streaming support (an HTTP/2 only feature!). It builds on top of the :stream plugin, and uses its block-based syntax to process incoming frames, while allowing the user to pipe more data to the request (from the same, or another thread/fiber).

http = HTTPX.plugin(:stream_bidi)
request = http.build_request(
  "POST",
  "https://your-origin.com/stream",
  headers: { "content-type" => "application/x-ndjson" },
  body: ["{\"message\":\"started\"}\n"]
)

chunks = []

response = http.request(request, stream: true)

Thread.start do
   response.each do |chunk|
     handle_data(chunk)
   end
end

# now send data...
request << "{\"message\":\"foo\"}\n"
request << "{\"message\":\"bar\"}\n"
# ...

You can read more about it in https://honeyryderchuck.gitlab.io/httpx/wiki/Stream-Bidi

:query plugin

The :query plugin adds public methods supporting the QUERY HTTP verb:

http = HTTPX.plugin(:query)

http.query("https://example.com/gquery", body: "foo=bar") # QUERY /gquery ....

You can read more about it in https://honeyryderchuck.gitlab.io/httpx/wiki/Query

this functionality was added as a plugin for explicit opt-in, as it’s experimental (RFC for the new HTTP verb is still in draft).

:response_cache plugin filesystem based store

The :response_cache plugin supports setting the filesystem as the response cache store (instead of just storing them in memory, which is the default :store).

# cache store in the filesystem, writes to the temporary directory from the OS
http = HTTPX.plugin(:response_cache, response_cache_store: :file_store)
# if you want a separate location
http = HTTPX.plugin(:response_cache).with(response_cache_store: HTTPX::Plugins::ResponseCache::FileStore.new("/path/to/dir"))

You can read more about it in https://honeyryderchuck.gitlab.io/httpx/wiki/Response-Cache#:file_store

:close_on_fork option

A new option :close_on_fork can be used to ensure that a session object which may have open connections will not leak them in case the process is forked (this can be the case of :persistent plugin enabled sessions which have add usage before fork):

http = HTTPX.plugin(:persistent, close_on_fork: true)

# http may have open connections here
fork do
  # http has no connections here
end

You can read more about it in https://honeyryderchuck.gitlab.io/httpx/wiki/Connection-Pools#Fork-Safety .

:debug_redact option

The :debug_redact option will, when enabled, replace parts of the debug logs (enabled via :debug and :debug_level options) which may contain sensitive information, with the "[REDACTED]" placeholder.

You can read more about it in https://honeyryderchuck.gitlab.io/httpx/wiki/Debugging .

:max_connections pool option

A new :max_connections pool option (settable under :pool_options) can be used to defined the maximum number overall of connections for a pool (“in-transit” or “at-rest”); this complements, and supersedes when used, the already existing :max_connections_per_origin, which does the same per connection origin.

HTTPX.with(pool_options: { max_connections: 100 })

You can read more about it in https://honeyryderchuck.gitlab.io/httpx/wiki/Connection-Pools .

Subplugins

An enhancement to the plugins architecture, it allows plugins to define submodules (“subplugins”) which are loaded if another plugin is in use, or is loaded afterwards.

You can read more about it in https://honeyryderchuck.gitlab.io/httpx/wiki/Custom-Plugins#Subplugins .

Improvements

• :persistent plugin: several improvements around reconnections of failure:
  • reconnections will only happen for “connection broken” errors (and will discard reconnection on timeouts)
  • reconnections won’t exhaust retries
• :response_cache plugin: several improements:
  • return cached response if not stale, send conditional request otherwise (it was always doing the latter).
  • consider immutable (i.e. "Cache-Control: immutable") responses as never stale.
• :datadog adapter: decorate spans with more tags (header, kind, component, etc…)
• timers operations have been improved to use more efficient algorithms and reduce object creation.

Bugfixes

• ensure that setting request timeouts happens before the request is buffered (the latter could trigger a state transition required by the former).
• :response_cache plugin: fix "Vary" header handling by supporting a new plugin option, :supported_vary_headers, which defines which headers are taken into account for cache key calculation.
• fixed query string encoded value when passed an empty hash to the :query param and the URL already contains query string.
• :callbacks plugin: ensure the callbacks from a session are copied when a new session is derived from it (via a .plugin call, for example).
• :callbacks plugin: errors raised from hostname resolution should bubble up to user code.
• fixed connection coalescing selector monitoring in cases where the coalescable connecton is cloned, while other branches were simplified.
• clear the connection write buffer in corner cases where the remaining bytes may be interpreted as GOAWAY handshake frame (and may cause unintended writes to connections already identified as broken).
• remove idle connections from the selector when an error happens before the state changes (this may happen if the thread is interrupted during name resolution).

Chore

httpx makes extensive use of features introduced in ruby 3.4, such as Module#set_temporary_name for otherwise plugin-generated anonymous classes (improves debugging and issue reporting), or String#append_as_bytes for a small but non-negligible perf boost in buffer operations. It falls back to the previous behaviour when used with ruby 3.3 or lower.

Also, and in preparation for the incoming ruby 3.5 release, dependency of the cgi gem (which will be removed from stdlib) was removed.

1.4.4 Improvements

• :stream plugin: response will now be partially buffered in order to i.e. inspect response status or headers on the response body without buffering the full response
  • this fixes an issue in the down gem integration when used with the :max_size option.
• do not unnecessarily probe for connection liveness if no more requests are inflight, including failed ones.
• when using persistent connections, do not probe for liveness right after reconnecting after a keep alive timeout.

Bugfixes

• :persistent plugin: do not exhaust retry attempts when probing for (and failing) connection liveness.
  • since the introduction of per-session connection pools, and consequentially due to the possibility of multiple inactive connections for the same origin being in the pool, which may have been terminated by the peer server, requests would fail before being able to establish a new connection.
• prevent retrying to connect the TCP socket object when an SSLSocket object is already in place and connecting.

1.4.3 Bugfixes

• webmock adapter: reassign headers to signature after callbacks are called (these may change the headers before virtual send).
• do not close request (and its body) right after sending, instead only on response close
  • prevents retries from failing under the :retries plugin
  • fixes issue when using faraday-multipart request bodies
• retry request with HTTP/1 when receiving an HTTP/2 GOAWAY frame with HTTP_1_1_REQUIRED error code.
• fix wrong method call on HTTP/2 PING frame with unrecognized code.
• fix EOFError issues on connection termination for long running connections which may have already been terminated by peer and were wrongly trying to complete the HTTP/2 termination handshake.