RubyFlow The Ruby and Rails community linklog

×

The Ruby and Rails community linklog

Made a library? Written a blog post? Found a useful tutorial? Share it with the Ruby community here or just enjoy what everyone else has found!

The Open-Core task runtime that unlocks Ractors, unifies your backends, and delivers

The Open-Core task runtime that unlocks Ractors, unifies your backends, and delivers a 3× performance leap. Background job scheduling in Ruby has historically forced a brutal hardware compromise: suffer from Global VM Lock (GVL) limitations in multi-threaded modes, or deploy heavy, RAM-hungry isolated instances to parallelize workloads (Sidekiq) [1.7, 1.10]. The hardware physics & performance answer under native Linux (Debian Bookworm) & Ruby 4.0.7: Production telemetry benchmarks executed across a full 288-case matrix prove the raw power of SolidEngine’s Shared-Nothing model paired with isolated Ruby 4 Ractors [1.7, 1.10]: ⚡ Mixed Workload Crushing: On mixed workloads combining dense CPU processing and network wait, the SolidJobs + SolidEngine stack scales linearly up to 1,763 jobs/s at concurrency 8, delivering a massive 3.1× throughput gain over Sidekiq bottlenecked at 572 jobs/s by the GVL [1.7]. ⚡ Multiprocess Parity at −72.8% RAM: On pure CPU workloads, the Ractor-native layout matches the throughput of multiple isolated Sidekiq processes (2,021 jobs/s), while collapsing overall physical memory from 250 MiB down to only 68 MiB [1.7, 1.17]. ⚡ Zero-Overhead Density: Powered by a geometric sub-table partitioning layout (split-64) and a dedicated thread handling asynchronous memory trimming, the server’s idle RSS drops straight down to 5.6 MiB at rest via malloc_trim(0) [1.6, 1.7]. The core paradigm shift: Total backend flexibility out of the box With SolidJobs, your business task API remains strictly unchanged, but you select and swap your storage engines dynamically [1.10]: 🔹 The :solid_redis adapter: Powered by the highly optimized solid-redis gem, routing jobs to your Redis, Sentinel, or Cluster layout with isolated state, pools, and sockets per Ractor [1.7, 1.10]. It maintains a 5.5× speedup over single-process Sidekiq threads on CPU work [1.7]. 🔹 The :solid_engine adapter: Bypassing Redis completely to stream tasks directly over TCP/Unix sockets to our dedicated Rust server (SolidEngine), unlocking binary fencing, atomic leases, and native deduplication [1.10]. This architectural freedom includes per-channel routing, allowing both engines to run side by side for granular migrations. Routing all channels to SolidEngine makes Redis entirely optional, completely removing it from the production environment [1.10]. Version 0.6.0 is live and will be featured tomorrow night at Paris.rb [1.10]. Check the official repository for documentation, CSV matrices, and multi-language protocol examples [1.10].

https://github.com/nicolasva/solid-redis

https://github.com/nicolasva/solid-engine

https://github.com/nicolasva/solid-jobs

Ruby #Rust #CloudExit #FinOps #Infrastructure #SystemsArchitecture

Post a comment

You can use basic HTML markup (e.g. <a>) or Markdown.

As you are not logged in, you will be
directed via GitHub to signup or sign in