RubyFlow The Ruby and Rails community linklog

×

The Ruby and Rails community linklog

Made a library? Written a blog post? Found a useful tutorial? Share it with the Ruby community here or just enjoy what everyone else has found!

Tracking LLM Latency & Cost with Rails Events

Wiring an LLM into a Rails app takes a handful of lines. Understanding what it actually costs you (feature by feature, user by user) is harder.

Most providers and SDKs already report tokens, latency, and even cost, but that data lives in their dashboard. It’s disconnected from your requests, your users, and the feature that made the call. And it sits apart from the APM and logs where you already watch the rest of your app.

In this post, Henrique puts Rails.event.notify to work on a real problem: instrumenting every LLM call in your app so token usage, latency, and cost become structured events you can log, graph, and forward to any APM or data warehouse.

Read more over here: Tracking LLM Latency & Cost with Rails Events

Post a comment

You can use basic HTML markup (e.g. <a>) or Markdown.

As you are not logged in, you will be
directed via GitHub to signup or sign in