RubyFlow The Ruby and Rails community linklog

×

The Ruby and Rails community linklog

Made a library? Written a blog post? Found a useful tutorial? Share it with the Ruby community here or just enjoy what everyone else has found!

Pragmatic Tokenizer

Pragmatic Tokenizer is a multilingual tokenizer to split a string into tokens. Looking for developers with knowledge in languages outside of English to help add specs or add stop word / abbreviation lists for languages with poor coverage.

Post a comment

You can use basic HTML markup (e.g. <a>) or Markdown.

As you are not logged in, you will be
directed via GitHub to signup or sign in