Mimicking Opaque Types in Sorbet

Editing note: This post first appeared in a Stripe-internal email. It was first cross-posted to this blog December 12, 2022.

I saw a neat trick the other day in Stripe’s Ruby codebase. It combines different features of Sorbet in a cute way to mimic a feature called opaque types present in other typed languages (Haskell, Rust, Flow). Opaque types allow making an abstract type, where the implementation of a type is hidden. In pseudocode, we might define an abstract type for emails like this:

-- Some abstract (opaque) type
type Email

-- All the ways to create that type
parse : String -> T.nilable(Email)

-- All the ways to use that type
extract_user : Email -> String
extract_hostname : Email -> String

This says, “There’s some type called Email, but I won’t tell you the concrete structure of the type. Instead, I’m going to provide you with functions for creating that type, and for using that type.” Since the concrete structure isn’t known, the public interface is the only way to interact with the type.

Here’s the interesting parts of the implementation. In classic Sorbet fashion, it’s kind of verbose 😅

module TypeIsOpaque; extend T::Helpers; final!; end
private_constant :TypeIsOpaque

Underlying = T.type_alias {String}
private_constant :Underlying

Type = T.type_alias {T.any(TypeIsOpaque, Underlying)}

→ View complete example on sorbet.run

Let’s break down what’s going on here:

• TypeIsOpaque is a private, final module. A final module can never be included in a class, so this type is uninhabited: there are no values of type TypeIsOpaque. Being private powers the opacity, as we’ll see. The name is crafted to give people a semblance of a hint as to what’s going on if they see it in error messages.

• Underlying is a (transparent) type alias to String. It says that our abstract email type is really a String at runtime, even if it isn’t a String for the purposes of type checking.

• Type (fully qualified: Email::Type) is the public interface to our abstract type. It’s a union of the two (private) types above. Since they’re both private, users can only mention Email::Type and not Email::Underlying nor Email::TypeIsOpaque.

That’s it! We have an abstract type that’s always a String at runtime but (mostly) can’t be used like a String statically. Let’s see it in action:

email = T.must(Email.parse('jez@example.com'))

# statically, it's an `Email::Type` (even though the error expands the type alias)
T.reveal_type(email) # Revealed type: `T.any(Email::TypeIsOpaque, String)`

# at runtime, it's a `String` (not some wrapper class)
T.unsafe(email).class # => String

Note that the type is so opaque, we can’t even ask for the runtime class without using T.unsafe!

Some more things that can and can’t be done:

# We CAN'T call `String` methods:
email.length # error: Method `length` does not exist

# We CAN call the public interface methods with a parsed email:
Email.extract_user(email)     # ok
Email.extract_hostname(email) # ok

# [bug] We CAN call the public interface methods with raw strings:
Email.extract_user('not an email') # ok (sadly)

# We CAN'T (safely) unwrap the union to the underlying type:
case email
when Email::TypeIsOpaque then # error: Non-private reference to private constant
when String then
else T.absurd(email)
end

# We CAN ONLY mention `Email::Type` in signatures:
sig {returns(Email::Type)}         # ok
sig {returns(Email::Underlying)}   # error: Non-private reference to private const
sig {returns(Email::TypeIsOpaque)} # error: Non-private reference to private const

Users could of course still call T.unsafe to use Email::Type and String interchangeably, but that’s something to discourage in code review (and is a perennial problem in Sorbet anyways, not unique to opaque types).

Again, you should probably take a look at the complete, interactive example; it’ll teach a lot more:

→ View complete example on sorbet.run

But now, I’d like to answer some common questions I imagine people will have.

Is this really useful?

Maybe the email example isn’t so motivating, but hopefully the idea of “zero overhead abstract types” are. Some more examples to spark your imagination:

• Database foreign key IDs. Make one opaque type for each kind of database object. Then you can say “this method accepts only user IDs, or only charge IDs.” The type system guarantees that different object types’ IDs aren’t interchangeable. It could even guarantee that if you have an ID, it represents an actual record in the database (or at least, a record that existed in the database). (Stripe’s in-house ORM actually does this for certain tokens.)

• HTML-sanitized strings. Draw a type-level distinction between “a String that came from the user and might have unescaped-HTML characters” and “a String that has had all it’s characters escaped.” Avoid defensively HTML-escaping because the type system tracks it.

Isn’t this really verbose?

Yep! But it’s easy to tack on concise syntax later. We’ve punted around some ideas for this over the years, but there are still a few unanswered questions which is the main reason why we haven’t built something yet:

• Should opaque types be file-private (like sealed classes), module-private (like private and private_constant), or package private?
• Should converting between the opaque and underlying type be explicit or implicit?
• Should the implementation be built on top of existing features (like the workaround above, but with syntactic sugar), or should Sorbet have a separate concept of “opaque type alias”?

None of these are insurmountable, but we prefer to let real-world usage and needs guide the features we build (“Would this have prevented an incident?” vs “Wouldn’t this be nice?”)

Also, building custom syntax for this into Sorbet would give us a chance to fix the bug mentioned above, where the opaque type isn’t quite opaque enough.

Doesn’t Sorbet already have abstract classes and methods? Why prefer opaque types?

A good point! For some kinds of abstract types, abstract classes or interfaces work totally fine.

But interfaces need classes to implement them. If I invent some application-specific interface, I probably shouldn’t monkey patch it into a standard library class like String. The alternative would be to make some sort of wrapper class which implements the interface:

class StringWrapper < T::Struct
  # (1) Wrap a String
  const :underlying, String

  # (2) Implement the interface
  include Email
  # ... override to implement the methods ...
end

But that means twice as many allocations (one allocation for the StringWrapper, one for the String) and worse memory locality (the String isn’t always in memory near the StringWrapper). For frequently-allocated objects like database IDs, that can make already slow operations even slower.

This trick isn’t a complete substitute for opaque types, and I still would love to implement them some day. But I love discovering clever tricks like this that people use to make Sorbet do what they wish it could do.

Thanks to David Judd for implementing this trick at Stripe, so that I could stumble on it.