Reading Rails - How Does MessageVerifier Work?

Cryptography is famously tricky, so as developers we are often told to leverage the work of others. Wise words indeed, all the more reason to read some code.

Rails' ActiveSupport provides two classes, MessageVerifier and MessageEncryptor to help us do some pretty neat stuff with Ruby's OpenSSL library. Today we'll look at what MessageVerifier does, what it does not do, and how it works.

Signing Data

MessageVerifier allows you to seal data so that other people cannot tamper with it. Why might you do this? Let's say you want to let someone reset their password. One solution might be to generate a secret value, store it in your database, and then email it to the user. Perhaps they click a link with that value in the url, which verifies it must be them, or someone with access to their email, like their cat.

What if you don't want to store that secret value in your database? Instead, you just want to give them a ticket that says, "This person can reset user 42's password for the rest of the day." MessageVerifier lets you do this taking some data, and attaching a signature that verifies the contents, and is difficult to forge.

Lets see an example:

# "SecretValue" would typically be some long secret string.
verifier ="SecretValue")
message = {id: 42, expires: +}
#=> {:id=>42, :expires=>2014-05-10 23:02:49 -0700} 
signed_message = verifier.generate(message)
# Abbreviated result: 
#=> "BAh7BzoHaWZUkiCFBEVAY6BkVU--9db24c432a4962998683fb7538b358" 
#=> {:id=>42, :expires=>2014-05-10 23:02:49 -0700} 

Neat, our message got converted to a String of gibberish, and back again. We shouldn't be able to tamper with the data once it is signed, so what happens if we change something?

tampered = "HELLOzzzaWZUkiCFBEVAY6BkVU--9db24c432a4962998683fb7538b358" 
#=> ActiveSupport::MessageVerifier::InvalidSignature

Splendid, changing the data caused an exception.

Before we go any further though, let's clarify one thing, MessageVerifier does not encrypt your data, it only signs it.

How it Works

You can find MessageVerifier in lib/activesupport/message_verifier.rb. Notice that MessageVerifier defines a custom Exception class:

class MessageVerifier
  class InvalidSignature < StandardError; end

We saw this raised when we tried to tamper with the signed message. The exception doesn't need to do anything special, so it is an empty class defined all on one line. This allows the user to specifically rescue InvalidSignature exceptions instead of having to rescue from a broad class of exceptions. If you have a special failure case a caller might want to treat differently, this is a good idiom to follow.

The initializer is straightforward:

def initialize(secret, options = {})
  @secret = secret
  @digest = options[:digest] || 'SHA1'
  @serializer = options[:serializer] || Marshal

@secret is a secret value you pass in that only you know. The @digest is the cryptographic hash function MessageVerifier should use to sign your data. See OpenSSL::Digest for a full list of supported hashing functions. Finally a serializer can be passed in as well. The default is Marshal, Ruby's standard serializer.

Marshal can transform Ruby objects into serialized data that can written to a disk, database, or sent over the network and then converted back into Ruby objects:

message = {id: 42, expires: +}
serialized = Marshal.dump(message)
#=> "\x04\b{\a:\aidi/:\fexpiresIu:\tTime\rf\x91\x1C\x80_\xE6\x10\v\a:\voffseti\xFE\x90\x9D:\tzoneI\"\bPDT\x06:\x06ET" 
#=> {:id=>42, :expires=>2014-05-10 23:02:49 -0700} 

This is a convenient way to store Ruby objects, but may expose you to some risks if someone gains access to your @secret. If you're concerned about this, you can replace Marshal with any object that responds to dump and load. The built in YAML and JSON classes follow this pattern. If you want to use another serialization mechanism, now might be a good time to try the Adapter Pattern.

Let's see how that data gets signed by looking at the generate method:

def generate(value)
  data = ::Base64.strict_encode64(@serializer.dump(value))

We know what @serialization.dump does, we just saw it in action, it transforms value into a String.

Perhaps you haven't come across Base64 though. The serialized data above looked like this: "\x04\b{\a:\aidi/:\fexpires.... Those backslashed bits indicate that the values that Ruby can't or won't normally print. Base64 encoding transforms those values into ASCII characters which are easier to pass around without fear of encoding issues.

Let's see Base64 encoding in action:

serialized = Marshal.dump(message)
#=> "\x04\b{\a:\aidi/:\fexpiresIu:\tTime\rf\x91\x1C\x80_\xE6\x10\v\a:\voffseti\xFE\x90\x9D:\tzoneI\"\bPDT\x06:\x06ET" 
#=> "BAh7BzoHaWRpLzoMZXhwaXJlc0l1OglUaW1lDWaRHIBf5hALBzoLb2Zmc2V0af6QnToJem9uZUkiCFBEVAY6BkVU"

Base64.strict_encode64 transforms the input into plain ASCII characters. The "strict" in strict_encode64 just means line breaks aren't added. Later though we'll see strict_decode64, where strictness is a bit more important. Before you start encoding everything in Base64, you should recognize that it comes with a tradeoff. The data is now less likely to run into encoding issues, but it also takes up about a third more space:

serialized.size                         #=> 66
Base64.strict_encode64(serialized).size #=> 88

Let's move on to generate_digest(data). This is where the signature that verifies our data is generated.

def generate_digest(data)
  require 'openssl' unless defined?(OpenSSL)
  OpenSSL::HMAC.hexdigest(OpenSSL::Digest.const_get(@digest).new, @secret, data)

That doesn't really seem like enough code to do something important, but there is actually a lot going on here, so let's unpack it carefully.

The first line is odd, normally one calls require at the top of a file, but here it is inside a method. require is just a method like any other in Ruby. You can even override it if you want. defined?(OpenSSL) on the other hand isn't a method. It is a Ruby keyword that returns a string if something is defined.

defined? OpenSSL    #=> "constant"
defined? serialized #=> "local-variable"
defined? "Hello!"   #=> "expression"
defined? xyz        #=> nil

Typically you might check to see if a variable is defined with xyz.nil?, but if you try that on a constant, Ruby will throw an exception:

OpenSSL.nil?    #=> false 
ClosedSSL.nil?  #=> NameError: uninitialized constant ClosedSSL

We're going to gloss over exactly why "Hello!" is considered an expression and why ClosedSSL.nil? raises an exception for now. OpenSSL will only be required if it hasn't been loaded yet. Why bother requiring it down here? Ruby is not always distributed with OpenSSL, and it is common practice to require all of ActiveSupport at once. If this require statement was at the top of the file, anyone who tried to use ActiveSupport without OpenSSL installed would run into a LoadError. This pattern avoids loading OpenSSL until it's actually needed.

Now, how about the second line? Let's work inside out and figure out OpenSSL::Digest.const_get(@digest).new. Calling const_get on OpenSSL::Digest returns a constant defined in OpenSSL::Digest such as OpenSSL::Digest::SHA1. These are just classes, so calling .new on them creates a new Digest instance. This is a tiny implemenation of the Factory pattern in Ruby, which lets us refer to the Digest by name instead of cumbersomely instantiating the digest ourselves.

The outermost call, OpenSSL::HMAC.hexdigest(digest, @secret, data), calls OpenSSL and asks it to create a Hashed Message Authentication Code, a short string or signature generated from the input data using the Digest. Ideally these values are easy to generate, but difficult to predict, so it is very hard for a malicious person to find another input that would match the signature.

Now we know how data gets encoded and signed with MessageVerifier, let's see how it gets verified and decoded.

def verify(signed_message)
  raise InvalidSignature if signed_message.blank?

  data, digest = signed_message.split("--")
  if data.present? && digest.present? && secure_compare(digest, generate_digest(data))
    rescue ArgumentError => argument_error
      raise InvalidSignature if argument_error.message =~ %r{invalid base64}
    raise InvalidSignature

It's actually not that complicated, the bulk of the method is concerned with raising exceptions whenever anything looks odd. When you don't trust your inputs, raise early, raise often.

The first check is if for a blank signed_message. Since this should never be the case, MessageVerifier will raise InvalidSignature.

Next, the data and digest (our signature) are extracted from the signed_message by splitting on --. Since the data should be Base64 encoded, and the signature was a hexdigest meaning its output is 0-9 a-f, we can safely split on --. This should always return a two part array, assigning those parts to data and digest.

Next, ActiveSupport checks to make sure that both data and digest are present. present? is just the inverse of blank?.

Now comes the heart of the method, secure_compare(digest, generate_digest(data)). Using the same secret, hashing function, and data should always yield the same signature. If the data was changed, the signature would no longer match.

secure_compare carefully avoids any optimization Ruby might make when comparing strings. It does this by iterating over each byte in the two digests so that it always takes about the same amount of time compare. If a normal comparison were made here, then some nefarious person might be able to tell whether the signatures differed early or later in the string based on how quickly the comparison was deemed invalid.

Assuming the signatures matched, MessageVerifier will transform your data back into its original form. First the data is decoded from Base64 into bytes, then it is deserialized using the @serializer specified in the initializer.

If the data gets decoded and deserialized properly, the original message will be passed back. If anything goes wrong, we run into some interesting exception handling:

rescue ArgumentError => argument_error
  raise InvalidSignature if argument_error.message =~ %r{invalid base64}

There are multiple reasons an ArgumentError might get raised. In one case the decoding could fail, in the other the deserializing could fail, but both cases may raise an ArgumentError.

To help users isolate the cause of the exception, MessageVerifier inspects the error message and raises an InvalidSignature if it looks like the error came from the Base64 module. Otherwise, the last raise in the block re-raises the original Exception. Another solution is to wrap these two steps in separate try/rescue blocks, but that would be more verbose.

That's it, we're done! Now that we have seen how MessageVerifier works, let's look at one thing it does not do. It does not encrypt data. The original secret not required to read the data.

data = signed_message.split("--").first
original_message = Marshal.load(Base64.strict_decode64(data))
#=> {:id=>42, :expires=>2014-05-10 23:02:49 -0700} 

To protect data from prying eyes, you will want to use MessageEncryptor, which we will look at next time.


MessageVerifier lets us sign, but not encrypt data. We also saw some other neat things:

Want to learn more? Read secure_compare, how does it avoid timing attacks? As always, let me know if you have any questions, or if I've horribly botched something.

blog comments powered by Disqus
Monkey Small Crow Small