Unexpected Deserialization pt.1 - JMS

On a recent engagement our task was to assess the security of a service built on IBM Integration Bus, an integration platform for Java Messaging Services. These scary looking enterprise buzzwords usually hide systems of different complexities connected with Message Queues. Since getting arbitrary test data in and out of these systems is usually non-trivial (more on this in the last paragraph), we opted for a white-box analysis, that allowed us to discover interesting cases of Java deserialization vulnerabilities.

First things first, some preliminary reading:

• If you are not familiar with message queues, read this tutorial on ZeroMQ, and you’ll realize that MQ’s are not magic, but they are magical :)
• Matthias Kaiser’s JMS research provided the basis for this post, you should read it before moving on

Our target received JMS messages using the Spring Framework. Transport of messages was done over IBM MQ (formerly IBM Websphere MQ), this communication layer and the JMS API implementation were both provided by IBM’s official MQ Client for Java.

Matthias provides the following vulnerable scenarios regarding Websphere MQ:

We used everyone’s favorite advanced source code analysis tool – grep – to find references to the getObject() and getObjectInternal() methods, but found no results in the source code. Then we compiled the code and set up a test environment using the message broker Docker image IBM provides (this spared a lot of time), and among some dynamic tests, we ran JMET against it. To our surprise we popped a calculator in our test environment!

Now this was great, but to provide meaningful resolutions to the client, we needed to investigate the root cause of the issue. The application was really simple: it received a JMS message, created an XML document from its contents, and done some other basic operations on the resulting Document objects. We recompiled the source with all the parsing logic removed to see if this is a framework issue – fortunately it wasn’t, the bug didn’t trigger. This narrowed down the code to just a few lines, since the original JMS message was essentially discarded after the XML document was constructed.

The vulnerability was in the code responsible for retrieving the raw contents of the JMS message. Although JMS provides strongly typed messages, and the expected payload were strings, the developers used the getBody() method of the generic JMSMessage class to get the message body as a byte array. One could think (I sure did) that such a method would simply take a slice of the message byte stream, and pass it back to the user, but there is a hint of something weirder in the method signature:

<T> T    getBody(java.lang.Class<T> c);

The method can return objects of arbitrary class?! After decompiling the relevant classes, all became clear: the method first checks if the class parameter is compatible with the JMS message type, and if it is, it casts the object in the body and returns it. If the JMS message is an Object message, it deserializes its contents, twice: first for the compatibility check, then to create the return object.

I don’t think this is something an average developer should think about, even if she knows about the dangers of deserialization. But this is not the only unintuitive thing that I encountered while falling down this rabbit hole.

Spring’s MessageConverter

At this point I have to emphasize, that our original target wasn’t exactly built according to best practices: JMSMessage is specific to IBM’s implementation, so using it directly chains the application to the messaging platform, which is probably undesirable. To hide the specifics of the transport, the more abstract Message class is provided by the JMS API, but there are even more elegant ways to handle incoming messages.

When using Spring one can rely on the built-in MessageConverter classes that can automatically convert Messages to more meaningful types. So – as demonstrated in the sample app – this code:

receiveMessage(Message m){ /* Do something with m, whatever that is */ }

can become this:

receiveMessage(Email e){ /* Do something with an E-mail */ } 

Of course, using this functionality to automatically convert messages to random Serializable objects is a call for trouble, but Spring’s SimpleMessageConverter implementation can also handle simple types like byte arrays.

To see if converters guard against insecure deserialization I created multiple branches of IBM’s sample application, with different signatures for receiveMessage(). To my surprise, RCE could be achieved in almost all of the variants, even if receiveMessage()’s argument is converted to a simple String or byte[]! IBM’s original sample is vulnerable to code execution too (when the class path contains appropriate gadgets).

After inspecting the code a bit more it seems, that listener implementations can’t expect received messages to be of a certain, safe type (such as TextMessage, when the application works with Strings), so they do their best to transform the incoming messages to a type expected by the developer. Additionally, in case when an attacker sends Object messages, it is up to the transport implementation to define the serialization format and other rules. To confirm this, I ran some tests using ActiveMQ for transport, and the issue couldn’t be reproduced – the reason is clear from the exception:

Caused by: javax.jms.JMSException: Failed to build body from content. Serializable class not available to broker. Reason: java.lang.ClassNotFoundException: Forbidden class org.apache.commons.collections4.comparators.TransformingComparator!
This class is not trusted to be serialized as ObjectMessage payload. Please take a look at http://activemq.apache.org/objectmessage.html for more information on how to configure trusted classes.
         at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:36) ~[activemq-client-5.15.9.jar!/:5.15.9]
         at org.apache.activemq.command.ActiveMQObjectMessage.getObject(ActiveMQObjectMessage.java:213) ~[activemq-client-5.15.9.jar!/:5.15.9]
         at org.springframework.jms.support.converter.SimpleMessageConverter.extractSerializableFromMessage(SimpleMessageConverter.java:219) ~[spring-jms-5.1.7.RELEASE.jar!/:5.1.7.RELEASE]

As we can see, ActiveMQ explicitly prevents the deserialization of objects of known dangerous classes (commons-collections4 in the above example), and Spring expects such safeguards to be the responsibility of the JMS implementations – too bad IBM MQ doesn’t have that, resulting in a deadly combination of technologies.

Update 2020.09.04.: I contacted Pivotal (Spring’s owner) about the issue, and they confirmed, that they “expect messaging channels to be trusted at application level”. They also agree, that handling ObjectMessages is a difficult problem, that should be avoided when possible: their recommendation is to implement custom MessageConverters that only accepts JMS message types, that can be safely handled (such as TextMessage or BytesMessage).

Conclusions and Countermeasures

In Spring, not relying on the default MessageConverters, and expecting simple Message (or JMSMessage in case of IBM MQ) objects in the JmsListener prevents the problem, independently from the transport implementation. Simple getters, such as getText() can be safely used after casting. The use of even the simplest converted types, such as TextMessage with IBM MQ is insecure! Common converters, such as the JSON based MappingJackson2MessageConverter need further research, as well as other transports, that decided not to implement countermeasures:

Static Analysis

After identifying vulnerable scenarios I wanted to create automated tests to discover similar issues in the future. When aiming for insecure uses of IBM MQ with Spring, the static detection method is pretty straightforward:

• Identify the parameters of methods annoted with JmsListener
• Find cases where generic objects are retrieved from these variables via the known vulnerable methods.

In CodeQL a simple predicate can be used to find appropriately annotated sources:

class ReceiveMessageMethod extends Method {
  ReceiveMessageMethod() {
      this.getAnAnnotation().toString().matches("JmsListener")
  }
}

ShiftLeft Ocular also exposes annotations, providing a simple way to retrieve sources:

val sources=cpg.annotation.name("JmsListener").method.parameter

Identifying uses of potentially dangerous API’s is also reasonably simple both in CodeQL:

predicate isJMSGetBody(Expr arg) {
  exists(MethodAccess call, Method getbody |
    call.getMethod() = getbody and
    getbody.hasName("getBody") and
    getbody.getDeclaringType().getAnAncestor().hasQualifiedName("javax.jms", "Message") and
    arg = call.getQualifier()
  )
}

… and in Ocular:

val sinks=cpg.typ.fullName("com.ibm.jms.JMSMessage").method.name("getBody").callIn.argument

Other sinks (like getObject()) can be added in both languages using simple boolean logic. An example run of Ocular can be seen on the following screenshot:

With Ocular, we can also get an exhaustive list of API’s that call ObjectInputStream.readObject() for the transport implementation in use, based on the available byte-code, without having to recompile the library:

ocular> val sinks = cpg.method.name("readObject")
 sinks: NodeSteps[Method] = io.shiftleft.semanticcpg.language.NodeSteps@22be2e19
ocular> val sources=cpg.typ.fullName("javax.jms.Message").derivedType.method
 sources: NodeSteps[Method] = io.shiftleft.semanticcpg.language.NodeSteps@4da2c297
ocular> sinks.calledBy(sources).newCallChain.l

This gives us the following entry points in IBM MQ:

• com.ibm.msg.client.jms.internal.JmsMessageImpl.getBody – Already identified
• com.ibm.msg.client.jms.internal.JmsObjectMessageImpl.getObject – Already identified
• com.ibm.msg.client.jms.internal.JmsObjectMessageImpl.getObjectInternal – Already identified
• com.ibm.msg.client.jms.internal.JmsMessageImpl.isBodyAssignableTo – Private method (used for type checks, see above)
• com.ibm.msg.client.jms.internal.JmsMessageImpl.messageToJmsMessageImpl – Protected method
• com.ibm.msg.client.jms.internal.JmsStreamMessageImpl.<init> – Deserializes javax.jms.StreamMessage objects.

The above logic can be reused for other implementations too, so accurate detections can be developed for reliant applications. Connecting paths between applications and transport implementations doesn’t seem possible with static analysis, as the JMS API loads the implementations dynamically. Our static queries are soon to be released on GitHub.

A Word About Enterprise Architecture and Threat Models

When dealing with targets similar to the one described in this article, it is usually difficult to create a practical test scenario that is technically achievable, and makes sense from a threat modeling perspective.

In our experience, this problem stems from the fact, that architectures like ESB and the tools built around them provide abstractions that hide the actual implementation details from the end users and even administrators. And when people think about things like “message-oriented middleware” instead of long-lived TCP connections between machines, it can be hard to figure out that at the end of day, one can simply send potentially malicious input to 10.0.0.1 by establishing a TCP connection to port 1414 on 10.1.2.3. This means that in many cases it’s surprisingly hard to find someone who can specify in technical terms where and how an application should be tested, not to to mention the approval of these tests. Another result of this, is that in many cases message queues are treated as inherently trusted – no one can attack a magic box, that no one (at least none of us) knows how it exactly works, right?

Technical security assessments can be great opportunities to not only discover vulnerabilities early, but also to get more familiar with the actual workings of these complex, but not incomprehensible systems. It the end, we are the ones whose job is to understand systems from top to bottom.

Special thanks to Matthias Kaiser and Fabian Yamaguchi for their tools and help in compiling this blog post! Featured image from Birds of Prey.