Issues with using Kryo for needed speed in serialization/deserialization

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Issues with using Kryo for needed speed in serialization/deserialization

CL Kim
Our enterprise application needs to batch very large numbers of log lines into a large (1G) mule message, which is taking too long to serialize/deserialize in a persisted vm:connector queue. 

So we wanted to try Kryo instead of the Java serializer/deserializer. But Kryo needs a serializable class to have a zero-arg constructor and fails on DefaultMuleEvent because of that. 

Is it possible to subclass and use our own DefaultMuleEvent (and probably also DefaultMuleMessage) class? 

We could then add a zero-argument constructor in our subclass. We are aware of the registry-bootstrap.properties file, but don't believe we can register the new subclass so that mule uses our subclass instead.

Reply | Threaded
Open this post in threaded view
|

Re: Issues with using Kryo for needed speed in serialization/deserialization

mgonzalez
This post has NOT been accepted by the mailing list yet.
Hello,

Kryo provide a way to overcome the default constructor limitation by using Objenesis. You can enable that feature by setting the instantiation strategy like this:

kryo.setInstantiatorStrategy(new StdInstantiatorStrategy());

It's nice to see that you talk about batching and kryo in the same note since we're adding a Batch Module and Kryo support in Mule 3.5.0 EE.

If you're a CloudHub user, you already have access to the Batch Module since the December 2013 release and Kryo support will be available in the February 2014 release.

If you're an on-premise user, you can access both features by joining the beta-program next February or when 3.5.0-EE is released next April.

Regards,
Reply | Threaded
Open this post in threaded view
|

Re: Issues with using Kryo for needed speed in serialization/deserialization

CL Kim
"… we're adding a Batch Module and Kryo support in Mule 3.5.0 EE."
Are you at liberty to say more on "Kyro Support": is that in conjunction with the Batch Module functionality only, or would Kyro Support be available generally as a replacement for the Java serialization/deserialization of classes in any mule message/payload.

"Kryo provide a way to overcome the default constructor limitation by using Objenesis."
Thanks very much for that pointer. Yes, we did figure that out as well to get around the error we were getting
"Class cannot be created (missing no-arg constructor): org.mule.DefaultMuleEvent (com.esotericsoftware.kryo.KryoException)"
but we immediately ran into other exceptions e.g. NullPointerException thrown in a DefaultMuleMessage method, e.g.:
  public DataHandler getOutboundAttachment(String name)
  {
    …
    return outboundAttachments.get(name);
  }
It looks like we might not be hooking up Kryo into the Mule framework correctly, and/or there are other post-instantiation initialization that is required for DefaultMuleMessage and perhaps other Mule core classes.

If the "Kyro Support" you mentioned is just for within Batch Module functionality, are there any help/pointers we could get to be able to use Kyro in mule framework for serialization/deserialization at this time? We are looking to go to EE version in 2014 soon :)

Thanks for your reply (and for Batch Module).