# About data serialization

Transporter needs a Serializer Module which serializes & deserializes the transferable data. The default Serializer is the JsonSerializer but there are several built-in Serializers can convert messages into MessagePack, BSON, CBOR, SMILE, Amazon ION or other binary formats.

# JSON Serializer

This is the built-in default Serializer. It serializes the packets to JSON string and deserializes the received JSON bytes to Tree objects. The performance of JSON Serializers in Java and JavaScript is very good, JSON serialization is usually faster than most binary Serializers. This Serializer is compatible with the JavaScript/Go version of Moleculer.

NatsTransporter transporter = new NatsTransporter("nats://nats.server:4222");

// Do not have to set it because this is the default
transporter.setSerializer(new JsonSerializer());

ServiceBroker broker = ServiceBroker.builder()
                                    .nodeID("server-1")
                                    .transporter(transporter)
                                    .readers("jackson,boon")
                                    .writers("jackson,fast")
                                    .build();

The "readers" and "writers" parameters are used to specify the JSON API to be used by Moleculer for deserialization / serialization. This is important if you have multiple JSON implementations on the classpath. Several implementations can be specified in order of importance, separated by commas. If not specified, ServiceBroker will automatically try to choose the fastest JSON API. The values of the "readers" and "writers" parameters are listed below:

So, for example, if you want to use a "FastJSON" implementation, put the implementation reference in the "dependencies" block of the (build.gradle or pom.xml) build script, then set "readers" and "writers" to "fast". To verify, type "info" command into the REPL console. The "info" command will display the current Moleculer configuration, including the the JSON API which is in use.

# MessagePack Serializer

Built-in MsgPack Serializer. MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it's smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves. This Serializer is compatible with the JavaScript version of Moleculer.

transporter.setSerializer(new MsgPackSerializer());

# BSON Serializer

Built-in BSON Serializer. BSON, short for Binary JSON, is a binary-encoded serialization of JSON-like documents. Like JSON, BSON supports the embedding of documents and arrays within other documents and arrays.

transporter.setSerializer(new BsonSerializer());

# CBOR Serializer

Built-in CBOR Serializer. CBOR is based on the wildly successful JSON data model: numbers, strings, arrays, maps (called objects in JSON), and a few values such as false, true, and null.

transporter.setSerializer(new CborSerializer());

# Amazon ION Serializer

Built-in ION Serializer. Amazon Ion is a richly-typed, self-describing, hierarchical data serialization format offering interchangeable binary and text representations. The binary representation is efficient to store, transmit, and skip-scan parse.

transporter.setSerializer(new IonSerializer());

# SMILE Serializer

Built-in SMILE Serializer. SMILE is a computer data interchange format based on JSON. It can also be considered as a binary serialization of generic JSON data model, which means that tools that operate on JSON may be used with SMILE as well, as long as proper encoder/decoder exists for tool to use. Compared to JSON, SMILE is both more compact and more efficient to process.

transporter.setSerializer(new SmileSerializer());

# Custom Serializer

Custom Serializer module can be created. To make your own Serializer, you need to derive it from the services.moleculer.serializer.Serializer superclass, and implement the "write" and "read" methods.

Create custom Serializer

public class CustomSerializer extends Serializer {

    // --- SERIALIZE TREE TO BYTE ARRAY ---

    public byte[] write(Tree value) throws Exception {
        Object content = value.asObject();
        // Write Java Object into byte array...
        // The "content" is "java.util.Map" or "java.util.List".
    }

    // --- DESERIALIZE BYTE ARRAY TO TREE ---

    public Tree read(byte[] source) throws Exception {
        Object content = // Read Java Object from "source"...
        return new CheckedTree(content);
    }

}

Use custom Serializer

transporter.setSerializer(new CustomSerializer());

# Message-level encryption

The BlockCipherSerializer is capable of encrypting data packets using the specified algorithm. You can give it a "parent" Serializer, if not specified it uses JSON serialization before encryption.

BlockCipherSerializer serializer = new BlockCipherSerializer();

// Same as "aes-256-cbc" in Node.js
serializer.setAlgorithm("AES/CBC/PKCS5Padding"); // Algorithm
serializer.setPassword("12345678901234567890123456789012"); // 32 bytes of password
serializer.setIv("1234567890123456"); // 16 bytes of IV block - required for AES/CBC

// Create Transporter and Service Broker
Transporter transporter = new NatsTransporter("localhost");
transporter.setSerializer(serializer);
ServiceBroker broker = ServiceBroker.builder()
                                    .nodeID("node1")
                                    .transporter(transporter)
                                    .build();

In Node.js-based Moleculer, code with similar functionality looks like this:

// JavaScript code
const broker = new ServiceBroker({
    transporter: "NATS",
    middlewares: [
        Middlewares.Transmit.Encryption("12345678901234567890123456789012", // Password
                                        "aes-256-cbc", // Algorithm
                                        "1234567890123456") // 16 bytes of IV block
    ]
});

# Compressing messages

The DeflaterSerializer can compress messages larger than the specified size. It can also have a "parent" Serializer, if not specified it uses JSON serialization before compression.

Transporter transporter = new NatsTransporter("localhost");
transporter.setSerializer(new DeflaterSerializer(512));

// Create Service Broker
ServiceBroker broker = ServiceBroker.builder()
                                    .nodeID("node1")
                                    .transporter(transporter)
                                    .build();

In Node.js-based Moleculer, code with similar functionality looks like this:

const broker = new ServiceBroker({
    transporter: "NATS",
    middlewares: [
        Middlewares.Transmit.Compression({ method: "deflateRaw",
                                           threshold: "512b" }),
    ]
});

# Chaining Serializers

By chaining Serializers, you can combine multiple Serializers. In the example below, the data is first serialized using the MessagePack algorithm. Deflater then compresses large data packets (larger than 1024 bytes) and then encrypts the compressed packet using the AES algorithm:

Transporter natsTransporter = new NatsTransporter("localhost");

// Create Serializer-chain
MsgPackSerializer msgPack = new MsgPackSerializer();
DeflaterSerializer deflater = new DeflaterSerializer(msgPack);
BlockCipherSerializer cipher = new BlockCipherSerializer(deflater, "1234567890123456");

// Set Serializer-chain to Transporter
natsTransporter.setSerializer(cipher);

// Create Service Broker
ServiceBroker broker = ServiceBroker.builder()
                                    .nodeID("node1")
                                    .transporter(natsTransporter)
                                    .build();

Double encryption

Chaining allows multiple encryption methods to be applied to a data packet. In the example below, the data is first encrypted using the "ARCFOUR" and then the "Blowfish" algorithm:

// Create JSON Serializer
JsonSerializer jsonSerializer = new JsonSerializer();
        
// First cipher
BlockCipherSerializer first = new BlockCipherSerializer(jsonSerializer,
                                                        "password1",
                                                        "ARCFOUR");
// Second cipher
BlockCipherSerializer second = new BlockCipherSerializer(first,
                                                         "password2",
                                                         "Blowfish");

// Set Serializer-chain to Transporter
transporter.setSerializer(second);

// Create Service Broker
ServiceBroker broker = ServiceBroker.builder()
                                    .nodeID("node1")
                                    .transporter(transporter)
                                    .build();