apache-ignite

1
// Licensed to the Apache Software Foundation (ASF) under one or more
2
// contributor license agreements.  See the NOTICE file distributed with
3
// this work for additional information regarding copyright ownership.
4
// The ASF licenses this file to You under the Apache License, Version 2.0
5
// (the "License"); you may not use this file except in compliance with
6
// the License.  You may obtain a copy of the License at
7
//
8
// http://www.apache.org/licenses/LICENSE-2.0
9
//
10
// Unless required by applicable law or agreed to in writing, software
11
// distributed under the License is distributed on an "AS IS" BASIS,
12
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
// See the License for the specific language governing permissions and
14
// limitations under the License.
15
= Binary Marshaller
16

17
== Basic Concepts
18

19
Binary Marshaller is a component of Ignite that is responsible for data serialization. It has the advantages:
20

21
* It enables you to read an arbitrary field from an object's serialized form without full object deserialization.
22
This ability completely removes the requirement to have the cache key and value classes deployed on the server node's classpath.
23
* It enables you to add and remove fields from objects of the same type. Given that server nodes do not have model classes
24
definitions, this ability allows dynamic change to an object's structure, and even allows multiple clients with different versions of class definitions to co-exist.
25
* It enables you to construct new objects based on a type name without having class definitions at all, hence
26
allowing dynamic type creation.
27

28
Binary objects can be used only when the default binary marshaller is used (i.e. no other marshaller is set to the configuration explicitly).
29

30
[NOTE]
31
====
32
[discrete]
33
=== Restrictions
34
There are several restrictions that are implied by the BinaryObject format implementation:
35

36
* Internally, Ignite does not write field and type names but uses a lower-case name hash to identify a field or a type.
37
It means that fields or types with the same name hash are not allowed. Even though serialization will not work out-of-the-box
38
in the case of hash collision, Ignite provides a way to resolve this collision at the configuration level.
39
* For the same reason, BinaryObject format does not allow identical field names on different levels of a class hierarchy.
40
* If a class implements `Externalizable` interface, Ignite will use `OptimizedMarshaller` instead of the binary one.
41
The `OptimizedMarshaller` uses `writeExternal()` and `readExternal()` methods to serialize and deserialize objects of
42
this class which requires adding classes of `Externalizable` objects to the classpath of server nodes.
43
====
44

45
The `IgniteBinary` facade, which can be obtained from an instance of Ignite, contains all the necessary methods to work with binary objects.
46

47
[NOTE]
48
====
49
[discrete]
50
=== Automatic Hash Code Calculation and Equals Implementation
51
There are several restrictions that are implied by the BinaryObject format implementation:
52

53
If an object can be serialized into a binary form, then Ignite will calculate its hash code during serialization and
54
write it to the resulting binary array. Also, Ignite provides a custom implementation of the equals method for the binary
55
object's comparison needs. This means that you do not need to override the GetHashCode and Equals methods of your custom
56
keys and values in order for them to be used in Ignite, unless they can not be serialized into the binary form.
57
For instance, objects of `Externalizable` type cannot be serialized into the binary form and require you to implement
58
the `hashCode` and `equals` methods manually. See Restrictions section above for more details.
59
====
60

61
== Configuring Binary Objects
62

63
In the vast majority of use cases, there is no need to additionally configure binary objects.
64

65
However, in a case when you need to override the default type and field IDs calculation, or to plug in `BinarySerializer`,
66
a `BinaryConfiguration` object should be defined in `IgniteConfiguration`. This object allows specifying a global
67
name mapper, a global ID mapper, and a global binary serializer as well as per-type mappers and serializers. Wildcards
68
are supported for per-type configuration, in which case, the provided configuration will be applied to all types
69
that match the type name template.
70

71
[tabs]
72
--
73
tab:XML[]
74
[source,xml]
75
----
76
<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
77

78
  <property name="binaryConfiguration">
79
    <bean class="org.apache.ignite.configuration.BinaryConfiguration">
80

81
      <property name="nameMapper" ref="globalNameMapper"/>
82
      <property name="idMapper" ref="globalIdMapper"/>
83

84
      <property name="typeConfigurations">
85
        <list>
86
          <bean class="org.apache.ignite.binary.BinaryTypeConfiguration">
87
            <property name="typeName" value="org.apache.ignite.examples.*"/>
88
            <property name="serializer" ref="exampleSerializer"/>
89
          </bean>
90
        </list>
91
      </property>
92
    </bean>
93
  </property>
94
</bean>
95
----
96
--
97

98
== BinaryObject API
99

100
By default, Ignite works with deserialized values as it is the most common use case. To enable `BinaryObject`
101
processing, a user needs to obtain an instance of `IgniteCache` using the `withKeepBinary()` method. When enabled,
102
this flag will ensure that objects returned from the cache will be in `BinaryObject` format, when possible. The same
103
applies to values being passed to the `EntryProcessor` and `CacheInterceptor`.
104

105
[NOTE]
106
====
107
[discrete]
108
=== Platform Types
109
Note that not all types will be represented as `BinaryObject` when the `withKeepBinary()` flag is enabled. There is a
110
set of 'platform' types that includes primitive types, String, UUID, Date, Timestamp, BigDecimal, Collections,
111
Maps and arrays of these that will never be represented as a `BinaryObject`.
112

113
Note that in the example below key type Integer does not change because it is a platform type.
114
====
115

116
[tabs]
117
--
118
tab:Java[]
119
[source,java]
120
----
121
// Create a regular Person object and put it to the cache.
122
Person person = buildPerson(personId);
123
ignite.cache("myCache").put(personId, person);
124

125
// Get an instance of binary-enabled cache.
126
IgniteCache<Integer, BinaryObject> binaryCache = ignite.cache("myCache").withKeepBinary();
127

128
// Get the above person object in the BinaryObject format.
129
BinaryObject binaryPerson = binaryCache.get(personId);
130
----
131
--
132

133
== Modifying Binary Objects Using BinaryObjectBuilder
134

135
`BinaryObject` instances are immutable. An instance of `BinaryObjectBuilder` must be used in order to update fields and
136
create a new `BinaryObject`.
137

138
An instance of `BinaryObjectBuilder` can be obtained from `IgniteBinary` facade. The builder may be created using a type
139
name, in this case the returned builder will contain no fields, or it may be created using an existing `BinaryObject`,
140
in this case the returned builder will copy all the fields from the given `BinaryObject`.
141

142
Another way to get an instance of `BinaryObjectBuilder` is to call `toBuilder()` on an existing instance of a `BinaryObject`.
143
This will also copy all data from the `BinaryObject` to the created builder.
144

145
[NOTE]
146
====
147
[discrete]
148
=== Limitations
149

150
* You cannot change the types of existing fields.
151
* You cannot change the order of enum values or add new constants at the beginning or in the middle of the list of enum's
152
values. You can add new constants to the end of the list though.
153
====
154

155
Below is an example of using the `BinaryObject` API to process data on server nodes without having user classes deployed
156
on servers and without actual data deserialization.
157

158
[tabs]
159
--
160
tab:Java[]
161
[source,java]
162
----
163
// The EntryProcessor is to be executed for this key.
164
int key = 101;
165

166
cache.<Integer, BinaryObject>withKeepBinary().invoke(
167
  key, new CacheEntryProcessor<Integer, BinaryObject, Object>() {
168
    public Object process(MutableEntry<Integer, BinaryObject> entry,
169
                          Object... objects) throws EntryProcessorException {
170
            // Create builder from the old value.
171
        BinaryObjectBuilder bldr = entry.getValue().toBuilder();
172

173
        //Update the field in the builder.
174
        bldr.setField("name", "Ignite");
175

176
        // Set new value to the entry.
177
        entry.setValue(bldr.build());
178

179
        return null;
180
     }
181
  });
182
----
183
--
184

185
== BinaryObject Type Metadata
186

187
As it was mentioned above, binary object structure may be changed at runtime hence it may also be useful to get
188
information about a particular type that is stored in a cache such as field names, field type names, and affinity
189
field name. Ignite facilitates this requirement via the `BinaryType` interface.
190

191
This interface also introduces a faster version of field getter called `BinaryField`. The concept is similar to java
192
reflection and allows to cache certain information about the field being read in the `BinaryField` instance, which is
193
useful when reading the same field from a large collection of binary objects.
194

195
[tabs]
196
--
197
tab:Java[]
198
[source,java]
199
----
200
Collection<BinaryObject> persons = getPersons();
201

202
BinaryField salary = null;
203

204
double total = 0;
205
int cnt = 0;
206

207
for (BinaryObject person : persons) {
208
    if (salary == null)
209
        salary = person.type().field("salary");
210

211
    total += salary.value(person);
212
    cnt++;
213
}
214

215
double avg = total / cnt;
216
----
217
--
218

219
== BinaryObject and CacheStore
220

221
Setting `withKeepBinary()` on the cache API does not affect the way user objects are passed to a `CacheStore`. This is
222
intentional because in most cases a single `CacheStore` implementation works either with deserialized classes, or with
223
`BinaryObject` representations. To control the way objects are passed to the store, the `storeKeepBinary` flag on
224
`CacheConfiguration` should be used. When this flag is set to `false`, deserialized values will be passed to the store,
225
otherwise `BinaryObject` representations will be used.
226

227
Below is an example pseudo-code implementation of a store working with `BinaryObject`:
228

229
[tabs]
230
--
231
tab:Java[]
232
[source,java]
233
----
234
public class CacheExampleBinaryStore extends CacheStoreAdapter<Integer, BinaryObject> {
235
    @IgniteInstanceResource
236
    private Ignite ignite;
237

238
    /** {@inheritDoc} */
239
    @Override public BinaryObject load(Integer key) {
240
        IgniteBinary binary = ignite.binary();
241

242
        List<?> rs = loadRow(key);
243

244
        BinaryObjectBuilder bldr = binary.builder("Person");
245

246
        for (int i = 0; i < rs.size(); i++)
247
            bldr.setField(name(i), rs.get(i));
248

249
        return bldr.build();
250
    }
251

252
    /** {@inheritDoc} */
253
    @Override public void write(Cache.Entry<? extends Integer, ? extends BinaryObject> entry) {
254
        BinaryObject obj = entry.getValue();
255

256
        BinaryType type = obj.type();
257

258
        Collection<String> fields = type.fieldNames();
259

260
        List<Object> row = new ArrayList<>(fields.size());
261

262
        for (String fieldName : fields)
263
            row.add(obj.field(fieldName));
264

265
        saveRow(entry.getKey(), row);
266
    }
267
}
268
----
269
--
270

271
== Binary Name Mapper and Binary ID Mapper
272

273
Internally, Ignite never writes full strings for field or type names. Instead, for performance reasons, Ignite writes
274
integer hash codes for type and field names. Testing has indicated that hash code conflicts for the type names or the
275
field names within the same type are virtually non-existent and, to gain performance, it is safe to work with hash codes.
276
For the cases when hash codes for different types or fields actually do collide, `BinaryNameMapper` and `BinaryIdMapper`
277
support overriding the automatically generated hash code IDs for the type and field names.
278

279
`BinaryNameMapper` - maps type/class and field names to different names.
280
`BinaryIdMapper` - maps given from `BinaryNameMapper` type and field name to ID that will be used by Ignite in internals.
281

282
Ignite provides the following out-of-the-box mappers implementation:
283

284
* `BinaryBasicNameMapper` - a basic implementation of `BinaryNameMapper` that returns a full or a simple name of a given
285
class depending on whether the `setSimpleName(boolean useSimpleName)` property is set.
286
* `BinaryBasicIdMapper` - a basic implementation of `BinaryIdMapper`. It has a configuration property called
287
`setLowerCase(boolean isLowerCase)`. If the property is set to `false` then a hash code of given type or field name
288
will be returned. If the property is set to `true` then a hash code of given type or field name in lower case will be returned.
289

290
If you are using Java or .NET clients and do not specify mappers in `BinaryConfiguration`, then Ignite will use
291
`BinaryBasicNameMapper` and the `simpleName` property will be set to `false`, and `BinaryBasicIdMapper` and the
292
`lowerCase` property will be set to `true`.
293

294
If you are using the C{pp} client and do not specify mappers in `BinaryConfiguration`, then Ignite will use
295
`BinaryBasicNameMapper` and the `simpleName` property will be set to `true`, and `BinaryBasicIdMapper` and the
296
`lowerCase` property will be set to `true`.
297

298
By default, there is no need to configure anything if you use Java, .NET or C{pp}. Mappers need to be configured if
299
there is a tricky name conversion when platform interoperability is needed.
300