apache-ignite

1
// Licensed to the Apache Software Foundation (ASF) under one or more
2
// contributor license agreements.  See the NOTICE file distributed with
3
// this work for additional information regarding copyright ownership.
4
// The ASF licenses this file to You under the Apache License, Version 2.0
5
// (the "License"); you may not use this file except in compliance with
6
// the License.  You may obtain a copy of the License at
7
//
8
// http://www.apache.org/licenses/LICENSE-2.0
9
//
10
// Unless required by applicable law or agreed to in writing, software
11
// distributed under the License is distributed on an "AS IS" BASIS,
12
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
// See the License for the specific language governing permissions and
14
// limitations under the License.
15
= TCP/IP Discovery
16

17
:javaFile: {javaCodeDir}/TcpIpDiscovery.java
18

19
In an Ignite  cluster, nodes can discover each other by using `DiscoverySpi`.
20
Ignite provides `TcpDiscoverySpi` as a default implementation of `DiscoverySpi` that uses TCP/IP for node discovery.
21
Discovery SPI can be configured for Multicast and Static IP based node
22
discovery.
23

24
== Multicast IP Finder
25

26
`TcpDiscoveryMulticastIpFinder` uses Multicast to discover other nodes
27
and is the default IP finder. Here is an example of how to configure
28
this finder via a Spring XML file or programmatically:
29

30

31
[tabs]
32
--
33
tab:XML[]
34
[source,xml]
35
----
36
include::code-snippets/xml/discovery-multicast.xml[tags=ignite-config, indent=0]
37
----
38
tab:Java[]
39
[source,java]
40
----
41
include::{javaFile}[tag=multicast,indent=0]
42
----
43
tab:C#/.NET[]
44
[source,csharp]
45
----
46
include::code-snippets/dotnet/ClusteringTcpIpDiscovery.cs[tag=multicast,indent=0]
47
----
48
tab:C++[unsupported]
49
--
50

51
== Static IP Finder
52

53
Static IP Finder, implemented in `TcpDiscoveryVmIpFinder`, allows you to specify a set of IP addresses and ports that will be checked for node discovery.
54

55
You are only required to provide at least one IP address of a remote
56
node, but usually it is advisable to provide 2 or 3 addresses of
57
nodes that you plan to start in the future. Once a
58
connection to any of the provided IP addresses is established, Ignite automatically discovers all other nodes.
59

60
[TIP]
61
====
62
Instead of specifying addresses in the configuration, you can specify them in
63
the `IGNITE_TCP_DISCOVERY_ADDRESSES` environment variable or in the system property
64
with the same name. Addresses should be comma separated and may optionally contain
65
a port range.
66
====
67

68
[TIP]
69
====
70
By default, the `TcpDiscoveryVmIpFinder` is used in the 'non-shared' mode.
71
If you plan to start a server node, then in this mode the list of IP addresses should contain the address of the local node as well. In this case, the node will not wait until other nodes join the cluster; instead, it will become the first cluster node and start to operate normally.
72
====
73

74
You can configure the static IP finder via XML configuration or programmatically:
75

76
[tabs]
77
--
78
tab:XML[]
79
[source,xml]
80
----
81
include::code-snippets/xml/discovery-static.xml[tags=ignite-config, indent=0]
82
----
83

84
tab:Java[]
85
[source,java]
86
----
87
include::{javaFile}[tag=static,indent=0]
88
----
89

90
tab:C#/.NET[]
91
[source,csharp]
92
----
93
include::code-snippets/dotnet/ClusteringTcpIpDiscovery.cs[tag=static,indent=0]
94
----
95

96
tab:Shell[]
97
[source,shell]
98
----
99
# The configuration should use TcpDiscoveryVmIpFinder without addresses specified:
100

101
IGNITE_TCP_DISCOVERY_ADDRESSES=1.2.3.4,1.2.3.5:47500..47509 bin/ignite.sh -v config/default-config.xml
102
----
103
--
104

105
[WARNING]
106
====
107
[discrete]
108
Provide multiple node addresses only if you are sure that those are reachable. The unreachable addresses increase the
109
time it takes for the nodes to join the cluster. Let's say you set five IP addresses, and nobody listens for incoming
110
connections on two addresses out of five. If Ignite starts connecting to the cluster via those two unreachable addresses,
111
it will impact the node's startup time.
112
====
113

114

115
== Multicast and Static IP Finder
116

117
You can use both Multicast and Static IP based discovery together. In
118
this case, in addition to any addresses received via multicast,
119
`TcpDiscoveryMulticastIpFinder` can also work with a pre-configured list
120
of static IP addresses, just like Static IP-Based Discovery described
121
above. Here is an example of how to configure Multicast IP finder with
122
static IP addresses:
123

124
[tabs]
125
--
126
tab:XML[]
127
[source,xml]
128
----
129
include::code-snippets/xml/discovery-static-and-multicast.xml[tags=ignite-config, indent=0]
130
----
131

132
tab:Java[]
133
[source,java]
134
----
135
include::{javaFile}[tag=multicastAndStatic,indent=0]
136
----
137

138
tab:C#/.NET[]
139
[source,csharp]
140
----
141
include::code-snippets/dotnet/ClusteringTcpIpDiscovery.cs[tag=multicastAndStatic,indent=0]
142
----
143

144
tab:C++[unsupported]
145

146
--
147

148

149
== Isolated Clusters on Same Set of Machines
150

151
Ignite allows you to start two isolated clusters on the same set of
152
machines. This can be done if nodes from different clusters use non-intersecting local port ranges for `TcpDiscoverySpi` and `TcpCommunicationSpi`.
153

154
Let’s say you need to start two isolated clusters on a single machine
155
for testing purposes. For the nodes from the first cluster, you
156
should use the following `TcpDiscoverySpi` and `TcpCommunicationSpi`
157
configurations:
158

159
[tabs]
160
--
161
tab:XML[]
162
[source,xml]
163
----
164
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
165
    <!--
166
    Explicitly configure TCP discovery SPI to provide list of
167
    initial nodes from the first cluster.
168
    -->
169
    <property name="discoverySpi">
170
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
171
            <!-- Initial local port to listen to. -->
172
            <property name="localPort" value="48500"/>
173

174
            <!-- Changing local port range. This is an optional action. -->
175
            <property name="localPortRange" value="20"/>
176

177
            <!-- Setting up IP finder for this cluster -->
178
            <property name="ipFinder">
179
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
180
                    <property name="addresses">
181
                        <list>
182
                            <!--
183
                            Addresses and port range of nodes from
184
                            the first cluster.
185
                            127.0.0.1 can be replaced with actual IP addresses
186
                            or host names. Port range is optional.
187
                            -->
188
                            <value>127.0.0.1:48500..48520</value>
189
                        </list>
190
                    </property>
191
                </bean>
192
            </property>
193
        </bean>
194
    </property>
195

196
    <!--
197
    Explicitly configure TCP communication SPI changing local
198
    port number for the nodes from the first cluster.
199
    -->
200
    <property name="communicationSpi">
201
        <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
202
            <property name="localPort" value="48100"/>
203
        </bean>
204
    </property>
205
</bean>
206
----
207

208
tab:Java[]
209
[source,java]
210
----
211
include::{javaFile}[tag=isolated1,indent=0]
212
----
213

214
tab:C#/.NET[]
215
[source,csharp]
216
----
217
include::code-snippets/dotnet/ClusteringTcpIpDiscovery.cs[tag=isolated1,indent=0]
218
----
219

220
tab:C++[unsupported]
221

222
--
223

224

225
For the nodes from the second cluster, the configuration might look like
226
this:
227

228
[tabs]
229
--
230
tab:XML[]
231
[source,xml]
232
----
233
<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
234
    <!--
235
    Explicitly configure TCP discovery SPI to provide list of initial
236
    nodes from the second cluster.
237
    -->
238
    <property name="discoverySpi">
239
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
240
            <!-- Initial local port to listen to. -->
241
            <property name="localPort" value="49500"/>
242

243
            <!-- Changing local port range. This is an optional action. -->
244
            <property name="localPortRange" value="20"/>
245

246
            <!-- Setting up IP finder for this cluster -->
247
            <property name="ipFinder">
248
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
249
                    <property name="addresses">
250
                        <list>
251
                            <!--
252
                            Addresses and port range of the nodes from the second cluster.
253
                            127.0.0.1 can be replaced with actual IP addresses or host names. Port range is optional.
254
                            -->
255
                            <value>127.0.0.1:49500..49520</value>
256
                        </list>
257
                    </property>
258
                </bean>
259
            </property>
260
        </bean>
261
    </property>
262

263
    <!--
264
    Explicitly configure TCP communication SPI changing local port number
265
    for the nodes from the second cluster.
266
    -->
267
    <property name="communicationSpi">
268
        <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
269
            <property name="localPort" value="49100"/>
270
        </bean>
271
    </property>
272
</bean>
273

274
----
275

276
tab:Java[]
277
[source,java]
278
----
279
include::{javaFile}[tag=isolated2,indent=0]
280
----
281

282
tab:C#/.NET[]
283
[source,csharp]
284
----
285
include::code-snippets/dotnet/ClusteringTcpIpDiscovery.cs[tag=isolated2,indent=0]
286
----
287

288
tab:C++[unsupported]
289

290
--
291

292
As you can see from the configurations, the difference between them is minor — only port numbers for SPIs and IP finder vary.
293

294
[TIP]
295
====
296
If you want the nodes from different clusters to be able to look for
297
each other using the multicast protocol, replace
298
`TcpDiscoveryVmIpFinder` with `TcpDiscoveryMulticastIpFinder` and set
299
unique `TcpDiscoveryMulticastIpFinder.multicastGroups` in each
300
configuration above.
301
====
302

303
[CAUTION]
304
====
305
[discrete]
306
=== Persistence Files Location
307

308
If the isolated clusters use Native Persistence, then every
309
cluster has to store its persistence files under different paths in the
310
file system. Refer to the link:persistence/native-persistence[Native Persistence documentation] to learn how you can change persistence related directories.
311
====
312

313

314
== JDBC-Based IP Finder
315
NOTE: Not supported in .NET/C#/{cpp}.
316

317
You can have your database be a common shared storage of initial IP addresses. With this IP finder, nodes will write their IP addresses to a database on startup. This is done via `TcpDiscoveryJdbcIpFinder`.
318

319
[tabs]
320
--
321
tab:XML[]
322
[source,xml]
323
----
324
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
325

326
  <property name="discoverySpi">
327
    <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
328
      <property name="ipFinder">
329
        <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.jdbc.TcpDiscoveryJdbcIpFinder">
330
          <property name="dataSource" ref="ds"/>
331
        </bean>
332
      </property>
333
    </bean>
334
  </property>
335
</bean>
336

337
<!-- Configured data source instance. -->
338
<bean id="ds" class="some.Datasource">
339

340
</bean>
341
----
342

343
tab:Java[]
344
[source,java]
345
----
346
include::{javaFile}[tag=jdbc,indent=0]
347
----
348

349
tab:C#/.NET[unsupported]
350

351
tab:C++[unsupported]
352

353
--
354

355

356
== Shared File System IP Finder
357

358
NOTE: Not supported in .NET/C#/{cpp}.
359

360
A shared file system can be used as a storage for nodes' IP addresses. The nodes will write their IP addresses to the file system on startup. This behavior is supported by `TcpDiscoverySharedFsIpFinder`.
361

362
[tabs]
363
--
364
tab:XML[]
365
[source,xml]
366
----
367
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
368
    <property name="discoverySpi">
369
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
370
            <property name="ipFinder">
371
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.sharedfs.TcpDiscoverySharedFsIpFinder">
372
                  <property name="path" value="/var/ignite/addresses"/>
373
                </bean>
374
            </property>
375
        </bean>
376
    </property>
377
</bean>
378
----
379
tab:Java[]
380
[source,java]
381
----
382
include::{javaFile}[tag=sharedFS,indent=0]
383
----
384
tab:C#/.NET[unsupported]
385
tab:C++[unsupported]
386
--
387

388
== ZooKeeper IP Finder
389

390
NOTE: Not supported in .NET/C#.
391

392
The link:extensions-and-integrations/zookeeper-ip-finder-ext[Apache Ignite ZooKeeper Ip Finder] extension provides
393
a TCP Discovery IP Finder that uses a ZooKeeper directory to locate other Ignite nodes to connect to.
394