apache-ignite

1
// Licensed to the Apache Software Foundation (ASF) under one or more
2
// contributor license agreements.  See the NOTICE file distributed with
3
// this work for additional information regarding copyright ownership.
4
// The ASF licenses this file to You under the Apache License, Version 2.0
5
// (the "License"); you may not use this file except in compliance with
6
// the License.  You may obtain a copy of the License at
7
//
8
// http://www.apache.org/licenses/LICENSE-2.0
9
//
10
// Unless required by applicable law or agreed to in writing, software
11
// distributed under the License is distributed on an "AS IS" BASIS,
12
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
// See the License for the specific language governing permissions and
14
// limitations under the License.
15
= Setting Up
16

17
[NOTE]
18
====
19
[discrete]
20
=== Configuring .NET, Python, Node.JS and other programming languages
21

22
* .NET developers: refer to the link:net-specific/net-configuration-options[Ignite.NET Configuration] section
23
* Developers of Python, Node.JS, and other programming languages: use this page to configure your
24
Java-powered Ignite cluster and link:thin-clients/getting-started-with-thin-clients[thin clients] section to set up
25
your language-specific applications that will be working with the cluster.
26
====
27

28
== System Requirements
29

30
Ignite was tested on:
31

32
include::includes/prereqs.adoc[]
33

34
== Running Ignite with Java 11 or later
35

36
include::includes/java9.adoc[]
37

38

39
== Using Binary Distribution
40

41
* Download the appropriate binary package from https://ignite.apache.org/download.cgi[Apache Ignite Downloads^].
42
* Unzip the archive into a directory.
43
* (Optional) Set the `IGNITE_HOME` environment variable to point to the
44
installation folder and make sure there is no trailing `/` in the path.
45

46
== Using Maven
47

48
The easiest way to use Ignite is to add it to your pom.
49

50
[source, xml,subs="attributes,specialchars" ]
51
----
52

53
<properties>
54
    <ignite.version>{version}</ignite.version>
55
</properties>
56

57
<dependencies>
58
    <dependency>
59
        <groupId>org.apache.ignite</groupId>
60
        <artifactId>ignite-core</artifactId>
61
        <version>${ignite.version}</version>
62
    </dependency>
63
</dependencies>
64
----
65

66
The 'ignite-core' library contains the core functionality of Ignite.
67
Addition functionality is provided by various Ignite modules.
68

69
The following are the two most commonly used modules:
70

71

72
* `ignite-spring` (support for link:understanding-configuration#spring-xml-configuration[XML-based configuration])
73
* `ignite-indexing` (support for SQL indexing)
74

75

76
[source, xml]
77
----
78
<dependency>
79
    <groupId>org.apache.ignite</groupId>
80
    <artifactId>ignite-spring</artifactId>
81
    <version>${ignite.version}</version>
82
</dependency>
83
<dependency>
84
    <groupId>org.apache.ignite</groupId>
85
    <artifactId>ignite-indexing</artifactId>
86
    <version>${ignite.version}</version>
87
</dependency>
88

89
----
90

91

92

93

94
== Using Docker
95

96
If you want to run Ignite in Docker, refer to the link:installation/installing-using-docker[Docker Deployment] section.
97

98
== Configuring Work Directory
99

100
Ignite uses a work directory to store your application data (if you use the link:persistence/native-persistence[Native Persistence] feature), index files, metadata information, logs, and other files. The default work directory is as follows:
101

102
* `$IGNITE_HOME/work`, if the `IGNITE_HOME` system property is defined. This is the case when you start Ignite using the `bin/ignite.sh` script from the distribution package.
103
* `./ignite/work`, this path is relative to the directory where you launch your application.
104

105
There are several ways you can change the default work directory:
106

107
. As an environmental variable:
108
+
109
[source, shell]
110
----
111
export IGNITE_WORK_DIR=/path/to/work/directory
112
----
113

114
. In the node configuration:
115
+
116
[tabs]
117
--
118
tab:XML[]
119

120
[source, xml]
121
----
122
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
123
    <property name="workDirectory" value="/path/to/work/directory"/>
124
    <!-- other properties -->
125
</bean>
126
----
127
tab:Java[]
128
[source,java]
129
----
130
include::{javaCodeDir}/UnderstandingConfiguration.java[tag=dir,indent=0]
131
----
132

133
tab:C#/.NET[]
134
[source,csharp]
135
----
136
include::code-snippets/dotnet/UnderstandingConfiguration.cs[tag=SettingWorkDir,indent=0]
137
----
138

139
tab:C++[]
140
[source,cpp]
141
----
142
include::code-snippets/cpp/src/setting_work_directory.cpp[tag=setting-work-directory,indent=0]
143
----
144
--
145

146

147

148
== Enabling Modules
149

150
Ignite ships with a number of modules and have a lot of extensions that provide various functionality.
151
You can enable modules or extensions one by one, as required.
152

153
All modules are included in the binary distribution, but by default they
154
are disabled (except for the `ignite-core`, `ignite-spring`, `ignite-control-utility` and
155
`ignite-indexing` modules). Modules can be found in the `libs/optional`
156
directory of the distribution package (each module is located in a
157
separate sub-directory).
158

159
Also, you can download any of Ignite extensions you require.
160

161
Depending on how you use Ignite, you can enable modules or extensions using one of
162
the following methods:
163

164
* If you use the binary distribution, move the
165
`libs/optional/{module-dir}` to the `libs` directory before starting the
166
node.
167
* Add libraries from `libs/optional/{module-dir}` to the classpath of
168
your application.
169
* Add a module as a Maven dependency to your project.
170
+
171
[source, xml]
172
----
173
<dependency>
174
    <groupId>org.apache.ignite</groupId>
175
    <artifactId>ignite-log4j2</artifactId>
176
    <version>${ignite.version}</version>
177
</dependency>
178
----
179

180

181
The following modules have LGPL dependencies and, therefore, can't be deployed on the Maven Central repository:
182

183
* ignite-hibernate (Apache Ignite Extensions)
184
* ignite-geospatial (Apache Ignite Extensions)
185
* ignite-schedule (deprecated)
186

187
To use these modules, you will need to build them from sources and add to your project.
188
For example, to install the `ignite-hibernate` into your local repository and create distribution binaries,
189
run the following command from the Ignite Extensions sources:
190

191
[source, shell]
192
----
193
mvn clean install -DskipTests -f modules/hibernate-ext -Pextension-release
194
----
195

196
The following modules are available:
197

198
[width="100%",cols="1,2",options="header",]
199
|=======================================================================
200
|Module’s artifactId |Description
201
|ignite-cassandra-serializers | The Ignite Cassandra Serializers module provides additional serializers to store objects as BLOBs in Cassandra. The module could be used as in conjunction with the Ignite Cassandra Store module.
202

203
|ignite-cassandra-store | Ignite Cassandra Store provides a CacheStore implementation backed by the  Cassandra database.
204

205
|ignite-direct-io | Ignite Direct IO is a plugin that provides a page store with the ability to write and read cache partitions in O_DIRECT mode.
206

207
|ignite-indexing | link:SQL/indexes[SQL querying and indexing]
208

209
|ignite-jcl |Support for the Jakarta Common Logging (JCL) framework.
210

211
|ignite-jta |Integration of Ignite transactions with JTA.
212

213
|ignite-kafka | Ignite Kafka Streamer provides capability to stream data from Kafka to Ignite caches.
214

215
|ignite-kubernetes | Ignite Kubernetes module provides a TCP Discovery IP Finder that uses a dedicated Kubernetes service for IP addresses lookup of Ignite pods containerized by Kubernetes.
216

217
|ignite-log4j2 |Support for Log4j2
218

219

220
|ignite-ml | Ignite ML Grid provides machine learning features and relevant data structures and methods of linear algebra, including on heap and off heap, dense and sparse, local and distributed implementations.
221
Refer to the link:machine-learning/ml[Machine Learning] documentation for details.
222

223
|ignite-rest-http | Ignite REST-HTTP starts a Jetty-based server within a node that can be used to execute tasks and/or cache commands in grid using HTTP-based link:restapi[RESTful APIs].
224

225
|ignite-schedule | This  module provides functionality for scheduling jobs locally using UNIX cron-based syntax.
226

227
|ignite-slf4j | Support for link:logging#using-slf4j[SLF4J logging framework].
228

229
|ignite-tensorflow | The Ignite TensorFlow Integration Module allows using TensorFlow with Ignite. In this scenario Ignite will be a datasource for any TensorFlow model training.
230

231
|ignite-urideploy | Ignite URI Deploy module provides capabilities to deploy tasks from different sources such as File System, HTTP, or even Email.
232

233
|ignite-visor-console |Open source command line management and monitoring tool
234

235
|ignite-web | Ignite Web allows you to start nodes inside any web container based on servlet and servlet context listener. In addition, this module provides capabilities to cache web sessions in an Ignite cache.
236

237
|link:clustering/zookeeper-discovery[ignite-zookeeper] | ZooKeeper Discovery implementation.
238

239
|=======================================================================
240

241
The following extensions are available:
242

243
[width="100%",cols="1,2",options="header",]
244
|=======================================================================
245
|Extension’s artifactId |Description
246
|ignite-aop-ext | Ignite AOP module provides capability to turn any Java method to a distributed closure by adding @Gridify annotation to it.
247

248
|ignite-aws-ext|Cluster discovery on AWS S3. Refer to link:clustering/discovery-in-the-cloud#amazon-s3-ip-finder[Amazon S3 IP Finder] for details.
249

250
|ignite-azure-ext| Ignite Azure provides Azure Blob Storage-based implementation of IP finder for TCP discovery.
251

252
|ignite-cloud-ext | Ignite Cloud provides Apache jclouds implementations of the IP finder for TCP discovery.
253

254
|ignite-osgi-ext | This module provides bridging components to make Ignite run seamlessly inside an OSGi container such as Apache Karaf.
255

256
|ignite-osgi-karaf-ext | This module contains a feature repository to facilitate installing Ignite into an Apache Karaf container.
257

258
|ignite-gce-ext | Ignite GCE provides Google Cloud Storage based implementations of IP finder for TCP discovery.
259

260
|ignite-spark-ext | This module provides an implementation of Spark RDD abstraction that enables easy access to Ignite caches.
261

262
|ignite-spring-data-ext | Ignite Spring Data provides an integration with Spring Data framework.
263

264
|ignite-spring-data_2.0-ext | Ignite Spring Data 2.0 provides an integration with Spring Data 2.0 framework.
265

266
|ignite-spring-data_2.2-ext | Ignite Spring Data 2.2 provides an integration with Spring Data 2.2 framework.
267

268
|ignite-ssh | The Ignite SSH module provides capabilities to start Ignite nodes on remote machines via SSH.
269

270
|link:extensions-and-integrations/zookeeper-ip-finder-ext[ignite-zookeeper-ip-finder-ext] | TCP Discovery IP Finder that uses a ZooKeeper directory to locate other Ignite nodes to connect to.
271
|=======================================================================
272

273
== Setting JVM Options
274

275
There are several ways you can set JVM options when starting a node with the `ignite.sh` script.
276
These ways are described in the following sections.
277

278
=== JVM_OPTS System Variable
279

280
You can set the `JVM_OPTS` environment variable:
281

282
[source, shell]
283
----
284
export JVM_OPTS="$JVM_OPTS -Xmx6G -DIGNITE_TO_STRING_INCLUDE_SENSITIVE=false"; $IGNITE_HOME/bin/ignite.sh
285
----
286

287
=== Command Line Arguments
288

289
You can also pass JVM options by using the `-J` prefix:
290

291
[tabs]
292
--
293
tab:Unix[]
294
[source, shell]
295
----
296
./ignite.sh -J-Xmx6G -J-DIGNITE_TO_STRING_INCLUDE_SENSITIVE=false
297
----
298
tab:Windows[]
299
[source, shell]
300
----
301
.\ignite.bat -J-Xmx6G -J-DIGNITE_TO_STRING_INCLUDE_SENSITIVE=false
302
----
303
--
304

305
== Setting Ignite System Properties
306

307
In addition to public configuration settings, you can adjust specific, usually low-level, Ignite behavior with internal system properties. You can find all the properties with their descriptions and default values by using the command below:
308

309
[tabs]
310
--
311
tab:Unix[]
312
[source, shell]
313
----
314
./ignite.sh -systemProps
315
----
316
tab:Windows[]
317
[source, shell]
318
----
319
.\ignite.bat -systemProps
320
----
321
--
322

323
Example of the output:
324

325
[source, text]
326
ignite.sh -systemProps
327
IGNITE_AFFINITY_HISTORY_SIZE                                    - [Integer] Maximum size for affinity assignment history. Default is 25.
328
IGNITE_ALLOW_ATOMIC_OPS_IN_TX                                   - [Boolean] Allows atomic operations inside transactions. Default is true.
329
IGNITE_ALLOW_DML_INSIDE_TRANSACTION                             - [Boolean] When set to true, Ignite will allow executing DML operation (MERGE|INSERT|UPDATE|DELETE) within transactions for non MVCC mode. Default is false.
330
IGNITE_ALLOW_START_CACHES_IN_PARALLEL                           - [Boolean] Allows to start multiple caches in parallel. Default is true.
331
...
332

333

334
== Configuration Recommendations
335

336
Below are some recommended configuration tips aimed at making it easier for
337
you to operate an Ignite cluster or develop applications with Ignite.
338

339
=== Setting Work Directory
340

341
If you are going to use either binary distribution or Maven, you are
342
encouraged to set up the work directory for Ignite.
343
The work directory is used to store metadata information, index files, your application data (if you use the link:persistence/native-persistence[Native Persistence] feature), logs, and other files.
344
We recommend you always set up the work directory.
345

346

347
=== Recommended Logging Configuration
348

349
Logs play an important role when it comes to troubleshooting and finding what went wrong. Here are a few general tips on how to manage your log files:
350

351
* Start Ignite in verbose mode:
352
   - If you use `ignite.sh`, specify the `-v` option.
353
   - If you start Ignite from Java code, set the following system variable: `IGNITE_QUIET=false`.
354
* Do not store log files in the `/tmp` folder. This folder is cleared up every time the server is restarted.
355
* Make sure that there is enough space available on the storage where the log files are stored.
356
* Archive old log files periodically to save on storage space.
357