apache-ignite
224 строки · 10.5 Кб
1// Licensed to the Apache Software Foundation (ASF) under one or more
2// contributor license agreements. See the NOTICE file distributed with
3// this work for additional information regarding copyright ownership.
4// The ASF licenses this file to You under the Apache License, Version 2.0
5// (the "License"); you may not use this file except in compliance with
6// the License. You may obtain a copy of the License at
7//
8// http://www.apache.org/licenses/LICENSE-2.0
9//
10// Unless required by applicable law or agreed to in writing, software
11// distributed under the License is distributed on an "AS IS" BASIS,
12// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13// See the License for the specific language governing permissions and
14// limitations under the License.
15= External Storage
16:javaFile: {javaCodeDir}/ExternalStorage.java
17
18== Overview
19
20You can use Ignite as a caching layer on top of an existing database such as an RDBMS or NoSQL databases, for example, Apache Cassandra or MongoDB.
21This use case accelerates the underlying database by employing in-memory processing.
22
23Ignite provides an out-of-the-box integration with Apache Cassandra.
24For other NoSQL databases for which integration is not available off-the-shelf, you can provide your own link:persistence/custom-cache-store[implementation of the `CacheStore` interface].
25
26The two main use cases where an external storage can be used include:
27
28* A caching layer to an existing database. In this scenario, you can improve the processing speed by loading data into memory. You can also bring SQL support to a database that does not have it (when all data is loaded into memory).
29
30* You want to persist the data in an external database (instead of using the link:persistence/native-persistence[native persistence]).
31
32image::images/external_storage.png[]
33
34////////////////////////////////////////////////////////////////////////////////
35What follows is a java-specific documentation
36////////////////////////////////////////////////////////////////////////////////
37
38
39The `CacheStore` interface extends both `javax.cache.integration.CacheLoader` and `javax.cache.integration.CacheWriter`, which are used for _read-through_ and _write-through_ features respectively. You can also implement each of the interfaces individually and provide them to the cache configuration separately.
40
41NOTE: In addition to key-value operations, Ignite writes through the results of SQL INSERT, UPDATE, and MERGE queries. However, SELECT queries never read through data from the external database.
42
43=== Read-Through and Write-Through
44
45Read-through means that the data is read from the underlying persistent store if it is not available in the cache.
46Note that this is true only for get operations made through the key-value API; SELECT queries never read through data from the external database.
47To execute select queries, the data must be pre-loaded from the database into the cache by calling the `loadCache()` method.
48
49Write-through means that the data is automatically persisted when it is updated in the cache.
50All read-through and write-through operations participate in cache transactions and are committed or rolled back as a whole.
51
52=== Write-Behind Caching
53
54In a simple write-through mode, each put and remove operation involves a corresponding request to the persistent store; therefore, the overall duration of the update operation might be relatively long. Additionally, an intensive cache update rate can cause an extremely high storage load.
55
56For such cases, you can enable the _write-behind_ mode, in which update operations are performed asynchronously. The key concept of this approach is to accumulate updates and asynchronously flush them to the underlying database as a bulk operation.
57You can trigger flushing of data based on time-based events (the maximum time that data entry can reside in the queue is limited), queue-size events (the queue is flushed when its size reaches some particular point), or both of them (whichever occurs first).
58
59[WARNING]
60====
61[discrete]
62=== Performance vs. Consistency
63
64Enabling write-behind caching increases performance by performing asynchronous updates, but this can lead to a potential drop in consistency as some updates could be lost due to node failures or crashes.
65
66====
67
68
69
70With the write-behind approach, only the last update to an entry is written to the underlying storage.
71If a cache entry with a key named `key1` is sequentially updated with values `value1`, `value2`, and `value3` respectively, then only a single store request for the `(key1, value3)` pair is propagated to the persistent store.
72
73[NOTE]
74====
75[discrete]
76=== Update Performance
77
78Batch operations are usually more efficient than a sequence of individual operations.
79You can exploit this feature by enabling batch operations in the write-behind mode.
80Update sequences of similar types (put or remove) can be grouped to a single batch.
81For example, if you put the pairs `(key1, value1)`, `(key2, value2)`, `(key3, value3)` into the cache sequentially, the three operations are batched into a single `CacheStore.putAll(...)` operation.
82====
83
84
85== RDBMS Integration
86
87To use an RDBMS as an underlying storage, you can use one of the following implementations of `CacheStore`.
88
89* `CacheJdbcPojoStore` -- stores objects as a set of fields using reflection. Use this implementation if you are adding Ignite on top of an existing database and want to use specific fields (or all of them) from the underlying table.
90* `CacheJdbcBlobStore` -- stores objects in the underlying database in the Blob format. This option is useful in scenarios when you use an external database as a persistent storage and want to store your data in a simple format.
91
92
93
94////////////////////////////////////////////////////////////////////////////////
95To configure a `CacheStore`:
96
97. Add the JDBC driver of the database you are using to the classpath of your application.
98. Set the `CacheConfiguration.cacheStoreFactory` property of `CacheConfiguration` to use one of the implementation of `CacheStore`. You will need to provide connection parameters in `cacheStoreFactory`.
99
100Once the configuration is set, you can use the `IgniteCache.loadCache(...)` method to load the data from the database into the respective caches.
101////////////////////////////////////////////////////////////////////////////////
102
103
104
105Below are configuration examples for both implementations of `CacheStore`.
106
107
108=== CacheJdbcPojoStore
109
110With `CacheJdbcPojoStore`, you can store objects as a set of fields and can configure the mapping between table columns and objects fields via the configuration.
111
112. Set the `CacheConfiguration.cacheStoreFactory` property to `org.apache.ignite.cache.store.jdbc.CacheJdbcPojoStoreFactory` and provide the following properties:
113+
114--
115* `dataSourceBean` -- database connection credentials: URL, user, password.
116* `dialect` -- the class that implements the SQL dialect compatible with your database.
117Ignite provides out-of-the-box implementations for MySQL, Oracle, H2, SQLServer, and DB2 databases.
118These dialects can be found in the `org.apache.ignite.cache.store.jdbc.dialect` package of the Ignite distribution.
119* `types` -- this property is required to define mappings between the database table and the corresponding POJO (see POJO configuration example below).
120--
121. Optionally, configure link:SQL/sql-api#query-entities[query entities] if you want to execute SQL queries on the cache.
122
123The following example demonstrates how to configure an Ignite cache on top of a MySQL table.
124The table has 2 columns: `id` (INTEGER) and `name` (VARCHAR), which are mapped to objects of the `Person` class.
125
126
127You can configure `CacheJdbcPojoStore` via both the XML configuration and Java code.
128
129
130[tabs]
131--
132tab:XML[]
133[source,xml]
134----
135include::code-snippets/xml/cache-jdbc-pojo-store.xml[tags=, indent=0]
136----
137
138tab:Java[]
139[source,java]
140----
141include::{javaFile}[tag=pojo,indent=0]
142----
143--
144
145.Person Class
146[source,java]
147----
148include::{javaFile}[tag=person,indent=0]
149----
150
151=== CacheJdbcBlobStore
152`CacheJdbcBlobStore` stores objects in the underlying database in the blob format.
153It creates a table named 'ENTRIES', with the 'akey' and 'val' columns (both have the `binary` type).
154
155You can change the default table definition by providing a custom create table query and DML queries used to load, delete, and update the data.
156Refer to javadoc:org.apache.ignite.cache.store.jdbc.CacheJdbcBlobStore[] for details.
157
158In the example below, the objects of the Person class are stored as an array of bytes in a single column.
159
160[tabs]
161--
162tab:XML[]
163[source,xml]
164----
165<bean id="mysqlDataSource" class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource">
166<property name="URL" value="jdbc:mysql://[host]:[port]/[database]"/>
167<property name="user" value="YOUR_USER_NAME"/>
168<property name="password" value="YOUR_PASSWORD"/>
169</bean>
170
171<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
172<property name="cacheConfiguration">
173<list>
174<bean class="org.apache.ignite.configuration.CacheConfiguration">
175<property name="name" value="PersonCache"/>
176<property name="cacheStoreFactory">
177<bean class="org.apache.ignite.cache.store.jdbc.CacheJdbcBlobStoreFactory">
178<property name="dataSourceBean" value = "mysqlDataSource" />
179</bean>
180</property>
181</bean>
182</list>
183</property>
184</bean>
185----
186tab:Java[]
187[source,java]
188----
189include::{javaFile}[tag=blob1,indent=0]
190----
191
192tab:C#/.NET[]
193tab:C++[]
194--
195
196== Loading Data
197
198After you configure the cache store and start the cluster, load the data from the database into your cluster as follows:
199
200[source,java]
201----
202include::{javaFile}[tag=blob2,indent=0]
203----
204
205== NoSQL Database Integration
206You can integrate Ignite with any NoSQL database by implementing the `CacheStore` interface.
207
208CAUTION: Even though Ignite supports distributed transactions, it doesn't make your NoSQL database transactional, unless the database supports transactions out of the box.
209
210
211=== Cassandra Integration
212
213Ignite provides an out-of-the-box implementation of `CacheStore` that enables you to use Apache Cassandra as a persistent
214storage. This implementation utilizes Cassandra's link:http://www.datastax.com/dev/blog/java-driver-async-queries[asynchronous queries, window=_blank]
215to provide high performance batch operations such as `loadAll()`, `writeAll()` and `deleteAll()`, and automatically creates
216all necessary tables and namespaces in Cassandra.
217
218Refer to link:extensions-and-integrations/cassandra/overview[this documentation section] for configuration and usage guidelines.
219
220////
221== Implementing Custom CacheStore
222
223See link:advanced-topics/custom-cache-store[Implementing Custom Cache Store].
224////
225