apache-ignite
65 строк · 3.3 Кб
1// Licensed to the Apache Software Foundation (ASF) under one or more
2// contributor license agreements. See the NOTICE file distributed with
3// this work for additional information regarding copyright ownership.
4// The ASF licenses this file to You under the Apache License, Version 2.0
5// (the "License"); you may not use this file except in compliance with
6// the License. You may obtain a copy of the License at
7//
8// http://www.apache.org/licenses/LICENSE-2.0
9//
10// Unless required by applicable law or agreed to in writing, software
11// distributed under the License is distributed on an "AS IS" BASIS,
12// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13// See the License for the specific language governing permissions and
14// limitations under the License.
15= Hyper-parameter tuning
16
17In machine learning, hyperparameter optimization or tuning is the problem of choosing a set of optimal hyperparameters for a learning algorithm. A hyperparameter is a parameter whose value is used to control the learning process. By contrast, the values of other parameters (typically node weights) are learned.
18
19In Apache Ignite ML you could tune the model by changing of hyper-parameters (preprocessor and trainer's hyper-parameters).
20
21The main object to keep the all possible values of hyper-parameters is the ParamGrid object.
22
23
24[source, java]
25----
26DecisionTreeClassificationTrainer trainerCV = new DecisionTreeClassificationTrainer();
27
28ParamGrid paramGrid = new ParamGrid()
29.addHyperParam("maxDeep", trainerCV::withMaxDeep,
30new Double[] {1.0, 2.0, 3.0, 4.0, 5.0, 10.0})
31.addHyperParam("minImpurityDecrease", trainerCV::withMinImpurityDecrease,
32new Double[] {0.0, 0.25, 0.5});
33----
34
35There are a few approaches to find the optimal set of hyper-parameters:
36
37* *BruteForce (GridSearch)* - The traditional way of performing hyperparameter optimization has been grid search, or a parameter sweep, which is simply an exhaustive searching through a manually specified subset of the hyperparameter space of a learning algorithm.
38* *Random search* - It replaces the exhaustive enumeration of all combinations by selecting them randomly.
39* *Evolutionary optimization* - Evolutionary optimization is a methodology for the global optimization of noisy black-box functions. In hyperparameter optimization, evolutionary optimization uses evolutionary algorithms to search the space of hyperparameters for a given algorithm.
40
41The Random Search ParamGrid is could be set up as follows:
42
43
44[source, java]
45----
46ParamGrid paramGrid = new ParamGrid()
47.withParameterSearchStrategy(
48new RandomStrategy()
49.withMaxTries(10)
50.withSeed(12L))
51.addHyperParam("p", normalizationTrainer::withP,
52new Double[] {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0})
53.addHyperParam("maxDeep", trainerCV::withMaxDeep,
54new Double[] {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0})
55.addHyperParam("minImpurityDecrease", trainerCV::withMinImpurityDecrease,
56new Double[] {0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 1.0});
57----
58
59
60[TIP]
61====
62Performance Tip:
63
64The GridSearch (BruteForce) and Evolutionary optimization methods could be easily parallelized because all training runs are independent from each other.
65====
66