1
<#import "/templates/guide.adoc" as tmpl>
2
<#import "/templates/kc.adoc" as kc>
3
<#import "/templates/links.adoc" as links>
4
<#import "/templates/profile.adoc" as profile>
7
title="Configuring logging"
8
summary="Learn how to configure Logging"
9
includedOptions="log log-*">
11
{project_name} uses the JBoss Logging framework. The following is a high-level overview for the available log handlers:18
</@profile.ifCommunity>
20
== Logging configuration
21
Logging is done on a per-category basis in {project_name}. You can configure logging for the root log level or for more specific categories such as `org.hibernate` or `org.keycloak`. This {section} describes how to configure logging.25
The following table defines the available log levels.
31
|FATAL|Critical failures with complete inability to serve any kind of request.
32
|ERROR|A significant error or problem leading to the inability to process requests.
33
|WARN|A non-critical error or problem that might not require immediate correction.
34
|INFO|{project_name} lifecycle events or important information. Low frequency.35
|DEBUG|More detailed information for debugging purposes, such as database logs. Higher frequency.
36
|TRACE|Most detailed debugging information. Very high frequency.
37
|ALL|Special level for all log messages.
38
|OFF|Special level to turn logging off entirely (not recommended).
41
=== Configuring the root log level
42
When no log level configuration exists for a more specific category logger, the enclosing category is used instead. When there is no enclosing category, the root logger level is used.
44
To set the root log level, enter the following command:
46
<@kc.start parameters="--log-level=<root-level>"/>
48
Use these guidelines for this command:
50
* For `_<root-level>_`, supply a level defined in the preceding table.
51
* The log level is case-insensitive. For example, you could either use `DEBUG` or `debug`.
52
* If you were to accidentally set the log level twice, the last occurrence in the list becomes the log level. For example, if you included the syntax `--log-level="info,...,DEBUG,..."`, the root logger would be `DEBUG`.
54
=== Configuring category-specific log levels
55
You can set different log levels for specific areas in {project_name}. Use this command to provide a comma-separated list of categories for which you want a different log level:57
<@kc.start parameters="--log-level=\"<root-level>,<org.category1>:<org.category1-level>\""/>
59
A configuration that applies to a category also applies to its sub-categories unless you include a more specific matching sub-category.
62
<@kc.start parameters="--log-level=\"INFO,org.hibernate:debug,org.hibernate.hql.internal.ast:info\""/>
64
This example sets the following log levels:
66
* Root log level for all loggers is set to INFO.
67
* The hibernate log level in general is set to debug.
68
* To keep SQL abstract syntax trees from creating verbose log output, the specific subcategory `org.hibernate.hql.internal.ast` is set to info. As a result, the SQL abstract syntax trees are omitted instead of appearing at the `debug` level.
70
== Enabling log handlers
71
To enable log handlers, enter the following command:
73
<@kc.start parameters="--log=\"<handler1>,<handler2>\""/>
75
The available handlers are
77
`console`, `file` and `gelf`.
78
</@profile.ifCommunity>
82
The more specific handler configuration mentioned below will only take effect when the handler is added to this comma-separated list.
85
The console log handler is enabled by default, providing unstructured log messages for the console.
87
=== Configuring the console log format
88
{project_name} uses a pattern-based logging formatter that generates human-readable text logs by default.90
The logging format template for these lines can be applied at the root level. The default format template is:
92
* `%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n`94
The format string supports the symbols in the following table:
98
|Symbol|Summary|Description
100
|%%|%|Renders a simple % character.
101
|%c|Category|Renders the log category name.
102
|++%d{xxx}++|Date|Renders a date with the given date format string.String syntax defined by `java.text.SimpleDateFormat`103
|%e|Exception|Renders a thrown exception.
104
|%h|Hostname|Renders the simple host name.
105
|%H|Qualified host name|Renders the fully qualified hostname, which may be the same as the simple host name, depending on the OS configuration.
106
|%i|Process ID|Renders the current process PID.
107
|%m|Full Message|Renders the log message and an exception, if thrown.
108
|%n |Newline|Renders the platform-specific line separator string.
109
|%N|Process name|Renders the name of the current process.
110
|%p|Level|Renders the log level of the message.
111
|%r|Relative time|Render the time in milliseconds since the start of the application log.
112
|%s|Simple message|Renders only the log message without exception trace.
113
|%t|Thread name|Renders the thread name.
114
|%t++{id}++|Thread ID|Render the thread ID.115
|%z{<zone name>}|Timezone|Set the time zone of log output to <zone name>.116
|%L|Line number|Render the line number of the log message.
119
=== Setting the logging format
120
To set the logging format for a logged line, perform these steps:
122
. Build your desired format template using the preceding table.
123
. Enter the following command:
125
<@kc.start parameters="--log-console-format=\"\'<format>\'\""/>
127
Note that you need to escape characters when invoking commands containing special shell characters such as `;` using the CLI. Therefore, consider setting it in the configuration file instead.
129
.Example: Abbreviate the fully qualified category name
130
<@kc.start parameters="--log-console-format=\"\'%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c{3.}] (%t) %s%e%n\'\""/>132
This example abbreviates the category name to three characters by setting `[%c{3.}]` in the template instead of the default `[%c]`.134
=== Configuring JSON or plain console logging
135
By default, the console log handler logs plain unstructured data to the console. To use structured JSON log output instead, enter the following command:
137
<@kc.start parameters="--log-console-output=json"/>
142
{"timestamp":"2022-02-25T10:31:32.452+01:00","sequence":8442,"loggerClassName":"org.jboss.logging.Logger","loggerName":"io.quarkus","level":"INFO","message":"Keycloak 18.0.0-SNAPSHOT on JVM (powered by Quarkus 2.7.2.Final) started in 3.253s. Listening on: http://0.0.0.0:8080","threadName":"main","threadId":1,"mdc":{},"ndc":"","hostName":"host-name","processName":"QuarkusEntryPoint","processId":36946}145
When using JSON output, colors are disabled and the format settings set by `--log-console-format` will not apply.
147
To use unstructured logging, enter the following command:
149
<@kc.start parameters="--log-console-output=default"/>
154
2022-03-02 10:36:50,603 INFO [io.quarkus] (main) Keycloak 18.0.0-SNAPSHOT on JVM (powered by Quarkus 2.7.2.Final) started in 3.615s. Listening on: http://0.0.0.0:8080
158
Colored console log output for unstructured logs is disabled by default. Colors may improve readability, but they can cause problems when shipping logs to external log aggregation systems. To enable or disable color-coded console log output, enter following command:
160
<@kc.start parameters="--log-console-color=<false|true>"/>
163
As an alternative to logging to the console, you can use unstructured logging to a file.
165
=== Enable file logging
166
Logging to a file is disabled by default. To enable it, enter the following command:
168
<@kc.start parameters="--log=\"console,file\""/>
170
A log file named `keycloak.log` is created inside the `data/log` directory of your Keycloak installation.
172
=== Configuring the location and name of the log file
174
To change where the log file is created and the file name, perform these steps:
176
. Create a writable directory to store the log file.
178
If the directory is not writable, {project_name} will start correctly, but it will issue an error and no log file will be created.182
<@kc.start parameters="--log=\"console,file\" --log-file=<path-to>/<your-file.log>"/>
184
=== Configuring the file handler format
185
To configure a different logging format for the file log handler, enter the following command:
187
<@kc.start parameters="--log-file-format=\"<pattern>\""/>
189
See <<Configuring the console log format>> for more information and a table of the available pattern configuration.
191
<@profile.ifCommunity>
193
== Centralized logging using GELF
194
NOTE: The support for GELF log handler is deprecated and will be removed in a future {project_name} release.196
{project_name} can send logs to a centralized log management system such as the following:199
* Logstash, inside the Elasticsearch, Logstash, Kibana (ELK) logging stack
200
* Fluentd, inside the Elasticsearch, Fluentd, Kibana (EFK) logging stack
202
{project_name} uses the https://quarkus.io/guides/centralized-log-management[Quarkus Logging GELF] extension to support these environments.204
=== Enabling the GELF handler
205
To enable logging using GELF, add it to the list of activated log handlers.
208
<@kc.start parameters="--log=\"console,gelf\""/>
210
=== Configuring the GELF handler
212
To configure the Host and Port of your centralized logging system, enter the following command and substitute the values with your specific values:
213
.Host and port of the GELF server:
214
<@kc.start parameters="--log=\"console,gelf\" --log-gelf-host=myhost --log-gelf-port=12345"/>
216
When the GELF handler is enabled, the host is using `localhost` as host value and UDP for communication. To use TCP instead of UDP, prefix the host value with `tcp:`. The Default port is `12201`.
218
.Include or exclude Stacktraces
219
{project_name} includes the complete Stacktrace inside the `StackTrace` field. To exclude this field, enter the following command:221
<@kc.start parameters="--log=\"console,gelf\" --log-gelf-include-stack-trace=false"/>
223
.Configure the timestamp format
224
You can change the format of the `timestamp` field. For example, you can include the date and time down to seconds by entering the following command:
226
<@kc.start parameters="--log=\"console,gelf\" --log-gelf-timestamp-format=\"\'yyyy-MM-dd HH:mm:ss\'\""/>
228
Alternatively, you could use the config file to avoid escaping:
232
log-gelf-timestamp-format=yyyy-MM-dd HH:mm:ss
235
The default timestamp format is `yyyy-MM-dd HH:mm:ss,SSS`. You can use the https://docs.oracle.com/javase/10/docs/api/java/text/SimpleDateFormat.html[available SimpleDateFormat patterns] to define an appropriate timestamp.
237
.Configure the facility
238
The `facility` field is an indicator of the process or program that is the source of log messages. The default value is `keycloak`. To set this field to your preferred identifier, enter the following command:
240
<@kc.start parameters="--log=\"console,gelf\" --log-gelf-facility=MyKeycloak"/>
242
To use the CLI to configure {project_name} and use whitespaces for `facility`, enter the following command:244
<@kc.start parameters="--log=\"console,gelf\" --log-gelf-facility=\"\'my keycloak\'\""/>
246
Alternatively, you could use the config file to avoid escaping:
250
log-gelf-facility=my keycloak
253
.Configure the default message size
254
To change the default message size of 8kb (8192 bytes) of GELF log messages for {project_name}, enter the following command:256
<@kc.start parameters="--log=\"console,gelf\" --log-gelf-max-message-size=16384"/>
258
The maximum size of one GELF log message is set in Bytes. The preceding example increases the size to 16kb. When messages exceed the maximum size, GELF submits the message in multiple chunks.
260
.Configure sending of message parameters
261
{project_name} includes message parameters of the occurred log event. These fields appear in the output as `MessageParam0`, `MessageParam1`, and so on, depending on the parameter length.262
To switch off this behavior, enter the following command:
264
<@kc.start parameters="--log=\"console,gelf\" --log-gelf-include-message-parameters=false"/>
266
.Configure sending of source code location
267
{project_name} includes the `SourceClassName`, `SourceMethodName` and `SourceSimpleClassName` fields in the GELF log messages. These fields provide detail on the location of an exception that occurred. To stop sending these fields, enter the following command:269
<@kc.start parameters="--log=\"console,gelf\" --log-gelf-include-location=false"/>
271
=== Example: Send logs to Graylog
272
The following example shows how to send {project_name} logs to the Graylog centralized logging stack. This example assumes you have a container tool such as https://www.docker.com/[docker] installed to start the `compose.yml`.274
==== Starting the Graylog stack
275
The composed stack consists of:
287
image: docker.io/elastic/elasticsearch:7.10.2
291
ES_JAVA_OPTS: "-Xms512m -Xmx512m"
292
discovery.type: "single-node"
302
image: graylog/graylog:4.3.3
308
GRAYLOG_HTTP_EXTERNAL_URI: "http://127.0.0.1:9000/"
309
# CHANGE ME (must be at least 16 characters)!
310
GRAYLOG_PASSWORD_SECRET: "forpasswordencryption"
312
GRAYLOG_ROOT_PASSWORD_SHA2: "8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918"
324
Copy and save the example locally into a `compose.yml` file and enter this command:
330
After a few seconds, the Stack is ready to serve requests.
332
==== Creating a Graylog UDP Input
333
Once the stack is running, you need to create a UDP Input Graylog listens to. You can create it from the Graylog web UI (System → Input → Select GELF UDP) available at http://localhost:9000 or using the API:
335
This `curl` example creates a new GELF UDP Input using the API and the default Graylog login credentials (admin/admin).
339
curl -H "Content-Type: application/json" -H "Authorization: Basic YWRtaW46YWRtaW4=" -H "X-Requested-By: curl" -X POST -v -d \
340
'{"title":"udp input","configuration":{"recv_buffer_size":262144,"bind_address":"0.0.0.0","port":12201,"decompress_size_limit":8388608},"type":"org.graylog2.inputs.gelf.udp.GELFUDPInput","global":true}' \341
http://localhost:9000/api/system/inputs
344
If the stack is still in the bootstrap phase, you receive a response containing `* Empty reply from server`. A successful response includes `HTTP/1.1 201 Created` to indicate that the UDP input is created.
346
==== Configure {project_name} to send logs using GELF347
{project_name} needs to be configured to send logs using GELF. The appropriate configuration can be seen in the following keycloak.conf example. The example includes the `log-gelf-host` and `log-gelf-port` values. These are optional values that are included for illustration purposes; default values exist.349
.{project_name} GELF Configuration354
log-gelf-host=localhost
358
==== Graylog: See the results
359
. Open your web browser, go to `http://localhost:9000`.
360
. Log in to the Graylog web UI using the administrator credentials (admin/admin).
361
. Go to Streams, All Messages.
362
. Start updating the stream by pressing the Play button in the upper right corner.
363
. Start {project_name} using `start` or `start-dev` and your GELF config.365
After a few seconds, {project_name} messages appear in the Graylog dashboard.367
=== Example Setup using the ELK Stack
368
The following example shows how to send {project_name} logs to the ELK centralized logging stack. It assumes you have a container tool such as https://www.docker.com/[docker] installed to start the `compose.yml`.370
==== Enable the logstash GELF plugin and create a pipeline
371
Logstash uses an input plugin that understands and parses the GELF format. To activate this plugin when you are starting the ELK stack later on, create a directory `pipelines` and a file `gelf.conf` located in this directory. Then create an empty `compose.yml` in the parent directory.
383
Add the following contents to `pipelines/gelf.conf` and save it:
395
hosts => ["http://elasticsearch:9200"]
400
This file activates and configures the logstash GELF plugin and points it to the right elasticsearch instance.
402
==== Starting the ELK stack
403
The composed stack consists of:
409
Copy the following content to your `compose.yml` file:
413
# Launch Elasticsearch
418
image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.2
423
ES_JAVA_OPTS: "-Xms512m -Xmx512m"
428
image: docker.elastic.co/logstash/logstash-oss:6.8.2
430
- source: ./pipelines #the source dir gelf.conf resides
431
target: /usr/share/logstash/pipeline
443
image: docker.elastic.co/kibana/kibana-oss:6.8.2
455
Start the stack by entering the following command:
461
After a few seconds the Stack should be ready to serve requests.
463
==== Configuring {project_name} to send logs using GELF464
{project_name} needs to be configured to send logs using GELF. The appropriate configuration can be seen in the following keycloak.conf example. This example includes the `log-gelf-host` and `log-gelf-port` values. These are optional values, which are included for illustration purposes; default values exist.466
.{project_name} Gelf Configuration471
log-gelf-host=localhost
475
With this configuration applied, start {project_name} using `start-dev` or `start`.477
==== Kibana: See the results
478
Open http://localhost:5601 to reach the Kibana dashboard. The exact configuration of a good monitoring dashboard is out of scope for this {section}. To find out if logs sent by {project_name} are delivered to Kibana, open the http://localhost:5601/app/kibana#/dev_tools/console?_g=()[Dev Tools] and execute the default `match_all` query. The logs should appear in the result field.480
=== Configure a different log level for the GELF logger
481
To keep log storage costs and verbosity low, it is often wanted to only store a subset of the verbose application logs inside a centralized log management system. To configure {project_name} to use a different log level for the logs you want to ingest, use the following configuration:486
log-gelf-level=<desired-log-level>
491
To only see occurred log levels of warn and above in your centralized logging stack, but still see INFO level logs on the applications console logs, use the following configuration:
501
Looking at your ingested logs, you will see only messages of level warn or above will appear.
503
Keep in mind that `--log-level` is setting the leading log level, so for example when you invoke the following command:
505
<@kc.start parameters="--log=\"console,gelf\" --log-level=error --log-gelf-level=info"/>
507
nothing below the error level will be sent to your logging stack. That means that even GELF in this example will receive only error level log messages.
509
=== Configure additional key values
510
Currently, the {project_name} configuration does not support partly dynamic configuration keys, as they are used in quarkus properties. For example, they are used when defining `quarkus.log.handler.gelf.additional-field.<my-name>.value`.512
To add user-defined fields, you can provide these fields through a quarkus.properties file. See <@links.server id="configuration"/> and the _Using raw Quarkus properties_ section.
514
</@profile.ifCommunity>