Keycloak

1
= Testing
2

3
The `tests` directory contains a testsuite for the documentation. It has a separate test for each guide.
4

5
Currently we are testing the following:
6

7
* Variables - check if there are variables that are not resolved
8
* Includes - check if there are includes where the included file is not found
9
* Images - check if there are images included that are not missing
10
* Internal links - check that all internal links (HTML anchors) point to a valid id
11
* External links - check that all external links work, including HTML anchors
12

13
More details about each test below and what to do if the tests are not working.
14

15
By default the tests run against the locally built documentation. It is also possible to run the test against an
16
externally built and hosted version of the documentation.
17

18

19
== Running the tests
20

21
The tests are ran with a regular build:
22

23
[source,bash]
24
----
25
# mvn clean install
26
----
27

28
The tests are also ran when building the product documentation:
29

30
[source,bash]
31
----
32
# mvn clean install -Dproduct
33
----
34

35
You can also run the tests from within your IDE, but you have to manually build the guides first.
36

37
== Testing externally built and hosted
38

39
The tests can check externally built and hosted documentation instead of the locally built version. To do this set
40
the `guideBaseUrl` system property to the base URL of the externally hosted documentation.
41

42
For example to test Keycloak documentation run:
43
[source,bash]
44
----
45
mvn -f test test -DguideBaseUrl=http://<HOSTNAME>/docs/{version}
46
----
47

48
Or for example to test RH-SSO documentation run:
49
[source,bash]
50
----
51
mvn -f test test -DguideBaseUrl=https://<HOSTNAME>/documentation/en-us/red_hat_single_sign-on/{version}/html-single/
52
----
53

54
NOTE: `{version}` is replaced with `{project_versionDoc}` from document-attributes.
55

56
== Travis
57

58
Travis will run the tests both for community and product versions of the documentation for every PR and against
59
branches when they are updated.
60

61

62
== Tests
63

64
=== Variables
65

66
Missing variables are detected by looking for '{name}' in the built documentation. There are some cases where we
67
want to have these though, so the following are excluded:
68

69
* If prefixed with `$` (for example `${name}`)
70
* If prefixed with `/` (for example `http://localhost:8080/{realm-name}`)
71
* If listed in `tests/src/resources/ignored-variables`
72

73
=== Includes
74

75
Checking for missing includes is pretty simple as AsciiDoctor will output _Unresolved directive..._ in the generated
76
HTML so we're just searching for that.
77

78
=== Images
79

80
Missing images are checked by searching for `<img ..>` tags and checking that they src attribute refers to an existing
81
image.
82

83
=== Internal links
84

85
Internal links are checked by searching for `<a href="#link">` and checking that the built documentation contains a
86
corresponding `<a id="link">` element.
87

88
=== External links
89

90
External links are checked by searching for `<a href="http://link">` then a connection is opened to verify if the link
91
is valid. If the returned status is 200 we're all good and we also check the returned document to see if it contains
92
a corresponding `<a id="link">` element or `<a name="link">` element.
93

94
Specific URLs can be excluded by adding them to `tests/src/resources/ignored-links`. This is used for example for
95
`http://localhost:8080` which won't be available unless Keycloak is running on the machine where the tests are ran.
96

97
For 302 redirects there is some special handling. The returned `Location` header needs to be added to
98
`tests/src/resources/ignored-link-redirects` otherwise the link will be marked as failed. It's important to verify the
99
redirected link before you add it to this file though as in some cases it redirects to some generic moved or unavailable
100
page.
101

102
To prevent the checking of links being to slow to execute there is a cache file with valid links. This is stored in
103
`.verified-links`. The first time you run the tests you will notice that the checking links is fairly slow, but this
104
will be faster the second time you run it. Entries are purged from the cache after 1 day.
105

106
Links to other guides within the documentation are handled specially. When testing local builds the link to the guide as
107
specified within document-attributes is replaced with the link to the locally built HTML file. When testing externally
108
built and hosted guides the base part of the links are replaced with `guideBaseUrl`. This allows leaving the links
109
in document-attributes to point to the location the documentation will eventually be published to while also allowing
110
testing cross referencing between guides.