OpenID self-certification

Test Automation for Maintaining OpenID Self-certification

This article describes how test automation is applied at Cloudentity to maintain OpenID self-certification.

What is OpenID Self-certification

OpenID self-certification is a formal declaration by a company that their products or services conform to the requirement profiles set by the OpenID foundation. Companies consuming such product or service may be looking for a proof that the product implements all the requirements correctly, and OpenID certification can help provide such an assurance. OpenID Foundation decided on self-certification, as in their opinion is easier, quicker, and cheaper than third-party certification — but it doesn’t mean that anyone can just stamp the certificate on their product. Self-certification is achieved by successfully testing a product or service with the OpenID Foundation Conformance Suite test application.

Once acquired, OpenID certifications do not expire, however, the company that wants to self-certify is responsible for maintaining officially certified profiles in the future versions of their product or service.

OpenID Implementation in Cloudentity

As of today, Cloudentity supports 39 OIDC profiles, including most Financial API profiles, and all FAPI-CIBA profiles, with more coming soon. To ensure the highest possible quality, we want to test all possible combinations that we are supporting. Currently, that amounts to exactly 3390 test cases. OpenID Foundation Conformance Suite test application is helping with that process, but by default the process still needs some manual input, for example to set the configuration, and interact with our application. This is why, right from the beginning of our self-certification process, we decided to go with test automation. As a result, we can run all of the certification checks on every commit and every change made in our application. Consequently, we can be sure that none of them are broken, and that we are conformant with all the profiles that we certified with in the past at all times. Cloudentity takes its quality commitments very seriously.

OpenID Automation at Scale

Test automation line

At first, we have implemented Conformance Suite test automation using Java and Selenium, and we have plugged the process into our continuous integration pipeline. We were basically mimicking the actions of a real user consuming the OpenID Foundation Conformance Suite test application, and setting up our application with REST APIs. This worked fine for a long time; however, as the amount of supported conformance profiles grew, so did the amount of test cases, which meant that the pipeline took longer and longer to run, and it started to be on the critical path. As an example, when we had about 2000 tests, the pipeline used to take about 25 minutes, and we also had problems with the stability of this solution.

We have discovered, however, that OpenID test tool also provides us with a REST API that we could have used to make it run a lot quicker by skipping the use of Selenium. So, this is what we did, and now all the communication with the OpenID tool, is done with APIs only, without the use of Selenium, saving us a lot of time by skipping the frontend altogether. This helped immensely, as it cut down the testing time required back to 6 minutes, and now it takes just 8 minutes to run all of the 3200 tests, and that is an acceptable time for our CI pipeline. However, we still decided to keep the compatibility with the old way of running the tests, since the actual certification process requires making screenshots of the application in specific conditions, so we still have an option to run the tests with Selenium.

Test Flow Handling

Since every test requires a set of different actions, where particular actions are similar between tests, instead of implementing each one as a separate test method, we decided to create a test runner based on a state machine. Here is a diagram of how the states are connected with each other:

flowchart LR CreateState --> WaitingForActionState WaitingForActionState --> ResultState WaitingForActionState --> RotateKeyState RotateKeyState --> StartState StartState --> WaitingForActionState WaitingForActionState <--> IgnoreAuthenticationState WaitingForActionState <--> CibaState WaitingForActionState --> VisitUrlState CallbackState --> WaitingForActionState VisitUrlState --> AuthorizationState AuthorizationState <--> LoginState AuthorizationState <--> ConsentState AuthorizationState <--> NotNeededConsentState AuthorizationState <--> CustomConsentState AuthorizationState --> WaitingForActionState AuthorizationState <--> ScreenshotState AuthorizationState <--> FormPostState AuthorizationState --> CallbackState

Every test begins with CreateState, and ends with ResultState, even if it fails. The test runner communicates with both the conformance test tool and our application and reacts accordingly to what needs to be done at the current moment. This allows us to use just one test method to handle all 3206 test cases, and to easily add new ones, if we want to support new profiles in the future.


Cloudentity continuously supports OpenID profiles at scale. The test framework presented above allows for adding new profiles and making sure they are always covered. If you need an OpenID-compliant CIAM platform, look no further.

Like what you see? Register for free to get access to a Cloudentity tenant and start exploring our platform!

Updated: Jan 23, 2023