OpenID Foundation Certification Process in Cloudentity
In the blog post Test Automation for Maintaining OpenID Self-certification, we described our automation process for the OIDC re-certification tests. This article describes how test automation utilizing Java and Selenium helps us with the actual OpenID Foundation certification process.
Generate API Token From Official Certification Tool
First, we go to official certification test tool instance and generate an API token. In our CI/CD testing process, instead of using the official certification test tool instance, we are using a local Docker instance hosted on the same machine as Cloudentity. This ensures that our tests do not fail due to random connectivity problems or server unavailability. But, of course, in the process of official certification, the tests results must be prepared using the official test tool instance. The API token is needed in order to get our test automation working. Right now we have almost 4000 test cases, so if we would like to re-certify them all it would take several days to perform all of them manually.
Prepare Cloudentity Certification Environment
As the second step, we need to prepare a public facing certification environment, so the official certification test tool instance is able to connect to it. Due to our test automation setup, the Cloudentity configuration, such as specific servers or clients, are all created before running the tests, and then removed to keep the test environment clean. This way, we do not need to manually create a specific configuration for every test plan we want to certify.
Set Tests to Selenium Mode
Now we need to run our tests in Selenium mode. As described in the previous blog post, we are normally running our tests locally using only REST API to interact with the OpenID tool. This approach, however, is not sufficient on its own, as several of the tests require presenting screenshots. This is the reason why we still kept the possibility of running our tests in Selenium mode. The slower speed is not a problem in this scenario, as ideally tests in this mode are ran just once per certification, and not with every commit.
Run Tests and Verify The Results
Finally, we run the tests and verify the results. As mentioned in previous point, the screenshots are reviewed manually by OpenID foundation when they are submitted, so we make sure to also review them beforehand, in order to avoid wasting the foundation’s time if our tests need any fixing. This process could perhaps be automated if we were to compare screenshots with some previous versions, but since we are not using this mode in our pipeline due to time constraints, it is better to just review them manually.
Prepare Certification Packages
At this stage, we sign the needed documents, and prepare the certification packages. Every test plan has a separate certification package that consists of tests results, screenshots, and a signed declaration of conformance detailing the name, version, and other information about the product. This process requires some manual work yet again, as every document must be signed. Fortunately, the certification test tool helps with this process by providing a convenient function that creates a ready zip package with all the needed files inside.
Send Certification Packages
The last part of the process is sending the certification packages to the OIDC certification help desk. At this stage, we also make the necessary payments. If everything goes right, then, usually in a few days, the OpenID foundation posts the results on the certification page. In case something goes wrong, the problems are analyzed, Cloudentity or our test automation are fixed, the tests get performed again, and sent once more for review.
Conclusion
As you can see, thanks to the Selenium mode, our test automation process not only helps us keep our existing certification compatibility but also facilitates getting new certifications.
Like what you see? Register for free to get access to a Cloudentity tenant and start exploring our platform!