How-tos

7 mins read

Migrating Users to Cloudentity Identity Pools

Learn how to migrate users from external CIAM platforms into Cloudentity Identity Pools using the dedicated APIs. Change your CIAM platform to persistently store users even at hyper scale!

About Migrating Users

Cloudentity allows you import users into the tenant’s Identity Pool. If the CIAM platform you used so far does not meet your needs or requirements, you can migrate external users to Cloudentity Identity Pools for persistent storage even at hyper-scale.

To migrate users, you will use the Cloudentity Identity Tenant Import API.

It is also possible to migrate users between Cloudentity tenants and between different Identity Pools. To do that, you would first use Cloudentity Identity Tenant Export API and, then, the Identity Tenant Import API. At this moment, however, exporting users is not recommended for production environments.

Prerequisites

You must have an Identity Pool to import/export the user data.
You have access to the System workspace.
You have a client application in the System workspace with the manage_configuration scope assigned. You will need this client to obtain an access token from Cloudentity for user import purposes.

Note

Preconfigure your workspace describes how to create the client application that you need, but for the Admin workspace and admin APIs. What you need to do, is to create similar application in the System workspace and assign the manage_configuration scope to it.

For import purposes, you must prepare input data conforming to the Import API schema. Prepare import batches of no more than 100 users (with credentials, identifiers, and addresses) per batch. Sending multiple requests is allowed - with 8 parallel requests running, the expected performance is around 1 500 000 users in half an hour.

The key item to prepare in order to achieve API idempotency is the random, unique ID (we strongly recommend using version 4 UUID). You will need to generate 4 sets of such IDs, since user data is kept in 4 separate objects, responsible for users, user credentials, user identifiers, and user addresses. This way, if an import request fails, you can simply run it again and be sure that no record gets added twice. Please check the Import API documentation and check the Request Body Schema for details.

Note that the user ID is especially important, since it must be mapped in all four user-related objects in the payload to correctly bind the entire user record (user.id to user_credentials.user_id, to user_identifiers.user_id and so on):

users - responsible for the core user information. You need to prepare id parameter for each record.
user_credentials - data with user credentials. You need to prepare id parameter for the record itself and map the user via user_id.

If you want to migrate users without passwords, make sure to add the expires_at parameter with a value set to a past date. As a result, after migration, user login attempts with OAuth2 Resource Owner Password flow fail with the credential expired hint:
```
{"error_description":"request lacks valid authentication credentials for the target resource","error_hint":"credential expired","status_code":401}
```
Similar outcome (with "error_description": "credential expired") is expected for attempts to change password or verify password.

Upon facing this error, users must request an OTP for password reset (forgot? in the login UI) and confirm resetting their password.

When expires_at is not set or set to 1900-01-01T00:00:00Z, the password never expires.
user_identifiers - data with user identifiers (e-mail or phone number used to log in). You need to prepare id parameter for the record itself and map the user via user_id.
user_verifiable_addresses - data with user’s addresses (e-mail and/or mobile). You need to prepare id parameter for the record itself and map the user via user_id.

Below you can find a sample payload conforming to the above schema with two users (including credentials, identifiers, and addresses):

{
"user_credentials": [
   {
         "created_at": "2022-08-03T11:03:38.34+02:00",
         "expires_at": "2019-08-24T14:15:22Z",
         "id": "e89e254c-f538-4ae2-9150-4d4580bcf886",
         "payload": {
            "hashed_password": {
               "config": {
                     "method": "sha",
                     "sha": {
                        "function": "SHA-256",
                        "salt": "lJgayFHwYelZGmrBnYqt",
                        "salt_length": 20
                     }
               },
               "value": "eUJBxl+dwVjPgwC2cm1K+hYNWFRly/RdCT/bgmIBowo="
            }
         },
         "tenant_id": "default",
         "type": "password",
         "updated_at": "2022-08-03T11:03:38.34+02:00",
         "user_id": "58e6184f-8e38-4193-b889-965b34326c7f",
         "user_pool_id": "caku6lrphdd3cfqro3mg"
   },
   {
         "created_at": "2022-08-03T11:03:38.343+02:00",
         "id": "20f45596-f407-495c-aa45-043609ae2280",
         "payload": {
            "hashed_password": {
               "config": {
                     "method": "sha",
                     "sha": {
                        "function": "SHA-256",
                        "salt": "lJgayFHwYelZGmrBnYqt",
                        "salt_length": 20
                     }
               },
               "value": "eUJBxl+dwVjPgwC2cm1K+hYNWFRly/RdCT/bgmIBowo="
            }
         },
         "tenant_id": "default",
         "type": "password",
         "updated_at": "2022-08-03T11:03:38.343+02:00",
         "user_id": "cdcb7a5b-a726-404c-ba66-b8552d9be8fc",
         "user_pool_id": "caku6lrphdd3cfqro3mg"
   }
],
"user_identifiers": [
   {
         "created_at": "2022-08-03T11:03:38.341+02:00",
         "id": "b7108442-7f1e-40b9-a4d9-0c9ab3b0d18b",
         "identifier": "user0@example.com",
         "tenant_id": "default",
         "type": "email",
         "updated_at": "2022-08-03T11:03:38.341+02:00",
         "user_id": "58e6184f-8e38-4193-b889-965b34326c7f",
         "user_pool_id": "caku6lrphdd3cfqro3mg"
   },
   {
         "created_at": "2022-08-03T11:03:38.343+02:00",
         "id": "cb1b6524-a769-442a-8b80-fe62ed9ed614",
         "identifier": "user1@example.com",
         "tenant_id": "default",
         "type": "email",
         "updated_at": "2022-08-03T11:03:38.343+02:00",
         "user_id": "cdcb7a5b-a726-404c-ba66-b8552d9be8fc",
         "user_pool_id": "caku6lrphdd3cfqro3mg"
   }
],
"user_verifiable_addresses": [
   {
         "address": "user0@example.com",
         "created_at": "2022-08-03T11:03:38.342+02:00",
         "id": "0b24d104-3d0e-40d7-bd2d-ac1e74b8fb34",
         "status": "active",
         "tenant_id": "default",
         "type": "email",
         "updated_at": "2022-08-03T11:03:38.342+02:00",
         "user_id": "58e6184f-8e38-4193-b889-965b34326c7f",
         "user_pool_id": "caku6lrphdd3cfqro3mg",
         "verified": true
   },
   {
         "address": "user1@example.com",
         "created_at": "2022-08-03T11:03:38.343+02:00",
         "id": "f0eedf44-f7d8-4401-9a0d-4a02d3ed92f4",
         "status": "active",
         "tenant_id": "default",
         "type": "email",
         "updated_at": "2022-08-03T11:03:38.343+02:00",
         "user_id": "cdcb7a5b-a726-404c-ba66-b8552d9be8fc",
         "user_pool_id": "caku6lrphdd3cfqro3mg",
         "verified": true
   }
],
"users": [
   {
         "created_at": "2022-08-03T11:03:38.33+02:00",
         "id": "58e6184f-8e38-4193-b889-965b34326c7f",
         "metadata": {
            "groups": [
               "admins",
               "users"
            ]
         },
         "metadata_schema_id": "default_metadata",
         "payload": {
            "name": "Keshia Mraz",
            "given_name": "Keshia",
            "family_name": "Mraz"
         },
         "payload_schema_id": "default_payload",
         "status": "active",
         "status_updated_at": "2022-08-03T11:03:38.33+02:00",
         "tenant_id": "default",
         "updated_at": "2022-08-03T11:03:38.33+02:00",
         "user_pool_id": "caku6lrphdd3cfqro3mg"
   },
   {
         "created_at": "2022-08-03T11:03:38.343+02:00",
         "id": "cdcb7a5b-a726-404c-ba66-b8552d9be8fc",
         "metadata": {
            "groups": [
               "users"
            ]
         },
         "metadata_schema_id": "default_metadata",
         "payload": {
            "name": "Vicente Moen",
            "given_name": "Vicente",
            "family_name": "Moen"
         },
         "payload_schema_id": "default_payload",
         "status": "active",
         "status_updated_at": "2022-08-03T11:03:38.343+02:00",
         "tenant_id": "default",
         "updated_at": "2022-08-03T11:03:38.343+02:00",
         "user_pool_id": "caku6lrphdd3cfqro3mg"
   }
]
}

Importing/Exporting Tenant’s User Data

With Cloudentity system API, you can export and import the users from a specific tenant.

Import Users

Call the OAuth 2.0 token endpoint to get an access token.
```
curl --location --request POST 'https://example.com/tenant_id/system/oauth2/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'client_id=value' \
--data-urlencode 'client_secret=secret'
```
Result

Your token is returned. Since you called the system endpoint as a client requesting the manage_configuration scope, the token should grant this scope.

Learn More

If you need help with getting the token, see the Getting started with Cloudentity REST API guide, section Call the token endpoint.
Call the Import Tenant Configuration endpoint for Identity. Pass user data in the request body.
Request Example
```
curl --location --request PUT 'https://example.com/api/identity/system/{tid}/configuration' \
-H 'Authorization: Bearer {ACCESS_TOKEN}' \
-d @configuration.json
```
The {tid} path parameter is your tenant identifier.

Replace the {ACCESS_TOKEN} variable with the access token that you got in the first step.

You can either put the payload directly in the request body or point to the file that contains the configuration using the -d @YourFileName.json parameter.
Result

204 NO CONTENT

Your tenant configuration is imported and available within Cloudentity.

Export Users

Testing purposes only

The Export API is not ready to be used in production environments with a large number of users. Please treat it as a testing/beta functionality.

Call the OAuth 2.0 token endpoint as the System client to get an access token with the manage_configuration scope.
```
curl --location --request POST 'https://example.com/tenant_id/system/oauth2/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'client_id=value' \
--data-urlencode 'client_secret=secret'
```
Result

Your token is returned. Since you called the system endpoint as a client requesting the manage_configuration scope, the token should grant this scope.

Learn More

If you need help with getting the token, see the Getting started with Cloudentity REST API guide, section Call the token endpoint.
Call the Export Tenant Configuration endpoint for Identity
Request Example
```
curl --location --request GET 'https://example.com/api/identity/system/{tid}/configuration' \
--header 'Authorization: Bearer {ACCESS_TOKEN}'
```
The {tid} path parameter is your tenant identifier. {ACCESS_TOKEN} is the token returned in the first step.
Result

200 OK

Your tenant configuration, including Identity Pools with their users, is returned as JSON in the request response body.

Insert Mode

The import endpoint has the optional mode query parameter available. This parameter defines what happens if there are any conflicts when importing your user records. For example, if a user already exists within Cloudentity and you are trying to import a configuration that already has a user with this ID, there are the following ways Cloudentity can handle the request:

mode set to ignore (default) - Cloudentity ignores the changes that come from your configuration import.
mode set to fail - Cloudentity stops import processing and returns an error.
mode set to update - Cloudentity updates the tenant’s configuration with the value provided in the request.

Updated: Aug 10, 2023

How-tos

Migrating Users to Cloudentity Identity Pools

About Migrating Users

Prerequisites

Note

Importing/Exporting Tenant’s User Data

Import Users

Result

Learn More

Request Example

Result

Export Users

Testing purposes only

Result

Learn More

Request Example

Result

Insert Mode