Deployment and Operations

10 mins read

Running Functions as a Service

With Cloudentity, you can run Extention scripts and evaluate authorization policies with the FaaS (Function as a Service) approach in a separate execution context to increase the system security.

FaaS Overview

FaaS, or Function-as-a-Service, is a cloud-computing service that allows customers to execute code without managing the complex infrastructure typically associated with building and launching microservices applications. Cloudentity supports ability to execute the authorization policy logics (policies written in REGO) as well as custom extensions scripts in a secure sandbox utilizing the Function as a Service (FaaS) approach. This can be achieved in two ways:

Using Fission

Cloudentity uses Fission to run functions of thousands of tenants that we have registered in our SaaS platform. If you do not expect your deployment to have a large number of tenants, use the Setting Up Self-Managed Direct FaaS Environment approach.
Setting Up Self-Managed Direct FaaS Environment

Setting up a custom FaaS environment is a recommended way of running Functions as a Service if there is not a large number of tenants running within the deployment. In comparison to setting up FaaS with Fission, creating a custom FaaS environment removes one layer of abstraction and simplifies the management and maintenance of your FaaS infrastructure.

Fission Integration for FaaS

Fission is a fast, open source serverless framework for Kubernetes with a focus on developer productivity and high performance. Fission is based on functions invoked in satelite environments when triggered.

Cloudentity deploys Fission environment pods in an isolated namespace. This namespace must be precreated. By default, acp-faas namespace is used. If you want to use private docker images like rego-env, a docker secret registry must be added to that namespace.

Prerequisites

Kubernetes cluster v1.16+
Kubernetes fission v1.15+
Helm v3.0+

Install Fission Using Helm

kubectl create namespace fission
kubectl create -k "github.com/fission/fission/crds/v1?ref=v1.16.0"
helm repo add fission-charts https://fission.github.io/fission-charts/
helm repo update
helm install --version v1.16.0 --namespace fission fission fission-charts/fission-all

Set Up Fission Workers For Cloudentity

kubectl create namespace acp-faas
kubectl create secret docker-registry docker.cloudentity.io \
  --namespace acp-faas \
  --docker-server=docker.cloudentity.io \
  --docker-username="$DOCKER_USR" \
  --docker-password="$DOCKER_PWD"

Enable Fission Integration

Configure your values.yaml file to enable Fission Integration and apply the changes:

fission:
  enabled: true
  namespace: acp-faas
  poolsize: 3
  resources:
    requests:
      cpu: 10m
      memory: 48Mi
    limits:
      cpu: 100m
      memory: 96Mi

Enabling Fission creates the following resources:

Fission environments for Node.js (where the Extension Scripts are executed) and REGO (where authorization policies are evaluated)
A network policy that allows only public egress traffic from Fission environments.

Network policy as well as common pods parameters like resources, tolerations, affinity, and more can be modified for fission pods. See values reference.

Verify Integration

Check if all fission pods are running.
```
kubectl get pods --namespace fission
```
Check if all fission pool pods are working.
```
kubectl get pods --namespace acp-faas
```
Login to the admin portal, navigate to Extensions > Scripts, create a script and execute it to check if everything works correctly.

Setting Up Self-Managed Direct FaaS Environment

Cloudentity supports running Extension scripts and evaluating REGO policies on dedicated node.js/REGO pods. In comparison to setting up FaaS with Fission, creating a custom FaaS environment removes one layer of abstraction and simplifies the management and maintenance of your FaaS infrastructure.

If your Cloudentity deployment is spread across different regions, set up custom FaaS environments for each region that you have.

It is possible to create a custom FaaS environment for one tenant or more - the rest will be using Fission, shared pods, or their own dedicated pods. For more details, see the Dedicated Node.js and REGO Pods

Prerequisites

Kubernetes cluster v1.16+

Set Up Node.js Pods For FaaS

Create a nodejs.yaml file with the following contents within your deployment directory:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    environmentName: nodejs
  name: acp-faas-nodejs
  namespace: acp-faas
spec:
  strategy:
    rollingUpdate:
      maxSurge: 50%
      maxUnavailable: 0
  selector:
    matchLabels:
      environmentName: nodejs
  template:
    metadata:
      labels:
        environmentName: nodejs
    spec:
      imagePullSecrets:
        - name: cloudentity-docker
      topologySpreadConstraints:
        - labelSelector:
            matchLabels:
              environmentName: nodejs
          maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: ScheduleAnyway
      securityContext:
        fsGroup: 65535
        runAsNonRoot: true
      containers:
      - name: nodejs-v2
        image: docker.cloudentity.io/node-env:4
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8888
          name: http-env
          protocol: TCP
        resources:
          limits:
            cpu: 1
            memory: 1Gi
          requests:
            cpu: 1
            memory: 1Gi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          privileged: false
          readOnlyRootFilesystem: true
          runAsGroup: 65535
          runAsNonRoot: true
          runAsUser: 65535
        lifecycle:
          preStop:
            exec:
              command:
              - /bin/sleep
              - "1"

Create a nodejs-service.yaml file with the following contents within your deployment directory:

apiVersion: v1
kind: Service
metadata:
  name: acp-faas-nodejs
  labels:
    environmentName: nodejs
  namespace: acp-faas
spec:
  selector:
    environmentName: nodejs
  ports:
    - name: http-env
      protocol: TCP
      port: 8888
      targetPort: http-env

Set Up REGO Pods for FaaS

Create a rego.yaml file with the following contents within your deployment directory:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    environmentName: rego
  name: acp-faas-rego
  namespace: acp-faas
spec:
  strategy:
    rollingUpdate:
      maxSurge: 50%
      maxUnavailable: 0
  selector:
    matchLabels:
      environmentName: rego
  template:
    metadata:
      labels:
        environmentName: rego
    spec:
      imagePullSecrets:
        - name: cloudentity-docker
      topologySpreadConstraints:
        - labelSelector:
            matchLabels:
              environmentName: rego
          maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: ScheduleAnyway
      securityContext:
        fsGroup: 65535
        runAsNonRoot: true
      containers:
      - name: rego-v2
        image: docker.cloudentity.io/rego-env:4
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8888
          name: http-env
          protocol: TCP
        resources:
          limits:
            cpu: 1
            memory: 1Gi
          requests:
            cpu: 1
            memory: 1Gi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          privileged: false
          readOnlyRootFilesystem: true
          runAsGroup: 65535
          runAsNonRoot: true
          runAsUser: 65535
        lifecycle:
          preStop:
            exec:
              command:
              - /bin/sleep
              - "1"

Create a rego-service.yaml file with the following contents within your deployment directory:

apiVersion: v1
kind: Service
metadata:
  name: acp-faas-rego
  labels:
    environmentName: rego
  namespace: acp-faas
spec:
  selector:
    environmentName: rego
  ports:
    - name: http-env
      protocol: TCP
      port: 8888
      targetPort: http-env

Set Up Network Policies

Create a networkpolicy.yaml file with the following contents within your deployment directory:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: only-allow-internet
spec:
  policyTypes:
  - Egress
  podSelector: {}
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 10.0.0.0/8
        - 192.168.0.0/16
        - 172.16.0.0/20

Optional: Set Up Horizontal Pod Autoscaling for Node.js and REGO Pods

Create a nodejs-hpa.yaml file with the following contents within your deployment directory:

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: acp-faas-nodejs
  labels:
    environmentName: nodejs
  namespace: acp-faas
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: acp-faas-nodejs
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 80
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Pods
        value: 1
        periodSeconds: 180
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
      - type: Pods
        value: 5
        periodSeconds: 30

Create a rego-hpa.yaml file with the following contents within your deployment directory:

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: acp-faas-rego
  labels:
    environmentName: rego
  namespace: acp-faas
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: acp-faas-rego
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 80
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Pods
        value: 1
        periodSeconds: 180
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
      - type: Pods
        value: 5
        periodSeconds: 30

Apply Changes to Deployment

Create a Kubernetes Secret in the acp-faas namespace and store your Docker credentials in it.

If you do not have the credentials that allow you to access artifacts at docker.cloudentity.io, contact Cloudentity Sales Team.

You may either use the kubectl create secret command or create a manifest file and apply it in the next step of this procedure.

Example:
```
kubectl create secret docker-registry cloudentity-docker --docker-server=docker.cloudentity.io --docker-username=$username --docker-password=$password -n acp-faas
```
Security recommendation

To increase security of secrets stored in your repository, it is recommended to encrypt your Kubernetes secrets. You can use tools like Mozilla SOPS or Bitnami Sealed Secrets to encrypt your secrets.

When the secrets are applied to your kubernetes deployment, the secrets are encrypted and visible as plain text. Anyone who is authorized to create a Pod in a namespace can read any secret in that namespace; this includes indirect access such as the ability to create a Deployment. To mitigate the risks, kubernetes recommends to:
- Enable encryption at Rest for secrets.
- Enable or configure RBAC rules that restrict reading data in secrets.
- Where appropriate, use mechanisms such as RBAC to limit which principals are allowed to create or replace secrets.
To learn more, visit kubernetes secrets documentation
Apply all the configuration files that we’ve prepared as resources to your deployment using the kubectl apply -f <file-name> -n <namespace> command.

Example:
```
kubectl apply -f nodejs.yaml -n acp-faas
```

Optionally, you can provide all files as resources in a kustomization.yaml file and apply those resources using the kubectl apply -k kustomization.yaml command.

Configure Cloudentity to Use Node.js and REGO Pods for FaaS

Configuring and deploying the platform must be the last step of setting up a custom FaaS environment in order for it to work correctly. For all available Cloudentity settings, see the configuration reference.

In your Cloudentity platform configuration file (the values.yaml file that you use to deploy Cloudentity):

Set the faas.provider setting value to docker in the config.data section of your values.yaml file.
```
config:
  data:
    faas:
      provider: docker
```
If provider is set to docker, all tenants are using shared Node.js and REGO dedicated pods.

Provide URLs for node_url and rego_url settings in the docker section:

config:
  data:
    faas:
      provider: docker
    docker:
      node_url: "http://{service-name}.{namespace}:8888"
      rego_url: "http://{service-name}.{namespace}:8888"

Tip

The service names are configured in the nodejs-service.yaml and rego-service.yaml files.

Example:

config:
  data:
    faas:
      provider: docker
    docker:
      node_url: "http://acp-faas-nodejs.acp-faas-{{tenantID}}:8888"
      rego_url: "http://acp-faas-rego.acp-faas-{{tenantID}}:8888"

Deploy Cloudentity platform.

Result

Your Extension scripts and REGO authorization policies are now executed in satelite Node.js and REGO pods shared by all Cloudentity platform tenants. If you wish some tenant’s to have dedicated Node.js and REGO pods, learn more by reading the Dedicated Node.js and REGO Pods section.

Dedicated Node.js and REGO Pods

In some cases, it may be beneficial to have tenants that have their own dedicated pods: one for Node.js and one for REGO. It can be set up regardless whether you use Fission or only dockerized FaaS environments.

Hybrid Approach: Fission + Dedicated Pods For Selected Tenants

If you have a large number of tenants for which it is possible to share a pair of Node.js and REGO pods for executing Extensions and REGO policies, you may:

Integrate Fission
Create a separate Node.js and REGO pod for each tenant that you want to have their own dedicated pods.

Repeat all the steps from subsections of the Setting Up Self-Managed Direct FaaS Environment section. Deploy pairs of Node.js and REGO pods into separate namespaces following the namespacename-tenantID namespace naming convention if you wish the tenants not to share Node.js and REGO pods and use dedicated ones.

Remember that you need to propagate your changes across all regions your tenants reside in.
Set the faas.provider setting to hybrid in the values.yaml configuration file of your deployment.
Configure the docker.node_url and docker.rego_url parameters. You have two options here:
- Desired number of tenants share the same Node.js and the same REGO pod, the rest is using Fission.
  
  [mermaid-begin]
  
  flowchart LR t1(Tenant 1) t2(Tenant 2) t3(Tenant 3) t4(Tenant 4) fis(Fission) dpod(Dedicated Node.js and REGO pods) t1-->fis t2-->fis t3-->dpod t4-->dpod
  
  In such a case, you may reuse the configuration for the docker.node_url and docker.rego_url parameters as illustrated in the Configure Cloudentity to Use Node.js and REGO Pods for FaaS section.
  
  Example:
```
config:
  data:
    faas:
      provider: hybrid
    docker:
      node_url: "http://{service-name}.{namespace}:8888"
      rego_url: "http://{service-name}.{namespace}:8888"
```
- Desired number of tenants have their own dedicated Node.js pod and REGO pod that they do not share with any other tenant, the rest of the tenants share Fission.
  
  [mermaid-begin]
  
  flowchart LR t1(Tenant 1) t2(Tenant 2) t3(Tenant 3) t4(Tenant 4) fis(Fission) dpod1(Dedicated Node.js and REGO pods pair 1) dpod2(Dedicated Node.js and REGO pods pair 2) t1-->fis t2-->fis t3-->dpod1 t4-->dpod2
  
  In such a case, the configuration for the docker.node_url and docker.rego_url parameters contain the {{tenantID}} parameter that Cloudentity uses to route traffic to pods.
  
  Do not change the {{tenantID}} variable. It is calculated automatically by the Cloudentity platform based on your tenant’s identifier.
```
config:
  data:
    faas:
      provider: hybrid
    docker:
      node_url: "http://{service-name}.{namespace}-{{tenantID}}:8888"
      rego_url: "http://{service-name}.{namespace}-{{tenantID}}:8888"
```
  Example:
```
config:
  data:
    faas:
      provider: hybrid
    docker:
      node_url: "http://acp-faas-nodejs.acp-faas-{{tenantID}}:8888"
      rego_url: "http://acp-faas-rego.acp-faas-{{tenantID}}:8888"
```
Set the dedicated_faas feature flag to true for tenants that need to use Node.js and REGO pods.

Tip

You may enable this feature flag in the Admin workspace of your deployment’s System tenant.
Deploy Cloudentity platform.

Self-Managed Direct FaaS: Dedicated Pods for All or Selected Tenants

There may be some cases in which you would like each tenant to have their own dedicated pair of Node.js and REGO pods or some tenants sharing a pair and the rest having their own pairs of pods.

Create a separate Node.js and REGO pod for each tenant that you want to have their own dedicated pods.

Repeat all the steps from subsections of the Setting Up Self-Managed Direct FaaS Environment section. Deploy pairs of Node.js and REGO pods into separate namespaces following the namespacename-tenantID namespace naming convention if you wish the tenants not to share Node.js and REGO pods and use dedicated ones.

Remember that you need to propagate your changes across all regions your tenants reside in.
Set the faas.provider setting to docker in the values.yaml configuration file of your deployment.
Configure the docker.node_url and docker.rego_url parameters. You have two options here:
- Desired number of tenants have their own pair of Node.js and REGO pods, the rest is using a shared pair of Node.js and REGO pods.
  
  [mermaid-begin]
  
  flowchart LR t1(Tenant 1) t2(Tenant 2) t3(Tenant 3) t4(Tenant 4) proxy(Reverse Proxy/Gateway) spod(Shared Node.js and REGO pods) dpod1(Dedicated Node.js and REGO pods pair 1) dpod2(Dedicated Node.js and REGO pods pair 2) t1-->proxy t2-->proxy t3-->proxy t4-->proxy proxy-->spod proxy-->dpod1 proxy-->dpod2
  
  In such a case, the configuration for the docker.node_url and docker.rego_url parameters contains the {{tenantID}} parameter that Cloudentity uses to route traffic to pods.
  
  Do not change the {{tenantID}} variable. It is calculated automatically by the Cloudentity platform based on your tenant’s identifier.
  
  Example:
```
config:
  data:
    faas:
      provider: docker
    docker:
      node_url: "https://proxy-nodejs/{{tenantID}}/"
      rego_url: "https://proxy-rego/{{tenantID}}/"
```
  You need to set up a gateway or reverse proxy that will direct traffic based on the tenant identifier that Cloudentity populates in the node_url and rego_url automatically when triggering scripts/policies. Then, distribute traffic as needed - some tenant’s may be directed to the shared pair of Node.js and REGO pods, the rest can be redirected to their own dedicated pair of pods.
- Each tenant have their own dedicated Node.js pod and REGO pod that they do not share with any other tenant.
  
  [mermaid-begin]
  
  flowchart LR t1(Tenant 1) t2(Tenant 2) t3(Tenant 3) t4(Tenant 4) dpod1(Dedicated Node.js and REGO pods pair 1) dpod2(Dedicated Node.js and REGO pods pair 2) dpod3(Dedicated Node.js and REGO pods pair 3) dpod4(Dedicated Node.js and REGO pods pair 4) t1-->dpod1 t2-->dpod2 t3-->dpod3 t4-->dpod4
  
  In such a case, the configuration for the docker.node_url and docker.rego_url parameters contains the {{tenantID}} parameter that Cloudentity uses to route traffic to pods.
  
  Do not change the {{tenantID}} variable. It is calculated automatically by the Cloudentity platform based on your tenant’s identifier.
```
config:
  data:
    faas:
      provider: docker
    docker:
      node_url: "http://{service-name}.{namespace}-{{tenantID}}:8888"
      rego_url: "http://{service-name}.{namespace}-{{tenantID}}:8888"
```
  Example:
```
config:
  data:
    faas:
      provider: docker
    docker:
      node_url: "http://acp-faas-nodejs.acp-faas-{{tenantID}}:8888"
      rego_url: "http://acp-faas-rego.acp-faas-{{tenantID}}:8888"
```
Deploy Cloudentity platform.
If you chose the solution where a Desired number of tenants have their own pair of Node.js and REGO pods, the rest is using a shared pair of Node.js and REGO pods, enable the dedicated_faas feature flag for tenant’s that need their own dedicated Node.js and REGO pods pair.

Tip

You may enable this feature flag in the Admin workspace of your deployment’s System tenant.

Updated: Oct 27, 2023

Deployment and Operations

Running Functions as a Service

FaaS Overview

Fission Integration for FaaS

Prerequisites

Install Fission Using Helm

Set Up Fission Workers For Cloudentity

Enable Fission Integration

Verify Integration

Setting Up Self-Managed Direct FaaS Environment

Prerequisites

Set Up Node.js Pods For FaaS

Set Up REGO Pods for FaaS

Set Up Network Policies

Optional: Set Up Horizontal Pod Autoscaling for Node.js and REGO Pods

Apply Changes to Deployment

Security recommendation

Configure Cloudentity to Use Node.js and REGO Pods for FaaS

Tip

Result

Dedicated Node.js and REGO Pods

Hybrid Approach: Fission + Dedicated Pods For Selected Tenants

Tip

Self-Managed Direct FaaS: Dedicated Pods for All or Selected Tenants

Tip