Private Edition

Introduction

Nirmata is designed using cloud-native principles and packaged as a set of containerized microservices. Nirmata is available in several different deployment models:

The following sections provide requirements for the Nirmata Private Edition.

The Nirmata Private Edition (Nirmata PE) solution comprises of two architectural components (service sets):

Each service set contains multiple containerized microservices.

image

Installation Requirements

A minimum Nirmata PE installation consists of a minimum of 2 VMs (6 VMs for HA configuration), one for each instance of the architectural component described above. Each VM will run multiple containerized microservices.

For production deployments, each component can be independently sized and scaled for availability and performance.

Compute

The table below provides node counts and specifications for the Nirmata components:

Component Counts Specifications
Core Services - PoCs: 1 - 16 GB RAM
- Production (HA): 3 - 4 vCPUs
- 256 GB SSD
Shared Services - PoCs: 1 - 16 GB RAM
- Production (HA): 3 - 4 vCPUs
- 256 GB SSD
Network
Ports

The table below details the networking requirements for Nirmata PE:

Component Connections
Core Services Inbound Connections
- HTTPS (443) from the LB / users
- HTTPS (443) from Nirmata agent on Container Hosts
- SSH (22) for secure shell access
Outbound Connections
- HTTPS (443) to image registry
Shared Services Inbound Connections
- TCP (27017) MongoDB from Nirmata Core Services
- TCP (9092) Kafka from Nirmata Core Services
- TCP (2181) ZooKeeper from Nirmata Core Services
- TCP (2888) ZooKeeper from Nirmata Shared Services
- TCP (3888) ZooKeeper from Nirmata Shared Services
- TCP (9200) ElasticSearch from Nirmata Core Services
- TCP (9300) ElasticSearch from Nirmata Shared Services
- SSH (22) for secure shell access
Outbound Connections

Additionally, a well known address will need to be configured for Nirmata (e.g. nirmata.company-name.com). This address will resolve to the load balancer IP address (VIP) or Nirmata server IP address, if load balancer is not used.

Note 1: A load-balancer is not required for the PoC deployment.

Note 2: It is assumed that Container Hosts in the private data-center (e.g. Diamanti VMs) and Container Hosts in public cloud (e.g. Azure) have direct L3 connectivity to the Nirmata Core Services.

Network Proxy

In case you are required to use network proxy for any external communication, please keep the proxy settings available so that they can be used during the installation.

Configure proxy for Docker Engine by adding the following to /etc/systemd/system/docker.service.d/http-proxy.conf:

[Service]
Environment="HTTP_PROXY=<proxy-address>"
Environment="HTTPS_PROXY=<proxy-address>"
Environment="NO_PROXY=127.0.0.1,localhost,<nirmata-services-host-ip,nirmata-shared-services-host-ip>

You can find more details on configuring HTTP/HTTPS proxy for Docker Engine at https://docs.docker.com/config/daemon/systemd/#httphttps-proxy.

DNS

It is required that host names for all hosts resolve via DNS.

image

Storage

There are no additional storage requirements. Nirmata shared services use the storage on the host instances.

Security

Here are some additional security related considerations:

X.509 Certificates

All Nirmata services use SSL for secure communications. The services use self-signed certificates. A CA signed certificate can be installed, if available.

For details on generating a self-signed certificate, see Step 5 - Install an x509 Certificate.

Single Sign On with SAML

Nirmata supports Single Sign On (SSO) using SAML 2.0. If SAML SSO is required, Nirmata will need connectivity to the SAML IDP service, for example ADFS 3.0 (Active Directory Federation Services).

HA Deployment

For production environment, Nirmata should be deployed in HA configuration. For HA setup, 6 VMs (nodes) are required – 3 for shared services and 3 for core services.

Note: Some distributed services, like Zookeeper, require an odd number of nodes to form a quorum and hence a 2-node configuration is not supported.

To ensure resiliency against cloud/datacenter outage, the nodes should be deployed in at least 2 different availability zones (preferably 3 different availability zones). Every node needs access to every other node via a routable IP address or domain name (L3).

The diagram below shows the details of the setup:

image

Installation Overview

There are four main components involved in the installation of Nirmata Private Edition:

1.  Docker Hub Registry
2.  Private Docker Registry
3.  Nirmata Core Services Host
4.  Nirmata Shared Services Host

image

At least two nodes are required to run Nirmata Private Edition:

1.  Nirmata Core Services Node
2.  Nirmata Shared Services Node

Installation on Virtual or Physical Servers

Step 1 – Push Nirmata images to your Private Docker Registry

Note: If your machine has access to Docker Hub (https://hub.docker.com) this step is not required and you can proceed to Step 2 - Configure a Node to Run Nirmata Shared Services.

The first step involves pushing Nirmata container images into your private Docker registry. Nirmata container images are available on Docker Hub. These images will be downloaded and pushed into a repository called “nirmata”.

  1. Copy the script nirmata-pe.sh you received from Nirmata on a Linux host which as a network connectivity with both your Docker private registry and Docker Hub.
  2. Edit the script nirmata-pe.sh and update the following fields with correct values:

    a. DOCKER_REGISTRY: IP address or DNS name of your Docker private registry b. DOCKER_USERNAME: Username used to login to the Docker private registry c. DOCKER_PASSWORD: Password used to login to the Docker private registry

    (If a Network Proxy is required)

    d. HTTP_PROXY: HTTP network proxy URL with username and password if needed e. HTTPS_PROXY: HTTPS network proxy URL with username and password if needed f. NO_PROXY: network addresses, names, or CIDRs for which the proxy is not used

  3. Execute the installer command :

    ./nirmata-pe.sh update –-registry
    

This command is going to pull from Docker Hub all the Nirmata images and push them into your Docker private registry. The list of images is as follows:

$TARGET_REGISTRY/nirmata/nirmata-pe-installer:pe_v1               
$TARGET_REGISTRY/nirmata/users:pe_v1               
$TARGET_REGISTRY/nirmata/webclient:pe_v1               
$TARGET_REGISTRY/nirmata/config:pe_v1               
$TARGET_REGISTRY/nirmata/cluster:pe_v1               
$TARGET_REGISTRY/nirmata/securit:pe_v1               
$TARGET_REGISTRY/nirmata/orchestrator:pe_v1               
$TARGET_REGISTRY/nirmata/cloudprovider:pe_v1               
$TARGET_REGISTRY/nirmata/imageregistry:pe_v1               
$TARGET_REGISTRY/nirmata/nginx-private-edition:pe_v1               
$TARGET_REGISTRY/nirmata/host-gateway:pe_v1               
$TARGET_REGISTRY/nirmata/nirmata-tunnel-server:pe_v1               
$TARGET_REGISTRY/nirmata/analytics:pe_v1               
$TARGET_REGISTRY/nirmata/clientgateway:pe_v1               
$TARGET_REGISTRY/nirmata/gateway:pe_v1               
$TARGET_REGISTRY/nirmata/registry:pe_v1   
$TARGET_REGISTRY/nirmata/kibana418:latest
$TARGET_REGISTRY/nirmata/kafka-es-connector:1.7.3
$TARGET_REGISTRY/mongodb-single:latest
$TARGET_REGISTRY/nirmata/kafka-single:latest
$TARGET_REGISTRY/nirmata/zookeeper-single:latest  
$TARGET_REGISTRY/nirmata/elasticsearch-single:latest  
Step 2 – Configure a Node to Run Nirmata Shared Services

This first Node will run Nirmata shared services - MongoDB, Zookeeper, Elasticsearch, Kafka and Kibana.

  1. SSH to your Node
  2. Disable RHEL7 the firewall (if applicable) :

    sudo systemctl stop firewalld
    sudo systemctl disable firewalld
    sudo systemctl status firewalld
    
  3. Install Docker (for RHEL see Annex A – Docker CE Installation on RHEL 7)

  4. Copy Nirmata Private Edition installation script (Nirmata-pe.sh) under /usr/bin :

    sudo cp nirmata-pe.sh /usr/bin/nirmata-pe
    sudo chmod a+x /usr/bin/nirmata-pe
    
  5. Make sure the following ports are accessible from outside the Node:

    a. Elasticsearch: 9200, 9300 b. Kafka: 9092 c. Zookeeper: 2181, 2888, 3888 d. MongoDB: 27017

Step 3 - Deploy Shared Services

1) Edit the script /usr/bin/nirmata-pe.sh to update the following fields with correct values:

a. DOCKER_REGISTRY: IP address or DNS name of your Docker private registry

b. DOCKER_USERNAME: Username used to login to the Docker private registry

c. DOCKER_PASSWORD: Password used to login to the Docker private registry

d. PRIVATE_IP: Private IP address of your Node

e. BACKEND_IP: Private IP of your Node (must be same as PRIVATE_IP)

(If a Network Proxy is required)

f. HTTP_PROXY: HTTP network proxy URL with username and password if needed.

g. HTTPS_PROXY: HTTPS network proxy URL with username and password if needed.

h. NO_PROXY: network addresses, names, or CIDRs for which the proxy is not used.

2) Execute the installer command: :

     ./nirmata-pe.sh install –-backend

3) Verify that all the backend services are up and running :

     sudo docker ps

At the end of the execution the command will display the list of container running on your Node. You should see an output similar to this:

IMAGE                                                  CREATED              STATUS              
$TARGET_REGISTRY/nirmata/kibana418:latest              About a minute ago   Up About a minute
$TARGET_REGISTRY/nirmata/kafka-es-connector:1.7.3      About a minute ago   Up About a minute
$TARGET_REGISTRY/nirmata/elasticsearch-single:latest   About a minute ago   Up About a minute
$TARGET_REGISTRY/nirmata/mongodb-single:latest         About a minute ago   Up About a minute
$TARGET_REGISTRY/nirmata/kafka-single:latest           About a minute ago   Up About a minute
$TARGET_REGISTRY/nirmata/zookeeper-single:latest       About a minute ago   Up About a minute

In case something goes wrong or if you need to change some of the configuration parameters, you can remove all the services and re-do the installation. To remove all the Nirmata shared services you can execute:

./nirmata-pe.sh remove –-backend
Step 4 - Configure a Node to Run Nirmata Services

This second Node will run the Nirmata microservices and nginx.

  1. SSH to your Node
  2. Disable RHEL7 the firewall :

    sudo systemctl disable firewalld
    
  3. Install Docker

  4. Copy Nirmata Private Edition installation script (Nirmata-pe.sh) under /usr/bin: :

    sudo cp nirmata-pe.sh /usr/bin/nirmata-pe
    sudo chmod a+x /usr/bin/nirmata-pe
    
  5. Make sure that the ports 443 and 80 are open

Step 5 - Install an x509 Certificate

In order to run nginx, you need to provide a certificate valid for the URL use want to use to access Nirmata. If you don’t already have a certificate, you can use a self-signed certificate. Here are the steps required to generate a self-sign certificate:

  1. SSH to the Node you will use to run the Nirmata services
  2. Create a directory where the certificate will be generated :

    sudo mkdir certificates
    
  3. Generate the certificate in this directory :

    sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout certificates/nginx.key -out certificates/nginx.crt
    
  4. Enter the required parameters

    a. Country Name (2 letter code) [AU]:US

    b. State or Province Name (full name) [Some-State]:New York

    c. Locality Name (eg, city) []:New York City

    d. Organization Name (eg, company) [Internet Widgits Pty Ltd]:Acme

    e. Organizational Unit Name (eg, section) []:I.T

    f. Common Name (e.g. server FQDN or YOUR name) []:server_IP_address or DNS

    g. Email Address []:admin@your_domain.com

Step 6 - Deploy Nirmata Services
  1. SSH to your Nirmata Services Node
  2. Edit the script nirmata-pe.sh to update the following fields with correct values:

    a. DOCKER_REGISTRY: IP address or DNS name of your Docker private registry

    b. DOCKER_USERNAME: Username used to login to the Docker private registry

    c. DOCKER_PASSWORD: Password used to login to the Docker privateregistry

    d. PRIVATE_IP: Private IP address of your Node

    e. BACKEND_IP: Private IP of your Node (must be same as PRIVATE_IP)

    f. DATA_DIR: Directory that will contain all the logs and data

    g. BACKUP_DIR: Directory where database backups must be stored

    h. NIRMATA_URL: URL that will be used to connected to Nirmata

    i. CERTIFICATE: Certificates associated to NIRMATA_URL

    j. KEY: PEM key associated to NIRMATA_URL

    k. ADMIN_NAME: Name of the Nirmata super admin user

    l. ADMIN_PASSWORD: Password of the Nirmata super admin user

    m. ADMIN_EMAIL: Email of the super admin user

    n. TENANT_NAME: Name of the first tenant. You can use your company name or team name for instance.

    o. TENANT_OWNER_EMAIL: Email of the user responsible for this tenant

    p. USER_NAME: Name of the first Nirmata user

    q. USER_PASSWORD: Password of the first Nirmata user

    r. USER_EMAIL: Email of the first Nirmata user

  3. Execute the installer command: :

    ./nirmata-pe.sh install –-nirmata
    
  4. Verify that all the backend services are up and running

At the end of the execution, the command will display the list of containers running on your Node. You should see an output similar to this:

IMAGE                                                   CREATED              STATUS             
$TARGET_REGISTRY/nirmata/nginx-private-edition:latest   30 seconds ago       Up 23 seconds
$TARGET_REGISTRY/nirmata/gateway:pe_v1                  35 seconds ago       Up 31 seconds
$TARGET_REGISTRY/nirmata/host-gateway:pe_v1             43 seconds ago       Up 42 seconds
$TARGET_REGISTRY/nirmata/orchestrator:pe_v1             50 seconds ago       Up 49 seconds
$TARGET_REGISTRY/nirmata/webclient:pe_v1                About a minute ago   Up 57 seconds
$TARGET_REGISTRY/nirmata/cluster:pe_v1                  About a minute ago   Up About a minute
$TARGET_REGISTRY/nirmata/nirmata-tunnel-server:pe_v1    About a minute ago   Up About a minute
$TARGET_REGISTRY/nirmata/clientgateway:pe_v1            About a minute ago   Up About a minute
$TARGET_REGISTRY/nirmata/cloudprovider:pe_v1            About a minute ago   Up About a minute
$TARGET_REGISTRY/nirmata/analytics:pe_v1                About a minute ago   Up About a minute
$TARGET_REGISTRY/nirmata/imageregistry:pe_v1            About a minute ago   Up About a minute
$TARGET_REGISTRY/nirmata/config:pe_v1                   2 minutes ago        Up 2 minutes
$TARGET_REGISTRY/nirmata/security:pe_v1                 2 minutes ago        Up 2 minutes
$TARGET_REGISTRY/nirmata/users:pe_v1                    2 minutes ago        Up 2 minutes
$TARGET_REGISTRY/nirmata/registry:pe_v1                 2 minutes ago        Up 2 minutes

In case something goes wrong or if you need to change some of the configuration parameters you can remove all the services and re-do the installation. To remove and re-install all the Nirmata services you can execute:

./nirmata-pe.sh remove –-nirmata
./nirmata-pe.sh install –-nirmata
Step 7 - Logon to Nirmata

You can login to Nirmata using the URL, the email and password you have specified in the installation script (NIRMATA_URL, USER_EMAIL, USER_PASSWORD).

Step 8 – Add the license

Once you log in to Nirmata, go to Account->Settings and update the license key. Once a valid license key is added, you will no longer see the ‘Trial’ message.

Installation on Kubernetes Cluster or Appliance

Step 1 – Download the install package and other dependencies

Download the install package on a node, which has access to the Kubernetes cluster and the Docker registry (public or private). Un-archive the contents of the package:

tar –xvf nirmata-pe-k8s-installer.tgz

You will also need to download and install the following:

Step 2 – Push Nirmata images to your Private Docker Registry

The next step involves pushing Nirmata container images into your private Docker registry. This step is optional and should be required only in case you do not have access to the external Docker registry from your Kubernetes cluster. Nirmata container images are available on Docker Hub. These images will be downloaded and pushed into a repository called “nirmata” in your registry.

  1. Go to the install directory in the Nirmata installer folder and edit the update-registry-images.sh script
  2. Update the following fields with correct values: a. TARGET_REGISTRY: IP address or DNS name of your Docker private registry
  3. Ensure that the SOURCE_TAG and TARGET_TAG are the same
  4. Execute the installer command: :

    update-registry-images.sh
    

It may take some time to transfer all the images. The time taken will vary based on your network speed.

Step 3 – Customize deployment settings

Next, we need to customize the deployment settings for your environment as follows:

  1. For shared services settings, edit the file yaml/shared-services-values.yaml
  2. For nirmata services settings, edit the file yaml/nirmata-services-values.yaml
  3. Ensure that all the required fields have been updated
  4. You will also need to provide base64 encoded certificate and key for the front-end proxy service (nginx)
Step 4 – Install Shared Services

Once all the deployment settings have been added, you can start the installation of the shared services In the install folder, run the shared services install script:

install-shared-services.sh

It may take a few minutes to complete the installation.

Check if all the pods were installed correctly:

kubectl get pods -n nirmata-shared

Note: Sometimes it may take a few minutes for the zookeeper cluster to be formed. Use this command on each zookeeper pod to check if the zookeeper cluster is formed:

kubectl exec  zookeeper-2 -n nirmata-shared bin/zkServer.sh status

You should see:

Using config: /opt/zookeeper-3.5.1/bin/../conf/zoo.cfg
Client port not found in static config file. Looking in dynamic config file.
Client port found: 2181. Client address: zookeeper-2.zookeeper.nirmata-shared.
Mode: leader
Step 5 – Install Nirmata Services

Once all the shared services are running, you can start the installation of the nirmata services

In the install folder, run the nirmata services install script:

install-nirmata-services.sh

It may take a few minutes to complete the installation.

Check if all the pods were installed correctly:

kubectl get pods -n nirmata-services

Note: The nginx pods may restart a few times as other services are coming up. Once all the services are up and running, nginx pod should be running as well

Step 6 – Log in to Nirmata

You can login to Nirmata using the URL, the email and password you have specified in the installation script (NIRMATA_URL, USER_EMAIL, USER_PASSWORD).

Step 7 – Add the license

Once you log in to Nirmata, go to Account->Settings and update the license key. Once a valid license key is added, you will no longer see the ‘Trial’ message.

Annex A – Docker CE Installation on RHEL 7

Disable firewall as it interferes with Docker:

sudo systemctl disable firewalld

Set up the Docker CE repository on RHEL:

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum makecache fast

Install the latest supported version of Docker CE on RHEL:

sudo yum -y install docker-ce-17.06.0.ce-1.fc25

Alternatively, you can specify a specific version of Docker CE:

sudo yum -y install docker-ce-<version>-<release>

Start Docker:

sudo systemctl start docker

Test your Docker CE installation:

sudo docker run hello-world

If SELinux is not already installed you can can following these instructions instead:

# pre-requisite for container-selinux-2.9-4.el7.noarch.rpm
sudo yum install policycoreutils-python

wget http://mirror.centos.org/centos/7/extras/x86_64/Packages/container-selinux-2.9-4.el7.noarch.rpm
sudo rpm -i container-selinux-2.9-4.el7.noarch.rpm

#Set up the Docker CE repository on RHEL:
sudo yum install -y yum-utils
sudo yum install -y device-mapper-persistent-data lvm2
sudo yum-config-manager --enable rhel-7-server-extras-rpms
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum makecache fast

# Install the latest supported version of Docker CE on RHEL:
sudo yum -y install docker-ce-17.06.0.ce-1.fc25

#Start Docker:
sudo systemctl start docker

#Test your Docker CE installation:
sudo docker run hello-world

# configure Docker to start on boot
sudo systemctl enable docker

# add user to the Docker group
sudo usermod -aG docker jethro

# install Docker Compose:
# install python-pip
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

sudo yum install ./epel-release-latest-7.noarch.rpm
sudo yum install -y python-pip

sudo pip install docker-compose

# upgrade your Python packages:
sudo yum upgrade python*

Annex B – Troubleshooting Installation Issues

1) Kafka container fails to start or keeps restarting This typically happens when Kafka cannot connect with Zookeeper. In this case, do the following:

Check if Zookeeper is running: kubectl exec zookeeper-0 -n nirmata-shared bin/zkServer.sh status

You should see:

Using config: /opt/zookeeper-3.5.1/bin/../conf/zoo.cfg
Client port not found in static config file. Looking in dynamic config file.
Client port found: 2181. Client address: zookeeper-2.zookeeper.nirmata-shared.
Mode: leader

2) Web console cannot be reached

Check if all the services are running using docker ps or kubectl get pods command. In case any service is not running, restart it. If it cannot be restarted, contact Nirmata support.

Check if the frontend load balancer is reachable using curl command:

curl https://<nirmata-url>/webclient

Check the frontend load balancer logs for any errors:

docker logs -f nginx

In case you see any errors, restart the load balancer. If it cannot be restarted, contact Nirmata support.

3) Unable to login to Nirmata

Check if all the services are running using docker ps or kubectl get pods command.

Check the logs for the users service and security service for any errors

In case you see any errors, restart the users service and security service. If they cannot be restarted, contact Nirmata support.