Deploying Metric Insights in Orchestrated Environment

Starting from Version 6.x, Metric Insights can be deployed using a container orchestration platform:

  • Kubernetes
  • Amazon ECS
  • Docker Swarm

1. Prerequisites

To deploy Metric Insights across the different orchestration platforms, the following architectural pieces are required:

  • kubectl command-line tool to manage a Kubernetes cluster (not required for ECS and Docker Swarm)
  • Remote Database Server to host the application database (e.g., MySQL/MariaDB, Amazon RDS or equivalent)
  • Persistent shared storage to store the application file system (e.g., NFS, Portworx, EFS)
  • Specific ports open on the network:
    • 80, 443 - HTTP and HTTPS ports for the UI Application Service (by default redirection to 443)
    • 2550 - TCP port for the Data Processor cluster within the kubernetes namespace
    • 2551 - TCP port for the Seed service within the kubernetes namespace
    • 32550 - TCP port for external access to the Data Processor cluster
    • 32551 - TCP port for external access to the Seed service
    • 3306 - MySQL port to get access from outside
    • 8080,8443 - HTTP and HTTPS ports for the REST API Data Processor Service (only one port is enabled at one time)
    • 8081 - TCP port for Monitoring Tool

For non-orchestrated environments, please see the help article on using Simple Installer

2. Obtain Docker Registry Credentials

Contact MI Support for access to the official Metric Insights Docker Registry. Credentials are needed to pull docker images for each Metric Insights service.

  • Note, the default MI Docker Registry address (docker.metricinsights.com:5002) is specified in the deployment configuration file for each orchestration type.

3. Choose Deployment Method (Kubernetes, Amazon ECS, Docker Swarm)

If deploying to Kubernetes, please see Deploying Metric Insights on Kubernetes.

If deploying to Amazon ECS, continue below.

Amazon ECS Prerequisites:

  1. Database (RDS or EC2 instance with custom database deployment)
  2. EFS or custom NFS shared storage
  3. Optional: If utilizing a private registry (non-Metric Insights), ensure that you have those credentials available.

 

4. Generate Configuration File to Deploy to Amazon ECS

The configuration file can be generated using the Metric Insights installer package:

  1. Download the installer package to a linux system and unpack
  2. Change into the installer directory then run the installer with the ecs command and specify a target filename to generate the configuration file:
    $ ./installer.py ecs -o <filename>.json
  3. The configuration file can now be used as a template with AWS CloudFormation to create and deploy the Metric Insights environment

5. Create the ECS Stack with AWS CloudFormation

Prepare the following:

  1. RDS address with root credentials
  2. EFS address to connect to Metric Insights application

Apply the configuration file through the CloudFormation UI:

  1. Upload the generated json file as a template

2.    Fill out each field then click [Next] at the bottom of the page. Some key notes:

  • To generate passwords for each service, you can either run echo -n '<pwd>' | base64 to encode a password of your choice, or run something like openssl rand -base64 8 to auto generate a password for you.
  • Use the full RDS address for the field `DBHostName`
  • Enter the RDS root user in the field `DBRootUserName`
  • Enter the full EFS address in the field `NFSServerAddress`
  • Select all Subnet IDs available in the field `SubnetIDs`
  • The field `WebReplicationsCount` represents the number of web slave containers (secondary to web master)

3.   Click [Next] to skip through the subsequent pages until you reach the following window. Click the checkbox to acknowledge that IAM resources might be created on deployment and click the button [Update stack].

4.   Allow incoming connections to RDS for new ECS/EC2 security group to complete the deployment

  1. As the new ECS Stack is being deployed, go to the EC2 Console and select one of the new EC2s created for ECS
  2. Go to the Security Group field and select on the new security group name
  3. Copy the Group ID (e.g., `sg-name`)
  4. Switch to the RDS Console and select the RDS instance being used for ECS
  5. Go to the VPC Security Group field and select the security group name
  6. Switch to the `Inbound` tab and click the [Edit] button
  7. Add the new EC2 security group to the list then [Save]: 
    • Add Rule > All Traffic > Paste Group ID

On adding the group, switch back to CloudFormation to monitor the ECS Stack deployment. The deploy should complete in 5-10 minutes.

6. Accessing Metric Insights Deployment

Once the ECS Stack is deployed:

  1. Switch back to the EC2 Console and select Load Balancers in the left menu pane.
  2. Identify the Load Balancer DNS name to access the Metric Insights application in a browser.
  3. For the best user experience, map the Load Balancer DNS name to a user-friendly name in Amazon Route 53. 

Metric Insights is now deployed in ECS and browser ready.

7. Resources Involved in Running Metric Insights in ECS

  • AWS ECS Task Definitions
  • AWS ECS Cluster
  • AWS ECS Services
  • AWS EC2 Auto Scaling group
  • AWS EC2 Launch Configuration
  • AWS EC2 Security Groups
  • AWS Target Groups
  • AWS Network Load Balancer
  • IAM Roles
  • AWS Secret Manager
  • AWS Cloud Formation (only for deployment and updates)

Non-ECS resources in AWS needed for deployment include:

  1. AWS RDS instance based on MariaDB 10.1 (custom parameter group with log_bin_function_creators enabled)
  2. AWS EFS Shared Storage