AWS Setup

AWS resources provisioning for deploying “Raven”

Introduction

This document guides you through provisioning AWS resources using Terraform for deploying the “Raven” SaaS product. The example repository is for guidance purposes to help you understand the requirements and needed AWS resources to run the Raven platform (self-hosted). It is not intended for production use.

Prerequisites

Before starting, ensure you have the following:

1. Terraform installed.
2. AWS CLI configured with your AWS account.
3. Pre-existing AWS resources:
   • VPC
   • Private subnets
   • EKS cluster
   • Subnet groups for RDS and Elasticache
   • Access to the Lambda’s docker_image_uri hosted by Raven

Step-by-Step Guide

1. Clone the Repository

git clone https://gitlab.com/Raven-IO/aws-terraform-example.git
cd aws-terraform-example

2. Configure Backend

Configure a proper Terraform backend to store the state files securely. Create or modify the backend.tf file:

terraform {
  backend "s3" {
    bucket         = "your-terraform-state-bucket"
    key            = "raven/terraform.tfstate"
    region         = "us-west-2"
    dynamodb_table = "terraform-locks"
  }
}

3. Configure AWS Provider

In providers.tf, configure the AWS provider with the required region and credentials:

provider "aws" {
  region = var.aws_region
  profile = var.aws_profile
}

4. Initialize Terraform

terraform init

5. Review and Modify Variables

Open the variables.tf file and set appropriate values for the variables. Modify defaults as needed:

variable "aws_region" {
  description = "The AWS region to deploy resources."
  default     = "us-west-2"
}

variable "aws_profile" {
  description = "The AWS CLI profile to use."
  default     = "default"
}

variable "private_subnets" {
  type        = list(string)
  description = "The private subnets to deploy the EC2."
}

variable "vpc_id" {
  type        = string
  description = "The VPC ID to deploy the resources."
}

variable "additional_tags" {
  default = {
    "Terraform" = "true"
  }
  description = "Additional tags to attach to the resources."
}

variable "elasticache_subnet_group_name" {
  type        = string
  description = "The name of the Elasticache subnet group to use for the Elasticache (Redis) instance."
}

variable "allowed_cidr_blocks" {
  type        = list(string)
  description = "The CIDR blocks that are allowed to access the RDS (Postgres) / Elasticache (Redis) / EC2 (ClickHouse) instance."
}

variable "allowed_security_groups" {
  type        = list(string)
  description = "The security groups that are allowed to access the RDS (Postgres) / Elasticache (Redis) instance."
}

variable "db_subnet_group_name" {
  type        = string
  description = "The name of the RDS subnet group to use for the RDS (Postgres) instance."
}

variable "docker_image_uri" {
  type        = string
  description = "The URI of the Docker image to use for the Lambda function | Your AWS account should have access to this image."
}

variable "prefix" {
  type        = string
  description = "The prefix to use for the resources."
}

6. Plan the Deployment

Generate an execution plan to preview the changes:

terraform plan

7. Apply the Terraform Configuration

Apply the changes to your AWS environment:

terraform apply

Confirm the apply step by typing yes when prompted.

Terraform Configuration Details

• EC2 Instances: The configuration includes setting up EC2 instances with appropriate IAM roles and security settings.

  • Defined in ec2.tf, the script provisions EC2 instances, attaches IAM roles, and configures security groups to allow necessary traffic.
  • We use Bitnami official AMI to deploy ClickHouse. You can change this in the clickhouse.tf file.

• Elasticache for Redis: Managed Redis instances using Elasticache.

  • Found in elasticache.tf, it sets up Redis with specified node types and cluster settings, ensuring proper subnet and security group configurations.
  • This module exports redis_host and redis_port which will then be passed to the Lambda as an ENV variables.

• RDS (Relational Database Service): Provisions a managed database instance.

  • The rds.tf file includes configurations for instance type, storage, and security groups, tailored to meet the requirements of the SaaS product.

• AWS Lambda: Sets up AWS Lambda functions triggered by S3 events.

  • In lambda.tf, the configuration includes creating Lambda functions, assigning IAM roles, and setting up S3 event triggers.
  • This module exports the following outputs:
    • lambda_arn which will be used to configure AWS S3 bucket to send events to the Lambda.
    • lambda_sg_id which will be used to allow the Lambda to access the ElastiCache/Redis database.

Post Deployment

Verify the resources in the AWS Management Console to ensure they are created as expected. Use CloudWatch and other monitoring tools to manage and observe the performance and health of the provisioned resources.

Clean Up

To destroy the resources created by Terraform when they are no longer needed:

terraform destroy

Confirm the destroy step by typing yes when prompted.

For more detailed information, refer to the GitLab repository.

---

Assistant

Responses are generated using AI and may contain mistakes.