GCP Setup

GCP resources provisioning for deploying “Raven”

Introduction

This document guides you through provisioning GCP resources using Terraform for deploying the “Raven” SaaS product. The example repository is for guidance purposes to help you understand the requirements and needed GCP resources to run the Raven platform (self-hosted). It is not intended for production use.

Prerequisites

Before starting, ensure you have the following:

1. Terraform installed.
2. Google Cloud SDK (gcloud) configured with your GCP account.
3. Pre-existing GCP resources:
   • VPC Network
   • Private subnets
   • GKE cluster
   • Subnet groups for Cloud SQL and Memorystore
   • Access to the Cloud Run service’s container image hosted by Raven

Step-by-Step Guide

1. Clone the Repository

git clone https://gitlab.com/Raven-IO/gcp-terraform-example.git
cd gcp-terraform-example

2. Configure Backend

Configure a proper Terraform backend to store the state files securely. Create or modify the backend.tf file:

terraform {
  backend "gcs" {
    bucket = "your-terraform-state-bucket"
    prefix = "raven/terraform.tfstate"
  }
}

3. Configure GCP Provider

In providers.tf, configure the GCP provider with the required project and credentials:

provider "google" {
  project = var.gcp_project
  region  = var.gcp_region
}

4. Initialize Terraform

terraform init

5. Review and Modify Variables

Open the variables.tf file and set appropriate values for the variables. Modify defaults as needed:

variable "gcp_project" {
  description = "The GCP project ID to deploy resources."
  type        = string
}

variable "gcp_region" {
  description = "The GCP region to deploy resources."
  default     = "us-central1"
}

variable "private_subnets" {
  type        = list(string)
  description = "The private subnets to deploy the Compute Engine instance."
}

variable "vpc_id" {
  type        = string
  description = "The VPC ID to deploy the resources."
}

variable "additional_labels" {
  default = {
    "terraform" = "true"
  }
  description = "Additional labels to attach to the resources."
}

variable "memorystore_subnet_name" {
  type        = string
  description = "The name of the Memorystore subnet to use for the Redis instance."
}

variable "allowed_cidr_blocks" {
  type        = list(string)
  description = "The CIDR blocks that are allowed to access the Cloud SQL (Postgres) / Memorystore (Redis) / Compute Engine (ClickHouse) instance."
}

variable "allowed_service_accounts" {
  type        = list(string)
  description = "The service accounts that are allowed to access the Cloud SQL (Postgres) / Memorystore (Redis) instance."
}

variable "cloud_sql_subnet_name" {
  type        = string
  description = "The name of the Cloud SQL subnet to use for the Postgres instance."
}

variable "container_image_uri" {
  type        = string
  description = "The URI of the container image to use for the Cloud Run service | Your GCP project should have access to this image."
}

variable "prefix" {
  type        = string
  description = "The prefix to use for the resources."
}

6. Plan the Deployment

Generate an execution plan to preview the changes:

terraform plan

7. Apply the Terraform Configuration

Apply the changes to your GCP environment:

terraform apply

Confirm the apply step by typing yes when prompted.

Terraform Configuration Details

• Compute Engine Instances: The configuration includes setting up Compute Engine instances with appropriate IAM roles and security settings.

  • Defined in compute.tf, the script provisions Compute Engine instances, attaches IAM roles, and configures firewall rules to allow necessary traffic.
  • We use Bitnami official image to deploy ClickHouse. You can change this in the clickhouse.tf file.

• Memorystore for Redis: Managed Redis instances using Memorystore.

  • Found in memorystore.tf, it sets up Redis with specified node types and cluster settings, ensuring proper subnet and security group configurations.
  • This module exports redis_host and redis_port which will then be passed to the Cloud Run service as environment variables.

• Cloud SQL (PostgreSQL): Provisions a managed database instance.

  • The cloudsql.tf file includes configurations for instance type, storage, and security groups, tailored to meet the requirements of the SaaS product.

• Cloud Run: Sets up Cloud Run services triggered by Cloud Storage events.

  • In cloudrun.tf, the configuration includes creating Cloud Run services, assigning IAM roles, and setting up Cloud Storage event triggers.
  • This module exports the following outputs:
    • cloudrun_service_url which will be used to configure Cloud Storage bucket to send events to the Cloud Run service.
    • cloudrun_service_account which will be used to allow the Cloud Run service to access the Memorystore/Redis database.

Post Deployment

Verify the resources in the Google Cloud Console to ensure they are created as expected. Use Cloud Monitoring and other monitoring tools to manage and observe the performance and health of the provisioned resources.

Clean Up

To destroy the resources created by Terraform when they are no longer needed:

terraform destroy

Confirm the destroy step by typing yes when prompted.

For more detailed information, refer to the GitLab repository.

Assistant

Responses are generated using AI and may contain mistakes.