OpenCost Deployment for Google Cloud

This guide explains how to deploy OpenCost for a GKE cluster.

Prerequisites

Before starting the deployment, ensure you have the following:

• Infrastructure:

  • GKE cluster (Standard mode, not Autopilot^*) with Workload Identity enabled

• Cloud permissions (GCP IAM roles):

  • roles/container.admin – required for GKE management

  • roles/iam.serviceAccountAdmin – required for Service Account management

  • roles/storage.admin – required for GCS bucket management

  • roles/apikeys.admin – required for API key management

• Tools:

  • gcloud CLI tool

  • kubectl CLI tool

  • helm CLI tool (v3.x or later)

^*Autopilot is not supported due to the managed Prometheus configuration.

STEP 1. Service account setup

Create a service account

1. In the Google Cloud console, navigate to IAM & Admin -> Service Accounts.

2. Click +Create service account.

3. Set the following:

   • Name: opencost-sa

   • Description: Service account for OpenCost

4. Assign the required roles:

   • roles/compute.viewer

   • roles/bigquery.admin (only if using BigQuery integration)

   • roles/storage.admin (required for Parquet export)

5. Click Done to create the account.

Configure workload identity

1. Return to the Service Accounts list. Open the newly created service account.

2. Select the tab ‘Principals with access’. Click Grant access.

3. Add the following principal: YOUR_GCP_PROJECT_ID.svc.id.goog[opencost/opencost-sa] where YOUR_GCP_PROJECT_ID is a placeholder for your GCP Project ID

4. Assign the Workload Identity User role and save.

Create a service account key

1. Navigate back to the Service Accounts list. Click on Actions (three-dot) menu → Manage keys.

2. Click Add Key → Create new key.

3. Select JSON and click Create.

4. Download and store the key securely. The key will be required later when creating the Kubernetes secret.

Create an API key

1. Navigate to APIs & Services -> Credentials.

2. Click +Create Credentials and select API key.

3. Copy the generated key.

4. From the Actions (three-dot) menu of the new key, select Edit API key.

5. Under API restrictions, select Restrict key. Check the box ‘Cloud Billing API’ (cloudbilling.googleapis.com) → OK. Click Save.

STEP 2. Storage bucket setup

1. Navigate to Cloud Storage -> Buckets.

2. Click +Create.

3. Configure the bucket:

   • Name: opencost-bucket

   • Location type: Region

   • Storage class: Standard
     Leave the remaining settings as is.

4. Click Create.

STEP 3. Create a cluster connection file

1. Navigate to Kubernetes Engine → Clusters.

2. Click the Actions (three-dot) menu next to the cluster and select Connect.

3. Copy the gcloud command and run it in your terminal. This generates the kubeconfig file needed to access the cluster.

STEP 4. Prometheus deployment

1. Add the Prometheus Helm repository:

   CODE

   helm repo add prometheus-community <https://prometheus-community.github.io/helm-charts>

2. Create the Prometheus namespace:

   CODE

   kubectl create namespace prometheus

3. Install Prometheus:

   CODE

   helm install prometheus prometheus-community/prometheus \\
     --namespace prometheus \\
     --set prometheus-pushgateway.enabled=false \\
     --set alertmanager.enabled=false

STEP 5. OpenCost deployment

1. Add the OpenCost Helm repository:

   CODE

   helm repo add opencost <https://opencost.github.io/opencost-helm-chart>

2. Create the OpenCost namespace:

   CODE

   kubectl create namespace opencost

3. Create a Kubernetes secret with the service account key. Replace /path/to/your/service-account-key.json with your file path.

   CODE

   kubectl create secret generic google-application-credentials \\
     --from-file=config.json=/path/to/your/service-account-key.json \\
     --namespace opencost

4. Install OpenCost. Replace <YOUR_PROJECT_ID> and <YOUR_API_KEY> in the command below:

   CODE

   helm --namespace opencost upgrade --install opencost opencost/opencost -f - <<EOF
   serviceAccount:
     create: true
     name: opencost-sa
     annotations:
       iam.gke.io/gcp-service-account: "opencost-sa@<YOUR_PROJECT_ID>.iam.gserviceaccount.com"
   opencost:
     prometheus:
       namespaceName: prometheus
     exporter:
       cloudProviderApiKey: <YOUR_API_KEY>
   podAnnotations:
     prometheus.io/path: /metrics
     prometheus.io/port: "9003"
     prometheus.io/scrape: "true"
   EOF

These podAnnotations ensure that Prometheus scrapes metrics from the OpenCost pods, enabling label-based cost attribution in reports.

STEP 6. Parquet Exporter setup

1. Install the OpenCost Parquet Exporter. Replace <YOUR_GKE_CLUSTER_ID> in the command below:

   CODE

   helm install parquet-exporter opencost/opencost-parquet-exporter \\
     --namespace opencost \\
     --set schedule="0 */12 * * *" \\
     --set existingServiceAccount=opencost-sa \\
     --values - <<EOF
   resources:
     limits:
       cpu: "1"
       memory: 1Gi
     requests:
       cpu: 100m
       memory: 100Mi
   env:
     - name: OPENCOST_PARQUET_SVC_HOSTNAME
       value: opencost.opencost.svc.cluster.local
     - name: OPENCOST_PARQUET_STORAGE_BACKEND
       value: gcp
     - name: OPENCOST_PARQUET_FILE_KEY_PREFIX
       value: "kubernetes_clusters/<YOUR_GKE_CLUSTER_ID>"
     - name: OPENCOST_PARQUET_JSON_SEPARATOR
       value: "_"
     - name: OPENCOST_PARQUET_GCP_BUCKET_NAME
       value: opencost-bucket
     - name: OPENCOST_PARQUET_GCP_CREDENTIALS_JSON
       valueFrom:
         secretKeyRef:
           name: google-application-credentials
           key: config.json
   EOF

^*For multi-cluster environments, deploy the Parquet Explorer separately on each cluster.

STEP 7. Verification

1. Check that all pods are running:

   CODE

   kubectl get pods -n prometheus
   kubectl get pods -n opencost

2. Access the OpenCost UI:

   CODE

   kubectl port-forward -n opencost service/opencost 9003:9003

   Then open http://localhost:9003 in your browser.
   

The first Parquet export may take up to 24 hours. To verify the export, check your storage bucket for newly created files.