OpenCost on Google Cloud

Use this guide to deploy OpenCost components for Kubernetes Billing exports to Google Cloud Storage.

Prerequisites

Before you begin, make sure you have:

• Infrastructure
  • A GKE cluster in Standard mode (not Autopilot) with Workload Identity enabled
• Required cloud permissions (GCP IAM roles)
  • roles/container.admin for GKE administration
  • roles/iam.serviceAccountAdmin for managing service accounts
  • roles/storage.admin for managing the GCS bucket
  • roles/apikeys.admin for managing API keys
• Required tools
  • Google Cloud CLI (gcloud)
  • kubectl
  • Helm (v3.x or later)

note

Autopilot is not supported due to the managed Prometheus configuration.

Set Up Access

Create a Service Account

1. In the Google Cloud console, navigate to IAM & Admin → Service Accounts.
2. Click + Create service account.
3. Set:
   • Name: opencost-sa (service account name used in this guide).
   • Description: Service account for OpenCost.
4. Assign the role:
   • roles/compute.viewer.
5. Click Done to create the account.

Configure Workload Identity

1. Open the newly created service account.
2. Select the Principals with access tab and click Grant access.
3. Add the following principal: {YOUR_GCP_PROJECT_ID}.svc.id.goog[opencost/opencost-sa] (replace {YOUR_GCP_PROJECT_ID} and opencost-sa as needed).
4. Assign the Workload Identity User role and save.

Create a Service Account Key

1. Navigate back to the Service Accounts list.
2. For opencost-sa, open the Actions (three‑dot) menu → Manage keys.
3. Click Add key → Create new key.
4. Select JSON and click Create.
5. Download and store the key securely; it will be used later when creating the Kubernetes secret.

Create an API Key

1. Navigate to APIs & Services → Credentials.
2. Click + Create credentials and select API key.
3. Copy the generated key.
4. From the Actions (three‑dot) menu of the new key, select Edit API key.
5. Under API restrictions, select Restrict key.
6. Check Cloud Billing API (cloudbilling.googleapis.com) → OK, then Save.

Set Up the Storage Bucket

1. Navigate to Cloud Storage → Buckets.
2. Click + Create and configure:
   • Name: for example, opencost-bucket (must be globally unique).
   • Location type: Region.
   • Storage class: Standard.
3. Leave remaining settings as is and click Create.

Grant bucket access:

1. Open the bucket and go to Permissions → + Grant access.
2. Select:
   • Principal: opencost-sa@{YOUR_GCP_PROJECT_ID}.iam.gserviceaccount.com (replace {YOUR_GCP_PROJECT_ID} as needed).
   • Role: Storage Object User.
3. Click Save.

note

Use the storage bucket name consistently in the later steps.

Create the Cluster Connection

1. In the Google Cloud console, go to Kubernetes Engine → Clusters.
2. For the target cluster, open the Actions (three‑dot) menu and select Connect.
3. Copy the gcloud command and run it in your terminal to generate/update your kubeconfig for the cluster.

Deploy Prometheus

1. Add the Prometheus Helm repository:

   helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

2. Create the Prometheus namespace:

   kubectl create namespace prometheus-system

3. Install Prometheus:

   helm install prometheus prometheus-community/prometheus \
     --namespace prometheus-system \
     --set prometheus-pushgateway.enabled=false \
     --set alertmanager.enabled=false

tip

If you use an external Prometheus instance, skip this step and review the Configuration notes (1) for external Prometheus below.

Deploy OpenCost

1. Add the OpenCost Helm repository:

   helm repo add opencost https://opencost.github.io/opencost-helm-chart

2. Create the OpenCost namespace:

   kubectl create namespace opencost

3. Create a Kubernetes secret with the service account key. Replace /path/to/your/service-account-key.json with the path to the JSON key created earlier:

   kubectl create secret generic google-application-credentials \
     --from-file=config.json=/path/to/your/service-account-key.json \
     --namespace opencost

4. Install OpenCost. Replace {YOUR_PROJECT_ID} and {YOUR_API_KEY}:

   helm --namespace opencost upgrade --install opencost opencost/opencost -f - <<EOF
   podSecurityContexts:
     runAsUser: 1001
     runAsGroup: 1001
     fsGroup: 1001
   extraVolumes:
     - name: configs
       emptyDir: {}
   serviceAccount:
     create: true
     name: opencost-sa
     annotations:
       iam.gke.io/gcp-service-account: "opencost-sa@{YOUR_PROJECT_ID}.iam.gserviceaccount.com"
   opencost:
     prometheus:
       namespaceName: prometheus-system
     exporter:
       cloudProviderApiKey: {YOUR_API_KEY}
       extraVolumeMounts:
         - mountPath: /var/configs
           name: configs
           readOnly: false
   podAnnotations:
     prometheus.io/path: /metrics
     prometheus.io/port: "9003"
     prometheus.io/scrape: "true"
   EOF

   tip

   The podAnnotations ensure that Prometheus scrapes metrics from the OpenCost pods, enabling label‑based cost attribution in reports.

Configuration Notes

Configure the following Helm values depending on your setup:

1. If your cluster already has an external Prometheus, configure OpenCost to use the existing Prometheus instance. Replace {YOUR_PROMETHEUS_URL} with the correct endpoint:

   opencost:
     prometheus:
       external:
         enabled: true
         url: {YOUR_PROMETHEUS_URL}

2. If your cluster uses prometheus-operator, enable ServiceMonitor; otherwise, OpenCost metrics will not be scraped automatically:

   opencost:
     metrics:
       serviceMonitor:
         enabled: true

Deploy the Parquet Exporter

Install the OpenCost Parquet exporter.

• To store reports in subfolders or use a naming prefix, use the command below, replacing {YOUR_CLUSTER_PREFIX}, {YOUR_GKE_CLUSTER_ID} and {YOUR_OPENCOST_PARQUET_GCP_BUCKET_NAME} as needed:

  helm install parquet-exporter opencost/opencost-parquet-exporter \
    --namespace opencost \
    --set schedule="0 */12 * * *" \
    --set existingServiceAccount=opencost-sa \
    --values - <<EOF
  resources:
    limits:
      cpu: "1"
      memory: 1Gi
    requests:
      cpu: 100m
      memory: 100Mi
  env:
    - name: OPENCOST_PARQUET_SVC_HOSTNAME
      value: "opencost.opencost.svc.cluster.local"
    - name: OPENCOST_PARQUET_STORAGE_BACKEND
      value: "gcp"
    - name: OPENCOST_PARQUET_FILE_KEY_PREFIX
      value: "{YOUR_CLUSTER_PREFIX}/{YOUR_GKE_CLUSTER_ID}"
    - name: OPENCOST_PARQUET_JSON_SEPARATOR
      value: "_"
    - name: OPENCOST_PARQUET_GCP_BUCKET_NAME
      value: "{YOUR_OPENCOST_PARQUET_GCP_BUCKET_NAME}"
    - name: OPENCOST_PARQUET_GCP_CREDENTIALS_JSON
      valueFrom:
        secretKeyRef:
          name: google-application-credentials
          key: config.json
  EOF

• To store reports in the root folder, use this variant instead, replacing {YOUR_GKE_CLUSTER_ID} and {YOUR_OPENCOST_PARQUET_GCP_BUCKET_NAME} as needed:

  helm install parquet-exporter opencost/opencost-parquet-exporter \
    --namespace opencost \
    --set schedule="0 */12 * * *" \
    --set existingServiceAccount=opencost-sa \
    --values - <<EOF
  resources:
    limits:
      cpu: "1"
      memory: 1Gi
    requests:
      cpu: 100m
      memory: 100Mi
  env:
    - name: OPENCOST_PARQUET_SVC_HOSTNAME
      value: "opencost.opencost.svc.cluster.local"
    - name: OPENCOST_PARQUET_STORAGE_BACKEND
      value: "gcp"
    - name: OPENCOST_PARQUET_FILE_KEY_PREFIX
      value: "{YOUR_GKE_CLUSTER_ID}/"
    - name: OPENCOST_PARQUET_JSON_SEPARATOR
      value: "_"
    - name: OPENCOST_PARQUET_GCP_BUCKET_NAME
      value: "{YOUR_OPENCOST_PARQUET_GCP_BUCKET_NAME}"
    - name: OPENCOST_PARQUET_GCP_CREDENTIALS_JSON
      valueFrom:
        secretKeyRef:
          name: google-application-credentials
          key: config.json
  EOF

note

For {YOUR_GKE_CLUSTER_ID}, Cloudaware expects a value in the format:
projects/{YOUR_PROJECT_ID}/zones/{YOUR_ZONE}/clusters/{YOUR_CLUSTER_NAME}

To retrieve this value, run:

gcloud container clusters describe {YOUR_CLUSTER_NAME} \
  --project {YOUR_GCP_PROJECT_ID} \
  --region {YOUR_GCP_REGION} \
  --format="value(selfLink)" | sed 's|.*/projects/|projects/|'

warning

For multi‑cluster environments, deploy the Parquet exporter separately on each cluster.

Verify the Deployment

1. Check that all pods are running:

   kubectl get pods -n prometheus-system
   kubectl get pods -n opencost

2. Access the OpenCost UI:

   kubectl port-forward -n opencost service/opencost 9003:9003

Then open http://localhost:9003 in your browser.

note

The first Parquet export may take up to 24 hours. To verify the export, check your storage bucket for newly created files.

Next Steps

Return to the parent guide Kubernetes Billing (OpenCost) to proceed with the Kubernetes Billing setup in Cloudaware.