Skip to main content

Ingestion

Cost Management ingestion is responsible for bringing detailed cost and usage records from each cloud into Cloudaware, validating them, and making them ready for allocation, reporting, and optimization.

This section explains how billing data flows into Cloudaware, how it connects to Billing Integrations, and how to keep the data healthy over time.

How Ingestion Fits into Cost Management​

  • Cloud billing exports are configured using provider billing guides for AWS, Microsoft Azure, Google Cloud, Oracle Cloud, and Kubernetes.
  • Cloudaware parses large, detailed billing files (for example, AWS CUR/Data Exports, Azure EA/MCA, Google Billing Export) and normalizes them into a unified cost model.
  • Cost and usage lines are enriched with CMDB data (accounts, projects, subscriptions, workloads, tags) so they can be allocated to business units, applications, and teams.
  • Historical billing data is preserved even if a cloud account or subscription is later removed from the cloud integration.

High-Level Ingestion Workflow​

  1. Review prerequisites.
    • Confirm that the relevant cloud integrations (AWS, Azure, Google Cloud, and others) are configured in Cloudaware with read‑only access.
    • Ensure you have the necessary permissions in each cloud portal to create or modify billing exports.
  2. Configure billing exports.
    • For AWS, configure Cost & Usage Reports or Data Exports (and legacy DBR if still required).
    • For Microsoft Azure, configure Cost Management exports for billing accounts or subscriptions.
    • For Google Cloud, enable Billing Export to BigQuery and/or file exports.
    • For Oracle Cloud, configure billing exports.
    • For Kubernetes, enable cost attribution using OpenCost.
  3. Verify ingestion.
  4. Monitor health.
    • Establish a lightweight routine (for example, weekly) to review data freshness and coverage using the guidance in Data Health & Freshness.
    • Investigate any gaps or anomalies before relying on the data for budgeting, anomaly detection, or optimization planning.

Reading Path​

Use these guides together to configure billing sources, understand the ingested data model, and monitor data quality over time:

Cloud Billing Sources​

Start here when you are planning which provider billing sources to connect. Use this guide to understand which billing exports and APIs Cloudaware uses for AWS, Microsoft Azure, Google Cloud, Oracle Cloud, Kubernetes, and custom billing sources.

Cloudaware Billing Data Upload Schema​

Understand how Cloudaware expects billing data to be structured, normalized, enriched, and published to analytics. Refer to this guide when you need to validate export contents, map fields, or prepare custom billing data for ingestion.

Data Health & Freshness​

Monitor billing data freshness, completeness, and provider latency. Refer to this guide when troubleshooting missing costs, delayed discounts, partial periods, or differences between Cloudaware and cloud billing consoles.