Automated using Terraform

The user performing the integration must have the following roles assigned:

1. Organisation Administrator on your organisation

2. Billing Account Administrator on your organisation/billing account

3. Service Usage Administrator on your organisation

Why this is needed?

Organisation Administrator role is used to create a new billing project and assign the organisation level roles.

Billing Account Administrator role is used to assign Billing account viewer role for the Service Account and the external user.

Service Usage Administrator role is used to check if the required APIs are enabled.

Below APIs are enabled as part of the script on your projects for OneLens to be able to read usage data on the respective services:

1. Vertex AI API (aiplatform.googleapis.com)

2. Cloud Functions API (cloudfunctions.googleapis.com)

3. Cloud SQL Admin API (sqladmin.googleapis.com)

4. Compute Engine API (compute.googleapis.com)

5. Kubernetes Engine API (container.googleapis.com)

6. Dataflow API (dataflow.googleapis.com)

7. Cloud Dataproc API (dataproc.googleapis.com)

8. Cloud Filestore API (file.googleapis.com)

9. Cloud Monitoring API (monitoring.googleapis.com)

10. Network Management API (networkmanagement.googleapis.com)

11. Recommender API (recommender.googleapis.com)

12. Google Cloud Memorystore for Redis API (redis.googleapis.com)

13. Service Usage API (serviceusage.googleapis.com)

14. Cloud Asset API (cloudasset.googleapis.com)

15. BigQuery API (bigquery.googleapis.com)

1

Uploading the Terraform code

• Login to the Google Cloud Platform console.

• In the top bar, click the Cloud Shell button to launch Google Cloud's interactive shell.

  

• In the opened pop-up window, authorize Cloud Shell to use the logged in user's credentials.

  

• After your credentials are verified, you will be greeted with the opened Cloud Shell terminal.

  

• Click the 3 dots on the top right of the terminal window, and select Upload.

  

• Select the Folder option, and use the Choose Folder button to navigate to and select the GCP Deployment Script folder shared to you by the OneLens team.

  

  The Terraform script would have been shared to you in a ZIP format. Please make sure to uncompress the archive into the folder named onelens-gcp-onboarding.

• The window displays the files that are going to be uploaded to your Cloud Shell terminal instance for your confirmation. Click Upload. For more details on what each file contains, please navigate to the Appendix section at the end of this guide.

  

• Cloud Shell provides a confirmation that the files are successfully uploaded.

  

• Enter the following command to navigate to the folder that was uploaded: cd onelens-gcp-onboarding

  

• Enter the following command to make the .SH script file executable: chmod +x deploy.sh

  

• Enter the following command to run the script: ./deploy.sh

  

2

Configuring your settings

• The script greets the user and proceeds to the first step of checking your permissions. The logged in user's ID is displayed for confirmation. As prompted, enter your Organisation ID.

  

• Next, enter your Billing Account ID.

  

  To find your Organisation ID:

  Go to Organisations, then Organisation details:

  

  To find your Billing account ID:

  Search for and open Account management, and select your Billing account in the dropdown on the left.

  

• The script will check your permissions against the provided organisation and the billing account. It checks for the below roles and displays a confirmation if they are present:

  • Organisation Administrator (roles/resourcemanager.organisationAdmin)

  • Billing Account Administrator (roles/billing.admin)

  • Service Usage Administrator (roles/serviceusage.serviceUsageAdmin)

    

• In case the required roles are not assigned, the script will prompt you to attempt to assign the Billing Account Administrator and Service Usage Administrator roles for you automatically. Note: Organisation Administrator role is required to attempt role assignment.

  

• On proceeding, the script will prompt you for your company name, to be entered in small characters. This is used to optionally create a unique billing project name.

  

• Previously provided Organisation ID and the Billing Account ID values are displayed to be used. The script prompts you to enter the External User email ID. On pressing enter, the default value [email protected] is used. Note: If another value is provided to you by the OneLens team during onboarding, please enter the same.

  

• Similarly, the script prompts you to enter the name of the Service Account ID to be created. On pressing enter, the default value onelens-reader-sa is used. Note: If another value is provided to you by the OneLens team during onboarding, please enter the same.

  

• The next step involves configuring the Billing project:

  • If you have an existing project set up to host Billing data through BigQuery, you can enter the full ID of the project.

    It is recommended to create a new project to ensure your existing resource hierarchy is unaffected by resources created by OneLens. Please check the next step on how to use the script to do so.

    

  • Recommended: If you want to create a new BigQuery project and dataset to export your billing data to, then press enter without entering a value. You can press enter again to use the default generated name.

    

• In the next step, you can choose to onboard folders or projects:

  • Folders Onboard all projects in a folder by providing a folder ID. You can provide multiple folder IDs if needed.

    

    When prompted to enter projects, you can provide any project that is not in the provided folder(s) to be added. If no additional projects are to be onboarded, simply press enter without entering a value. A summary will be displayed of all entered values.

    

  • Projects Onboard one or multiple projects. When prompted to enter folders, simply press enter without entering a value. The script will then prompt you to enter project IDs.

    

    A summary will be displayed of all entered values.

    

• On proceeding, the script runs the terraform init and terraform plan commands. Then, a summary of the resources to be created is displayed for your reference.

  
• Type yes and hit enter to apply the Terraform plan. This runs the terraform apply command.

  

3

Completing the onboarding

Wait ~2 minutes for the script to finish enabling the required APIs, create the billing export project and assign the required roles on selected resources.

After successfully completing, the script will display a confirmation and the following values:

• New billing project ID

• New service account email

• New BigQuery dataset ID

You have now successfully integrated your Google Cloud Platform environment with OneLens.

Please share the following values to the OneLens team to facilitate the connection on our end:

• Billing project ID

• Service Account email ID

• BigQuery dataset ID

4

Appendix

In the below section, an overview of what the code does and what each files contain is provided at a high-level:

• deploy.sh (Bash script)

  • This is a user-facing shell wrapper script that orchestrates the entire deployment.

  • It interactively prompts the user for required inputs, performs gcloud permission pre-checks, auto-generates the terraform.tfvars file, and then executes the terraform init, plan, and apply commands in sequence.

• main.tf (Terraform Configuration)

  • This is the core file containing the declarative infrastructure-as-code (IaC) logic.

  • Core Logic

    • Project: Creates a new GCP project or uses/validates var.existing_billing_export_project_id if provided. All subsequent resources depend on this project being available.

  • Billing Export Project & Config

    • google_project.billing_export: Creates a new project if var.existing_billing_export_project_id is empty.

    • data.external.check_existing_billing_project: Validates an existing project's accessibility.

    • google_project_service.billing_export_apis: Enables serviceusage, cloudresourcemanager, billingbudgets, bigquery, cloudbilling, and cloudasset APIs on the billing project.

    • google_bigquery_dataset.billing_export_dataset: Creates the billing_export dataset (in US) if it doesn't exist.

    • data.external.check_billing_export_dataset: Checks if billing_export dataset already exists.

    • null_resource.configure_*_export: Logs instructions to manually configure billing exports. No exports are automated.

  • FinOps Service Account (SA)

    • google_service_account.finops_sa: Creates the FinOps SA (from var.finops_service_account_id) in the billing project if it doesn't exist.

    • data.external.check_existing_finops_sa: Checks if the FinOps SA already exists.

  • Target Project Onboarding

    • data.google_projects.*: Discovers all target projects (from var.target_project_ids, var.target_folder_ids, or entire Org if both are empty).

    • null_resource.enable_billing_on_projects: Links all target projects to var.billing_account_id if not already billed.

    • google_project_service.target_project_apis_*: Enables required APIs on all target projects.

      • Billing Required APIs: compute.googleapis.com, container.googleapis.com, dataflow.googleapis.com, dataproc.googleapis.com, file.googleapis.com, redis.googleapis.com

      • No-Billing APIs: aiplatform.googleapis.com, cloudfunctions.googleapis.com, sqladmin.googleapis.com, monitoring.googleapis.com, networkmanagement.googleapis.com, recommender.googleapis.com, serviceusage.googleapis.com, cloudasset.googleapis.com, bigquery.googleapis.com

  • IAM Bindings: FinOps SA (serviceAccount:${local.finops_sa_email})

    • Scope: Organization

      • roles/resourcemanager.organizationViewer

      • roles/cloudasset.viewer

      • roles/browser

    • Scope: Billing Account

      • roles/billing.viewer

    • Scope: Billing Export Project

      • roles/bigquery.dataViewer

      • roles/bigquery.metadataViewer

      • roles/bigquery.jobUser

      • roles/bigquery.readSessionUser

    • Scope: All Target Projects & Folders

      • roles/aiplatform.viewer

      • roles/cloudfunctions.viewer

      • roles/cloudsql.viewer

      • roles/compute.viewer

      • roles/container.viewer

      • roles/dataflow.viewer

      • roles/dataproc.viewer

      • roles/file.viewer

      • roles/monitoring.viewer

      • roles/networkmanagement.viewer

      • roles/recommender.viewer

      • roles/redis.viewer

      • roles/serviceusage.serviceUsageViewer

      • roles/bigquery.metadataViewer

      • roles/bigquery.jobUser

      • roles/bigquery.resourceViewer

  • IAM Bindings: OneLens Backend SA (serviceAccount:[email protected])

    • Scope: FinOps SA

      • roles/iam.serviceAccountTokenCreator (Allows platform to impersonate FinOps SA)

  • IAM Bindings: External User (user:${var.external_user_email})

    • Scope: FinOps SA

      • roles/iam.serviceAccountTokenCreator (Allows user to impersonate FinOps SA)

    • Scope: Organization

      • roles/resourcemanager.organizationViewer

      • roles/cloudasset.viewer

      • roles/browser

    • Scope: Billing Account

      • roles/billing.viewer

    • Scope: Billing Export Project

      • roles/bigquery.jobUser

      • roles/bigquery.dataViewer

      • roles/bigquery.metadataViewer

    • Scope: All Target Projects

      • roles/viewer

      • roles/bigquery.resourceViewer

      • roles/bigquery.metadataViewer

      • roles/bigquery.jobUser

    • Scope: All Target Folders

      • roles/viewer

      • roles/bigquery.resourceViewer

      • roles/bigquery.metadataViewer

      • roles/bigquery.jobUser

• provider.tf (Terraform Configuration)

  • This is a mandatory Terraform file that specifies the required providers for this configuration (e.g., hashicorp/google).

  • It defines provider version constraints and the basic provider configuration block, telling Terraform how to interact with the target GCP APIs.

• variables.tf (Terraform Configuration)

  • This is a mandatory Terraform file that defines the input API for the configuration.

  • It declares all variables the module accepts (e.g., organization_id, target_project_ids), along with their types, descriptions, and default values.

• outputs.tf (Terraform Configuration)

  • This is a mandatory Terraform file that declares values to be exported after the configuration is applied.

  • It makes key resource attributes, like the newly created service_account_email or billing_project_id, available on the command line for validation or use in other scripts.

• terraform.tfvars (Terraform Data)

  • This is a mandatory Terraform file (or must be supplied via CLI flags) that provides the actual values for the variables defined in variables.tf.

  • This specific file is auto-generated by the deploy.sh script to separate user-specific data (like project IDs) from the resource logic in main.tf.