From 1e3b3fbd4f5c65a3643c3afaaebe678149bb27cb Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Mon, 29 Jun 2026 17:42:03 +0000 Subject: [PATCH 1/5] docs: clarify BigQuery project-level permissions for query history and Hex integration - Restructure grant instructions to separate project-level IAM roles (Resource Viewer, Metadata Viewer) from dataset-level access (Data Viewer) - Add explicit note that bigquery.jobs.listAll requires project-level grant - Update permissions-and-security snippet with BigQuery-specific role details - Add prerequisites section to Hex integration page explaining query history dependency - Document multi-project setup for Hex when using a separate execution project Co-Authored-By: Yosef Arbiv --- docs/cloud/integrations/bi/hex.mdx | 19 ++++++++ .../integrations/permissions-and-security.mdx | 10 +++-- .../grant_user_access_on_dataset_level.mdx | 44 ++++++++++++++----- 3 files changed, 59 insertions(+), 14 deletions(-) diff --git a/docs/cloud/integrations/bi/hex.mdx b/docs/cloud/integrations/bi/hex.mdx index fb5e5be30..94e765784 100644 --- a/docs/cloud/integrations/bi/hex.mdx +++ b/docs/cloud/integrations/bi/hex.mdx @@ -6,6 +6,16 @@ badge: "Context Engine" After you connect Hex, Elementary will automatically and continuously extend the lineage to the project & cell level. This will provide you end-to-end data lineage to understand your downstream dependencies, called exposures. +### Prerequisites + +The Hex integration relies on BigQuery query history to discover which tables Hex projects query. For the integration to work, Elementary's service account must have **read access to `INFORMATION_SCHEMA.JOBS`** in the BigQuery project(s) where Hex executes queries. + +This requires the **BigQuery Resource Viewer** role (`roles/bigquery.resourceViewer`) granted at the **project level**. See [BigQuery permissions](/cloud/integrations/dwh/bigquery) for detailed setup instructions. + + +If Elementary cannot read query history, the Hex integration will not discover any exposures. Verify that the service account has the `bigquery.jobs.listAll` permission on the relevant GCP project(s). + + ### Create API Token Elementary needs a workspace token in your account in order to access Hex's API on your behalf.
@@ -20,6 +30,15 @@ Choose the Hex connection and provide the following details to validate and comp - **Base URL**: Your Hex workspace URL. For example: `https://app.hex.tech/my-workspace/` - **Workspace Token**: The Hex workspace token you've created on the previous step. +### Using a different BigQuery project for Hex queries + +By default, Elementary reads query history from the BigQuery project(s) referenced in your dbt project. If Hex executes queries through a **different GCP project** (e.g., a dedicated Hex execution project), you will need to: + +1. **Grant permissions on that project** — assign the **BigQuery Resource Viewer** and **BigQuery Metadata Viewer** roles to the Elementary service account at the project level for the Hex execution project (same as for your primary project). + +2. **Let us know** — contact Elementary support with the GCP project name so we can configure your environment to also read query history from that project. + +Without both steps, Elementary will not find any Hex queries in that project's `INFORMATION_SCHEMA.JOBS`, and no Hex exposures will appear. ### Limitations diff --git a/docs/snippets/cloud/integrations/permissions-and-security.mdx b/docs/snippets/cloud/integrations/permissions-and-security.mdx index ed4e58c23..f8bfe9c44 100644 --- a/docs/snippets/cloud/integrations/permissions-and-security.mdx +++ b/docs/snippets/cloud/integrations/permissions-and-security.mdx @@ -1,9 +1,13 @@ #### Permissions and security -Elementary cloud doesn't require read permissions to your tables and schemas, but only the following: +Elementary Cloud doesn't require read permissions to your tables and schemas, but only the following: -- Read-only access to the elementary schema. -- Access to read metadata in information schema and query history, related to the tables in your dbt project. +- Read-only access to the Elementary schema. +- Access to read metadata in `INFORMATION_SCHEMA` (table metadata and query history), related to the tables in your dbt project. + +For BigQuery, this translates to: +- **BigQuery Data Viewer** on the Elementary dataset (for reading Elementary schema data). +- **BigQuery Metadata Viewer** and **BigQuery Resource Viewer** at the **project level** (for reading `INFORMATION_SCHEMA.JOBS` query history and table metadata). These project-level roles are required for BI integrations like Hex that rely on query history. It is recommended to create a user using the instructions specified above to avoid granting excess privileges. For more details, refer to [security and privacy](/cloud/general/security-and-privacy). diff --git a/docs/snippets/dwh/bigquery/grant_user_access_on_dataset_level.mdx b/docs/snippets/dwh/bigquery/grant_user_access_on_dataset_level.mdx index 5c1a22d5e..fec0be6fb 100644 --- a/docs/snippets/dwh/bigquery/grant_user_access_on_dataset_level.mdx +++ b/docs/snippets/dwh/bigquery/grant_user_access_on_dataset_level.mdx @@ -1,15 +1,39 @@ -### Grant service user access to specific datasets +### Grant service user access -In order for a service user to work with Elementary cloud, it requires the following permissions: +In order for a service user to work with Elementary Cloud, it requires the following permissions: -- Role "BigQuery Data Viewer" on your Elementary dataset. -- Roles "BigQuery Metadata Viewer", "BigQuery Resource Viewer" on the entire project and any external sources it is referencing. +- Role **"BigQuery Data Viewer"** on your Elementary dataset. +- Roles **"BigQuery Metadata Viewer"** and **"BigQuery Resource Viewer"** at the **project level** for the project containing your dbt models and any external projects it references. -To grant a role on a specific dataset, follow these steps: + +"BigQuery Metadata Viewer" and "BigQuery Resource Viewer" **must be granted at the project level** (via IAM & Admin), not at the dataset level. These roles include permissions such as `bigquery.jobs.listAll` which are required for reading query history and metadata from `INFORMATION_SCHEMA` — they cannot be granted on individual datasets. + -1. Go to your project in [BigQuery console](https://console.cloud.google.com/bigquery) +#### Step 1: Grant project-level roles -2. In the "Explorer" tab, find your desired dataset. +These roles are required for Elementary to read query history (`INFORMATION_SCHEMA.JOBS`) and table metadata. Without them, BI integrations (such as Hex) that rely on query history will not function. + +1. Go to [IAM & Admin](https://console.cloud.google.com/iam-admin/iam) in the Google Cloud Console. + +2. Select the project that contains your dbt models. + +3. Click **"Grant Access"** at the top. + +4. Fill out the form: + - In the "New principals" field, enter the **email address** of the Elementary service account. + - Add the role **"BigQuery Metadata Viewer"** (`roles/bigquery.metadataViewer`). + - Click **"Add another role"** and add **"BigQuery Resource Viewer"** (`roles/bigquery.resourceViewer`). + - Click **"Save"**. + +5. **Repeat for any additional GCP projects** referenced in your dbt sources or used by BI tools (e.g., Hex, Looker) for query execution. + +#### Step 2: Grant dataset-level role on the Elementary dataset + +This role provides read access to your Elementary schema data. + +1. Go to your project in [BigQuery console](https://console.cloud.google.com/bigquery). + +2. In the "Explorer" tab, find your Elementary dataset. 3. Click on the three dots icon next to the dataset name, then Share. @@ -26,13 +50,11 @@ To grant a role on a specific dataset, follow these steps: /> 5. Fill out the form: - - In the "New principals" textbox, write the **email address** of your user. - - In the "Select a role" dropdown menu, choose the desired role (BigQuery Data Viewer for your Elementary dataset, BigQuery Metadata Viewer, BigQuery Resource Viewer for your dbt dataset). + - In the "New principals" textbox, write the **email address** of your service account. + - In the "Select a role" dropdown menu, choose **"BigQuery Data Viewer"**. - Click "Save". Grant access - -Make sure to grant the correct access to your Elementary dataset **and** your dbt dataset. From 5504bb6ac27cc54f9feee0c233200b290c7cf0ce Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Mon, 29 Jun 2026 17:49:02 +0000 Subject: [PATCH 2/5] docs: move BigQuery-specific text to BigQuery page, fix Hex role listing - Remove BigQuery-specific role details from shared permissions-and-security.mdx (was incorrectly shown on Snowflake, Redshift, Postgres, ClickHouse, Athena pages) - Add the BigQuery role details to bigquery.mdx instead (only shown on BigQuery page) - Fix Hex prerequisites to mention both required roles (Metadata Viewer + Resource Viewer) Co-Authored-By: Yosef Arbiv --- docs/cloud/integrations/bi/hex.mdx | 2 +- docs/snippets/cloud/integrations/bigquery.mdx | 4 ++++ docs/snippets/cloud/integrations/permissions-and-security.mdx | 4 ---- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/cloud/integrations/bi/hex.mdx b/docs/cloud/integrations/bi/hex.mdx index 94e765784..2ba192872 100644 --- a/docs/cloud/integrations/bi/hex.mdx +++ b/docs/cloud/integrations/bi/hex.mdx @@ -10,7 +10,7 @@ This will provide you end-to-end data lineage to understand your downstream depe The Hex integration relies on BigQuery query history to discover which tables Hex projects query. For the integration to work, Elementary's service account must have **read access to `INFORMATION_SCHEMA.JOBS`** in the BigQuery project(s) where Hex executes queries. -This requires the **BigQuery Resource Viewer** role (`roles/bigquery.resourceViewer`) granted at the **project level**. See [BigQuery permissions](/cloud/integrations/dwh/bigquery) for detailed setup instructions. +This requires the **BigQuery Metadata Viewer** (`roles/bigquery.metadataViewer`) and **BigQuery Resource Viewer** (`roles/bigquery.resourceViewer`) roles granted at the **project level**. See [BigQuery permissions](/cloud/integrations/dwh/bigquery) for detailed setup instructions. If Elementary cannot read query history, the Hex integration will not discover any exposures. Verify that the service account has the `bigquery.jobs.listAll` permission on the relevant GCP project(s). diff --git a/docs/snippets/cloud/integrations/bigquery.mdx b/docs/snippets/cloud/integrations/bigquery.mdx index 3be5e6381..a3d661837 100644 --- a/docs/snippets/cloud/integrations/bigquery.mdx +++ b/docs/snippets/cloud/integrations/bigquery.mdx @@ -25,6 +25,10 @@ Select a tab below and follow the steps for your chosen method. +For BigQuery, this translates to: +- **BigQuery Data Viewer** on the Elementary dataset (for reading Elementary schema data). +- **BigQuery Metadata Viewer** and **BigQuery Resource Viewer** at the **project level** (for reading `INFORMATION_SCHEMA.JOBS` query history and table metadata). These project-level roles are required for BI integrations like Hex that rely on query history. + ### Fill the connection form Use the **Authentication method** toggle at the top of the form to select either **Service account** or **Workload Identity Federation**, matching the method you set up above. The credentials upload field changes based on your selection: diff --git a/docs/snippets/cloud/integrations/permissions-and-security.mdx b/docs/snippets/cloud/integrations/permissions-and-security.mdx index f8bfe9c44..57bafc763 100644 --- a/docs/snippets/cloud/integrations/permissions-and-security.mdx +++ b/docs/snippets/cloud/integrations/permissions-and-security.mdx @@ -5,9 +5,5 @@ Elementary Cloud doesn't require read permissions to your tables and schemas, bu - Read-only access to the Elementary schema. - Access to read metadata in `INFORMATION_SCHEMA` (table metadata and query history), related to the tables in your dbt project. -For BigQuery, this translates to: -- **BigQuery Data Viewer** on the Elementary dataset (for reading Elementary schema data). -- **BigQuery Metadata Viewer** and **BigQuery Resource Viewer** at the **project level** (for reading `INFORMATION_SCHEMA.JOBS` query history and table metadata). These project-level roles are required for BI integrations like Hex that rely on query history. - It is recommended to create a user using the instructions specified above to avoid granting excess privileges. For more details, refer to [security and privacy](/cloud/general/security-and-privacy). From 7f4186f9784d1c7c5bdb5de5a0e6fd06b51d8d7a Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Mon, 29 Jun 2026 20:06:58 +0000 Subject: [PATCH 3/5] docs: make Hex page DWH-agnostic instead of BigQuery-specific - Rewrite prerequisites to reference query history generically with links to BigQuery, Snowflake, and Databricks permissions pages - Rename 'Using a different BigQuery project' to 'Hex queries on a different warehouse project' with generic language Co-Authored-By: Yosef Arbiv --- docs/cloud/integrations/bi/hex.mdx | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/docs/cloud/integrations/bi/hex.mdx b/docs/cloud/integrations/bi/hex.mdx index 2ba192872..7a6f6fecf 100644 --- a/docs/cloud/integrations/bi/hex.mdx +++ b/docs/cloud/integrations/bi/hex.mdx @@ -8,12 +8,15 @@ This will provide you end-to-end data lineage to understand your downstream depe ### Prerequisites -The Hex integration relies on BigQuery query history to discover which tables Hex projects query. For the integration to work, Elementary's service account must have **read access to `INFORMATION_SCHEMA.JOBS`** in the BigQuery project(s) where Hex executes queries. +The Hex integration relies on **query history** to discover which tables Hex projects query. Elementary's service account must have permissions to read query history from the data warehouse where Hex executes queries. -This requires the **BigQuery Metadata Viewer** (`roles/bigquery.metadataViewer`) and **BigQuery Resource Viewer** (`roles/bigquery.resourceViewer`) roles granted at the **project level**. See [BigQuery permissions](/cloud/integrations/dwh/bigquery) for detailed setup instructions. +Refer to the permissions section in your warehouse's setup page for the required roles: +- [BigQuery permissions](/cloud/integrations/dwh/bigquery#grant-service-user-access) +- [Snowflake permissions](/cloud/integrations/dwh/snowflake) +- [Databricks permissions](/cloud/integrations/dwh/databricks) -If Elementary cannot read query history, the Hex integration will not discover any exposures. Verify that the service account has the `bigquery.jobs.listAll` permission on the relevant GCP project(s). +If Elementary cannot read query history, the Hex integration will not discover any exposures. ### Create API Token @@ -30,15 +33,15 @@ Choose the Hex connection and provide the following details to validate and comp - **Base URL**: Your Hex workspace URL. For example: `https://app.hex.tech/my-workspace/` - **Workspace Token**: The Hex workspace token you've created on the previous step. -### Using a different BigQuery project for Hex queries +### Hex queries on a different warehouse project -By default, Elementary reads query history from the BigQuery project(s) referenced in your dbt project. If Hex executes queries through a **different GCP project** (e.g., a dedicated Hex execution project), you will need to: +By default, Elementary reads query history from the warehouse projects and databases referenced in your dbt project. If Hex executes queries through a **different project or database** (e.g., a dedicated Hex execution project), you will need to: -1. **Grant permissions on that project** — assign the **BigQuery Resource Viewer** and **BigQuery Metadata Viewer** roles to the Elementary service account at the project level for the Hex execution project (same as for your primary project). +1. **Grant query history permissions on that project** — the same permissions required for your primary project. See the [prerequisites](#prerequisites) above for links to your warehouse's permissions page. -2. **Let us know** — contact Elementary support with the GCP project name so we can configure your environment to also read query history from that project. +2. **Let us know** — contact Elementary support with the project or database name so we can configure your environment to also read query history from it. -Without both steps, Elementary will not find any Hex queries in that project's `INFORMATION_SCHEMA.JOBS`, and no Hex exposures will appear. +Without both steps, Elementary will not find any Hex queries in that project and no Hex exposures will appear. ### Limitations From eeb637b5b425db840cccd0e2254d7ddc69b2c3d2 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Tue, 30 Jun 2026 08:32:39 +0000 Subject: [PATCH 4/5] docs: replace 'project' with DWH-agnostic terminology on Hex page Use 'environment' instead of 'project' since not all warehouses use the term 'project' (e.g., Snowflake has accounts, Databricks has workspaces). Co-Authored-By: Yosef Arbiv --- docs/cloud/integrations/bi/hex.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/cloud/integrations/bi/hex.mdx b/docs/cloud/integrations/bi/hex.mdx index 7a6f6fecf..cb9b9076a 100644 --- a/docs/cloud/integrations/bi/hex.mdx +++ b/docs/cloud/integrations/bi/hex.mdx @@ -33,15 +33,15 @@ Choose the Hex connection and provide the following details to validate and comp - **Base URL**: Your Hex workspace URL. For example: `https://app.hex.tech/my-workspace/` - **Workspace Token**: The Hex workspace token you've created on the previous step. -### Hex queries on a different warehouse project +### Hex queries on a different warehouse environment -By default, Elementary reads query history from the warehouse projects and databases referenced in your dbt project. If Hex executes queries through a **different project or database** (e.g., a dedicated Hex execution project), you will need to: +By default, Elementary reads query history from the warehouse environments (e.g., BigQuery projects, Snowflake accounts, Databricks workspaces) referenced in your dbt project. If Hex executes queries through a **different environment** (e.g., a dedicated Hex execution environment), you will need to: -1. **Grant query history permissions on that project** — the same permissions required for your primary project. See the [prerequisites](#prerequisites) above for links to your warehouse's permissions page. +1. **Grant query history permissions** — the same permissions required for your primary environment. See the [prerequisites](#prerequisites) above for links to your warehouse's permissions page. -2. **Let us know** — contact Elementary support with the project or database name so we can configure your environment to also read query history from it. +2. **Let us know** — contact Elementary support with the environment details so we can configure Elementary to also read query history from it. -Without both steps, Elementary will not find any Hex queries in that project and no Hex exposures will appear. +Without both steps, Elementary will not find any Hex queries in that environment and no Hex exposures will appear. ### Limitations From 505d5d89af4ccc3bcdaa2577d826856ba166ce13 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Tue, 30 Jun 2026 12:45:59 +0000 Subject: [PATCH 5/5] docs: list only BigQuery and Snowflake as supported DWHs for Hex Hex query history filters only exist for BigQuery and Snowflake. Removed Databricks from the prerequisites links and added explicit 'supported on BigQuery and Snowflake' statement. Co-Authored-By: Yosef Arbiv --- docs/cloud/integrations/bi/hex.mdx | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/cloud/integrations/bi/hex.mdx b/docs/cloud/integrations/bi/hex.mdx index cb9b9076a..84a560755 100644 --- a/docs/cloud/integrations/bi/hex.mdx +++ b/docs/cloud/integrations/bi/hex.mdx @@ -8,12 +8,11 @@ This will provide you end-to-end data lineage to understand your downstream depe ### Prerequisites -The Hex integration relies on **query history** to discover which tables Hex projects query. Elementary's service account must have permissions to read query history from the data warehouse where Hex executes queries. +The Hex integration relies on **query history** to discover which tables Hex projects query. It is supported on **BigQuery** and **Snowflake**. -Refer to the permissions section in your warehouse's setup page for the required roles: +Elementary's service account must have permissions to read query history. Refer to your warehouse's setup page for the required roles: - [BigQuery permissions](/cloud/integrations/dwh/bigquery#grant-service-user-access) - [Snowflake permissions](/cloud/integrations/dwh/snowflake) -- [Databricks permissions](/cloud/integrations/dwh/databricks) If Elementary cannot read query history, the Hex integration will not discover any exposures. @@ -35,7 +34,7 @@ Choose the Hex connection and provide the following details to validate and comp ### Hex queries on a different warehouse environment -By default, Elementary reads query history from the warehouse environments (e.g., BigQuery projects, Snowflake accounts, Databricks workspaces) referenced in your dbt project. If Hex executes queries through a **different environment** (e.g., a dedicated Hex execution environment), you will need to: +By default, Elementary reads query history from the warehouse environments (e.g., BigQuery projects, Snowflake accounts) referenced in your dbt project. If Hex executes queries through a **different environment** (e.g., a dedicated Hex execution environment), you will need to: 1. **Grant query history permissions** — the same permissions required for your primary environment. See the [prerequisites](#prerequisites) above for links to your warehouse's permissions page.