From 2c08634d496bf26bd2c99013f07b257778ce45d7 Mon Sep 17 00:00:00 2001
From: Andrey Fedorov <andrey.fedorov@gmail.com>
Date: Thu, 30 Jul 2026 09:18:50 -0400
Subject: [PATCH] add notebooks to explore related external data

---
 .../advanced_topics/clinical_data_intro.ipynb | 11857 ++++++++--------
 .../cddp_eagle_gdc_pathology_reports.ipynb    |   451 +
 .../gtex_pathology_annotations.ipynb          |   330 +
 .../tcga_gdc_pathology_reports.ipynb          |   470 +
 4 files changed, 7179 insertions(+), 5929 deletions(-)
 create mode 100644 notebooks/advanced_topics/related_external_data/cddp_eagle_gdc_pathology_reports.ipynb
 create mode 100644 notebooks/advanced_topics/related_external_data/gtex_pathology_annotations.ipynb
 create mode 100644 notebooks/advanced_topics/related_external_data/tcga_gdc_pathology_reports.ipynb

diff --git a/notebooks/advanced_topics/clinical_data_intro.ipynb b/notebooks/advanced_topics/clinical_data_intro.ipynb
index 0e20a95..9a1c3ac 100644
--- a/notebooks/advanced_topics/clinical_data_intro.ipynb
+++ b/notebooks/advanced_topics/clinical_data_intro.ipynb
@@ -1,6001 +1,6000 @@
 {
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "view-in-github",
-    "colab_type": "text"
-   },
-   "source": [
-    "<a href=\"https://colab.research.google.com/github/ImagingDataCommons/IDC-Tutorials/blob/master/notebooks/advanced_topics/clinical_data_intro.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "m_TcNSmCPUVC"
-   },
-   "source": [
-    "# IDC clinical data exploration\n",
-    "\n",
-    "The goal of this notebook is to introduce the users of NCI Imaging Data Commons (IDC) to the organization of clinical data that accompany some of the IDC imaging data. For a quick visual summary of the clinical data available in IDC, please check out [this DataStudio dashboard](https://datastudio.google.com/u/0/reporting/04cf5976-4ea0-4fee-a749-8bfd162f2e87/page/p_s7mk6eybqc).\n",
-    "\n",
-    "[NCI Imaging Data Commons (IDC)](https://imaging.datacommons.cancer.gov) is a cloud-based environment containing publicly available cancer imaging data co-located with analysis and exploration tools and resources. IDC is a node within the broader NCI Cancer Research Data Commons (CRDC) infrastructure that provides secure access to a large, comprehensive, and expanding collection of cancer research data.\n",
-    "\n",
-    "If you are not familiar with IDC, we recommend you first take a look at the [Getting started](https://github.com/ImagingDataCommons/IDC-Tutorials/blob/master/notebooks/getting_started/) notebooks that are intended to serve as the introduction into working with IDC programmatically.\n",
-    "\n",
-    "If you have any questions about this tutorial, please post your questions on the [IDC user forum](https://discourse.canceridc.dev/) (preferred) or email IDC support at support@canceridc.dev!\n",
-    "\n",
-    "Authored by Andrey Fedorov and George White\n",
-    "\n",
-    "Prepared: July 2022\n",
-    "\n",
-    "Updated: Feb 2026"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "40SRMJpLRbx3"
-   },
-   "source": [
-    "# Prerequisites\n",
-    "\n",
-    "The only prerequisite is [`idc-index`](https://github.com/ImagingDataCommons/idc-index) - python package that contains various utilities to simplify access to IDC data."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "source": [
-    "%%capture\n",
-    "!pip install --upgrade idc-index"
-   ],
-   "metadata": {
-    "id": "QZMwa6Fg7QTm"
-   },
-   "execution_count": 41,
-   "outputs": []
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "P7BDaMjXykCZ"
-   },
-   "source": [
-    "# Clinical data - background\n",
-    "\n",
-    "By clinical data we refer to the broad spectrum of image-related data that may accompany images. Such data may include demographics of the patients, observations related to their clinical history (therapies, diagnoses, findings), lab tests, surgeries.\n",
-    "\n",
-    "Clinical data is often critical in understanding imaging data, and is essential for the development and validation of imaging biomarkers. However, such data is most often stored in spreadsheets that follow conventions specific to the site that collected the data, may not be accompanied by the dictionary defining the terms used in describing clinical data, and is rarely harmonized. As an example, you can consider examing the clinical data that accompanies the [ACRIN 6698 collection](https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=50135447) curated by The Cancer Imaging Archive (TCIA). File named \"Full Ancillary Patient Information file.xlsx\" linked from the collection page contains two sheets, one of which is the dictionary, and the othe one contains per-patient data with the columns defined by the dictionary.\n",
-    "\n",
-    "Not only the terms used in the clinical data accompanying individual collection are not harmonized, but the format of the spreadsheets is also collection-specific. In order to search and navigate clinical data, one has to parse those collection specific tables, and there is no interface to support searching across collections.\n",
-    "\n",
-    "With the release v11 of IDC, we make the attempt to lower the barriers for accessing clinical data accompanying IDC imaging collections. We parse collection-specific tables, and organize the underlying data into BigQuery tables that can be accessed using standard SQL queries. You can also see the summary of clinical data available for IDC collections in [this dashboard](https://datastudio.google.com/u/0/reporting/04cf5976-4ea0-4fee-a749-8bfd162f2e87/page/p_s7mk6eybqc). Further, we make the content of the extracted clinical data available via the [`idc-index`](https://github.com/ImagingDataCommons/idc-index) python package.\n",
-    "\n",
-    "At the completion of this tutorial you will learn how IDC clinical data is organized, and how to write queries to interrogate this data."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "29H1Kp5dTYBb"
-   },
-   "source": [
-    "# Organization of clinical data in IDC\n",
-    "\n",
-    "`idc-index` packages _indices_ - tables containing key metadata describing data available in IDC. The main index that supports API calls related to download and search is installed by default. To support search of the clinical data accompanying IDC images you will need the `clinical_index` table, which helps navigating clinical data accompanying images."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {
-    "id": "TLhEHQkNWkac",
-    "outputId": "344eca93-377c-48cf-e35f-2fd79dd7de4f",
-    "colab": {
-     "base_uri": "https://localhost:8080/"
-    }
-   },
-   "outputs": [
+  "cells": [
     {
-     "output_type": "stream",
-     "name": "stdout",
-     "text": [
-      "Columns avaialable in clinical_index:\n",
-      "collection_id\n",
-      "table_name\n",
-      "short_table_name\n",
-      "column\n",
-      "column_label\n",
-      "values\n"
-     ]
-    }
-   ],
-   "source": [
-    "from idc_index import IDCClient\n",
-    "\n",
-    "c = IDCClient()\n",
-    "\n",
-    "c.fetch_index('clinical_index')\n",
-    "\n",
-    "print('Columns avaialable in clinical_index:\\n'+'\\n'.join(c.clinical_index.keys()))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "B9Vgq-gtI9G4"
-   },
-   "source": [
-    "This table is documented in https://idc-index.readthedocs.io/en/latest/column_descriptions.html#clinical-index.\n",
-    "\n",
-    "\n",
-    "Accessing relevant clinical data for a given collection is a two-step process:\n",
-    "\n",
-    "* Step 1: use `clinical_index` table to identify relevant metadata attributes and the names of the tables where the corresponding metadata is located.\n",
-    "* Step 2: load the specific clinical data table with the selected attribute referenced from `clinical_index` and access clinical metadata for the individual patients.\n",
-    "\n",
-    "As a reminder, all of the data - including clinical metadata - in IDC is anonymized!\n",
-    "\n",
-    "For the sake of this example, we will start with identifying clinical data attributes that accompany NLST collection.\n",
-    "\n",
-    "`collection_id` column of `clinical_index` can be used to associate clinical data attribute to the collection it accompanies."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "## Understanding individual clinical data attributes\n",
-    "\n",
-    "Here's an example of the columns that correspond to the clinical metadata in one of the ACRIN collections, where `column` is not particularly helpful, but `column_label` provides human readable information to allow interpretation of the column."
-   ],
-   "metadata": {
-    "id": "CM-vI0FJClLy"
-   }
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "metadata": {
-    "id": "nGsS6SvPdrWn",
-    "outputId": "2f888f38-daff-410f-d150-96a7575b1265",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 653
-    }
-   },
-   "outputs": [
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "view-in-github",
+        "colab_type": "text"
+      },
+      "source": [
+        "<a href=\"https://colab.research.google.com/github/ImagingDataCommons/IDC-Tutorials/blob/master/notebooks/advanced_topics/clinical_data_intro.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+      ]
+    },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "m_TcNSmCPUVC"
+      },
+      "source": [
+        "# IDC clinical data exploration\n",
+        "\n",
+        "The goal of this notebook is to introduce the users of NCI Imaging Data Commons (IDC) to the organization of clinical data that accompany some of the IDC imaging data. For a quick visual summary of the clinical data available in IDC, please check out [this DataStudio dashboard](https://datastudio.google.com/u/0/reporting/04cf5976-4ea0-4fee-a749-8bfd162f2e87/page/p_s7mk6eybqc).\n",
+        "\n",
+        "[NCI Imaging Data Commons (IDC)](https://imaging.datacommons.cancer.gov) is a cloud-based environment containing publicly available cancer imaging data co-located with analysis and exploration tools and resources. IDC is a node within the broader NCI Cancer Research Data Commons (CRDC) infrastructure that provides secure access to a large, comprehensive, and expanding collection of cancer research data.\n",
+        "\n",
+        "If you are not familiar with IDC, we recommend you first take a look at the [Getting started](https://github.com/ImagingDataCommons/IDC-Tutorials/blob/master/notebooks/getting_started/) notebooks that are intended to serve as the introduction into working with IDC programmatically.\n",
+        "\n",
+        "If you have any questions about this tutorial, please post your questions on the [IDC user forum](https://discourse.canceridc.dev/) (preferred) or email IDC support at support@canceridc.dev!\n",
+        "\n",
+        "Authored by Andrey Fedorov and George White\n",
+        "\n",
+        "Prepared: July 2022\n",
+        "\n",
+        "Updated: Jan 2025"
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "            collection_id                                   short_table_name  \\\n",
-       "2359  acrin_nsclc_fdg_pet                             acrin_nsclc_fdg_pet_A0   \n",
-       "2360  acrin_nsclc_fdg_pet                             acrin_nsclc_fdg_pet_A0   \n",
-       "2361  acrin_nsclc_fdg_pet                             acrin_nsclc_fdg_pet_A0   \n",
-       "2362  acrin_nsclc_fdg_pet                             acrin_nsclc_fdg_pet_A0   \n",
-       "2363  acrin_nsclc_fdg_pet                             acrin_nsclc_fdg_pet_A0   \n",
-       "...                   ...                                                ...   \n",
-       "3643  acrin_nsclc_fdg_pet  acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation   \n",
-       "3644  acrin_nsclc_fdg_pet  acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation   \n",
-       "3645  acrin_nsclc_fdg_pet  acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation   \n",
-       "3646  acrin_nsclc_fdg_pet  acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation   \n",
-       "3647  acrin_nsclc_fdg_pet  acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation   \n",
-       "\n",
-       "                           column  \\\n",
-       "2359                      inst_no   \n",
-       "2360                     entryage   \n",
-       "2361                        a0e26   \n",
-       "2362                          rec   \n",
-       "2363                        a0e4d   \n",
-       "...                           ...   \n",
-       "3643             studyinstanceuid   \n",
-       "3644          ptseriesinstanceuid   \n",
-       "3645          ctseriesinstanceuid   \n",
-       "3646               aisegmentation   \n",
-       "3647  commentsaboutaisegmentation   \n",
-       "\n",
-       "                                           column_label  \\\n",
-       "2359                   De-identified Institution Number   \n",
-       "2360                        Patient age at Registration   \n",
-       "2361                            RTOG INSTITUTION NUMBER   \n",
-       "2362                      Data receipt (from base date)   \n",
-       "2363  \"Days from Base_dt to DATE THE  STUDY-SPECIFIC...   \n",
-       "...                                                 ...   \n",
-       "3643                                   StudyInstanceUID   \n",
-       "3644                                PTSeriesInstanceUID   \n",
-       "3645                                CTSeriesInstanceUID   \n",
-       "3646                                     AISegmentation   \n",
-       "3647                        CommentsAboutAISegmentation   \n",
-       "\n",
-       "                                                 values  \n",
-       "2359  [{'option_code': '\"nan\"', 'option_description'...  \n",
-       "2360  [{'option_code': '\"nan\"', 'option_description'...  \n",
-       "2361  [{'option_code': '\"nan\"', 'option_description'...  \n",
-       "2362  [{'option_code': '\"nan\"', 'option_description'...  \n",
-       "2363  [{'option_code': '\"nan\"', 'option_description'...  \n",
-       "...                                                 ...  \n",
-       "3643                                                 []  \n",
-       "3644                                                 []  \n",
-       "3645                                                 []  \n",
-       "3646                                                 []  \n",
-       "3647                                                 []  \n",
-       "\n",
-       "[1289 rows x 5 columns]"
-      ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-c19868ab-06c0-48fd-8619-1f65ede862ed\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>collection_id</th>\n",
-       "      <th>short_table_name</th>\n",
-       "      <th>column</th>\n",
-       "      <th>column_label</th>\n",
-       "      <th>values</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>2359</th>\n",
-       "      <td>acrin_nsclc_fdg_pet</td>\n",
-       "      <td>acrin_nsclc_fdg_pet_A0</td>\n",
-       "      <td>inst_no</td>\n",
-       "      <td>De-identified Institution Number</td>\n",
-       "      <td>[{'option_code': '\"nan\"', 'option_description'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2360</th>\n",
-       "      <td>acrin_nsclc_fdg_pet</td>\n",
-       "      <td>acrin_nsclc_fdg_pet_A0</td>\n",
-       "      <td>entryage</td>\n",
-       "      <td>Patient age at Registration</td>\n",
-       "      <td>[{'option_code': '\"nan\"', 'option_description'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2361</th>\n",
-       "      <td>acrin_nsclc_fdg_pet</td>\n",
-       "      <td>acrin_nsclc_fdg_pet_A0</td>\n",
-       "      <td>a0e26</td>\n",
-       "      <td>RTOG INSTITUTION NUMBER</td>\n",
-       "      <td>[{'option_code': '\"nan\"', 'option_description'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2362</th>\n",
-       "      <td>acrin_nsclc_fdg_pet</td>\n",
-       "      <td>acrin_nsclc_fdg_pet_A0</td>\n",
-       "      <td>rec</td>\n",
-       "      <td>Data receipt (from base date)</td>\n",
-       "      <td>[{'option_code': '\"nan\"', 'option_description'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2363</th>\n",
-       "      <td>acrin_nsclc_fdg_pet</td>\n",
-       "      <td>acrin_nsclc_fdg_pet_A0</td>\n",
-       "      <td>a0e4d</td>\n",
-       "      <td>\"Days from Base_dt to DATE THE  STUDY-SPECIFIC...</td>\n",
-       "      <td>[{'option_code': '\"nan\"', 'option_description'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>...</th>\n",
-       "      <td>...</td>\n",
-       "      <td>...</td>\n",
-       "      <td>...</td>\n",
-       "      <td>...</td>\n",
-       "      <td>...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3643</th>\n",
-       "      <td>acrin_nsclc_fdg_pet</td>\n",
-       "      <td>acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation</td>\n",
-       "      <td>studyinstanceuid</td>\n",
-       "      <td>StudyInstanceUID</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3644</th>\n",
-       "      <td>acrin_nsclc_fdg_pet</td>\n",
-       "      <td>acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation</td>\n",
-       "      <td>ptseriesinstanceuid</td>\n",
-       "      <td>PTSeriesInstanceUID</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3645</th>\n",
-       "      <td>acrin_nsclc_fdg_pet</td>\n",
-       "      <td>acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation</td>\n",
-       "      <td>ctseriesinstanceuid</td>\n",
-       "      <td>CTSeriesInstanceUID</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3646</th>\n",
-       "      <td>acrin_nsclc_fdg_pet</td>\n",
-       "      <td>acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation</td>\n",
-       "      <td>aisegmentation</td>\n",
-       "      <td>AISegmentation</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3647</th>\n",
-       "      <td>acrin_nsclc_fdg_pet</td>\n",
-       "      <td>acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation</td>\n",
-       "      <td>commentsaboutaisegmentation</td>\n",
-       "      <td>CommentsAboutAISegmentation</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "<p>1289 rows × 5 columns</p>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c19868ab-06c0-48fd-8619-1f65ede862ed')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-c19868ab-06c0-48fd-8619-1f65ede862ed button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-c19868ab-06c0-48fd-8619-1f65ede862ed');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "<div id=\"df-eaa84ef7-34b8-443b-a32e-051b4e5b2093\">\n",
-       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-eaa84ef7-34b8-443b-a32e-051b4e5b2093')\"\n",
-       "            title=\"Suggest charts\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "     width=\"24px\">\n",
-       "    <g>\n",
-       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
-       "    </g>\n",
-       "</svg>\n",
-       "  </button>\n",
-       "\n",
-       "<style>\n",
-       "  .colab-df-quickchart {\n",
-       "      --bg-color: #E8F0FE;\n",
-       "      --fill-color: #1967D2;\n",
-       "      --hover-bg-color: #E2EBFA;\n",
-       "      --hover-fill-color: #174EA6;\n",
-       "      --disabled-fill-color: #AAA;\n",
-       "      --disabled-bg-color: #DDD;\n",
-       "  }\n",
-       "\n",
-       "  [theme=dark] .colab-df-quickchart {\n",
-       "      --bg-color: #3B4455;\n",
-       "      --fill-color: #D2E3FC;\n",
-       "      --hover-bg-color: #434B5C;\n",
-       "      --hover-fill-color: #FFFFFF;\n",
-       "      --disabled-bg-color: #3B4455;\n",
-       "      --disabled-fill-color: #666;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart {\n",
-       "    background-color: var(--bg-color);\n",
-       "    border: none;\n",
-       "    border-radius: 50%;\n",
-       "    cursor: pointer;\n",
-       "    display: none;\n",
-       "    fill: var(--fill-color);\n",
-       "    height: 32px;\n",
-       "    padding: 0;\n",
-       "    width: 32px;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart:hover {\n",
-       "    background-color: var(--hover-bg-color);\n",
-       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "    fill: var(--button-hover-fill-color);\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart-complete:disabled,\n",
-       "  .colab-df-quickchart-complete:disabled:hover {\n",
-       "    background-color: var(--disabled-bg-color);\n",
-       "    fill: var(--disabled-fill-color);\n",
-       "    box-shadow: none;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-spinner {\n",
-       "    border: 2px solid var(--fill-color);\n",
-       "    border-color: transparent;\n",
-       "    border-bottom-color: var(--fill-color);\n",
-       "    animation:\n",
-       "      spin 1s steps(1) infinite;\n",
-       "  }\n",
-       "\n",
-       "  @keyframes spin {\n",
-       "    0% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "      border-left-color: var(--fill-color);\n",
-       "    }\n",
-       "    20% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    30% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    40% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    60% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    80% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "    90% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "  }\n",
-       "</style>\n",
-       "\n",
-       "  <script>\n",
-       "    async function quickchart(key) {\n",
-       "      const quickchartButtonEl =\n",
-       "        document.querySelector('#' + key + ' button');\n",
-       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
-       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
-       "      try {\n",
-       "        const charts = await google.colab.kernel.invokeFunction(\n",
-       "            'suggestCharts', [key], {});\n",
-       "      } catch (error) {\n",
-       "        console.error('Error during call to suggestCharts:', error);\n",
-       "      }\n",
-       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
-       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
-       "    }\n",
-       "    (() => {\n",
-       "      let quickchartButtonEl =\n",
-       "        document.querySelector('#df-eaa84ef7-34b8-443b-a32e-051b4e5b2093 button');\n",
-       "      quickchartButtonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "    })();\n",
-       "  </script>\n",
-       "</div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "40SRMJpLRbx3"
+      },
+      "source": [
+        "# Prerequisites\n",
+        "\n",
+        "The only prerequisite is [`idc-index`](https://github.com/ImagingDataCommons/idc-index) - python package that contains various utilities to simplify access to IDC data."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "%%capture\n",
+        "!pip install --upgrade idc-index"
       ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "summary": "{\n  \"name\": \"acrin_nsclc_fdg_pet_clinical_columns[['collection_id','short_table_name','column','column_label','values']]\",\n  \"rows\": 1289,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"acrin_nsclc_fdg_pet\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"short_table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 21,\n        \"samples\": [\n          \"acrin_nsclc_fdg_pet_A0\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1211,\n        \"samples\": [\n          \"aee34\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 789,\n        \"samples\": [\n          \"AREAS OF INTEREST WITHIN MRI SCAN - CHEST - IMAGING DATE\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
-      }
-     },
-     "metadata": {},
-     "execution_count": 5
-    }
-   ],
-   "source": [
-    "acrin_nsclc_fdg_pet_clinical_columns = c.clinical_index[c.clinical_index['collection_id']=='acrin_nsclc_fdg_pet']\n",
-    "acrin_nsclc_fdg_pet_clinical_columns[['collection_id','short_table_name','column','column_label','values']]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "m0H78L-VdxHn"
-   },
-   "source": [
-    "For some columns, the values come from a defined set. In the example above, we can, for example, examine the values assigned to encode patient race."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "metadata": {
-    "id": "9NCP15tRd2gU",
-    "outputId": "b92fdc9a-7f19-406b-d885-0d27250154c5",
-    "colab": {
-     "base_uri": "https://localhost:8080/"
-    }
-   },
-   "outputs": [
+      "metadata": {
+        "id": "QZMwa6Fg7QTm"
+      },
+      "execution_count": 41,
+      "outputs": []
+    },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "[array([{'option_code': '1.0', 'option_description': 'American Indian or Alaska Native'},\n",
-       "        {'option_code': '2.0', 'option_description': 'Asian'},\n",
-       "        {'option_code': '3.0', 'option_description': 'Black or African American'},\n",
-       "        {'option_code': '4.0', 'option_description': 'Native Hawaiian or other Pacific Islander'},\n",
-       "        {'option_code': '5.0', 'option_description': 'White'},\n",
-       "        {'option_code': '6.0', 'option_description': 'More than one race'},\n",
-       "        {'option_code': '9.0', 'option_description': 'Unknown'}],\n",
-       "       dtype=object)]"
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "P7BDaMjXykCZ"
+      },
+      "source": [
+        "# Clinical data - background\n",
+        "\n",
+        "By clinical data we refer to the broad spectrum of image-related data that may accompany images. Such data may include demographics of the patients, observations related to their clinical history (therapies, diagnoses, findings), lab tests, surgeries.\n",
+        "\n",
+        "Clinical data is often critical in understanding imaging data, and is essential for the development and validation of imaging biomarkers. However, such data is most often stored in spreadsheets that follow conventions specific to the site that collected the data, may not be accompanied by the dictionary defining the terms used in describing clinical data, and is rarely harmonized. As an example, you can consider examing the clinical data that accompanies the [ACRIN 6698 collection](https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=50135447) curated by The Cancer Imaging Archive (TCIA). File named \"Full Ancillary Patient Information file.xlsx\" linked from the collection page contains two sheets, one of which is the dictionary, and the othe one contains per-patient data with the columns defined by the dictionary.\n",
+        "\n",
+        "Not only the terms used in the clinical data accompanying individual collection are not harmonized, but the format of the spreadsheets is also collection-specific. In order to search and navigate clinical data, one has to parse those collection specific tables, and there is no interface to support searching across collections.\n",
+        "\n",
+        "With the release v11 of IDC, we make the attempt to lower the barriers for accessing clinical data accompanying IDC imaging collections. We parse collection-specific tables, and organize the underlying data into BigQuery tables that can be accessed using standard SQL queries. You can also see the summary of clinical data available for IDC collections in [this dashboard](https://datastudio.google.com/u/0/reporting/04cf5976-4ea0-4fee-a749-8bfd162f2e87/page/p_s7mk6eybqc). Further, we make the content of the extracted clinical data available via the [`idc-index`](https://github.com/ImagingDataCommons/idc-index) python package.\n",
+        "\n",
+        "At the completion of this tutorial you will learn how IDC clinical data is organized, and how to write queries to interrogate this data."
       ]
-     },
-     "metadata": {},
-     "execution_count": 7
-    }
-   ],
-   "source": [
-    "race_values = acrin_nsclc_fdg_pet_clinical_columns[acrin_nsclc_fdg_pet_clinical_columns[\"column_label\"] == \"RACE\"][\"values\"]\n",
-    "\n",
-    "race_values.tolist()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "yaoJvf2nd6wL"
-   },
-   "source": [
-    "On the other hand, if we look at the metadata available for the `c4kc_kits` collection, `column_label` and `column` are identical.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "metadata": {
-    "id": "AYMJ8kjTd9eb",
-    "outputId": "1ce1396b-66f6-4b22-e372-c3848e5a52e6",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 303
-    }
-   },
-   "outputs": [
+    },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "29H1Kp5dTYBb"
+      },
+      "source": [
+        "# Organization of clinical data in IDC\n",
+        "\n",
+        "`idc-index` packages _indices_ - tables containing key metadata describing data available in IDC. The main index that supports API calls related to download and search is installed by default. To support search of the clinical data accompanying IDC images you will need the `clinical_index` table, which helps navigating clinical data accompanying images."
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "     collection_id                                         table_name  \\\n",
-       "3828     c4kc_kits  bigquery-public-data.idc_v20_clinical.c4kc_kit...   \n",
-       "3829     c4kc_kits  bigquery-public-data.idc_v20_clinical.c4kc_kit...   \n",
-       "3830     c4kc_kits  bigquery-public-data.idc_v20_clinical.c4kc_kit...   \n",
-       "\n",
-       "        short_table_name                                       column  \\\n",
-       "3828  c4kc_kits_clinical                      comorbidities__dementia   \n",
-       "3829  c4kc_kits_clinical                          comorbidities__aids   \n",
-       "3830  c4kc_kits_clinical  intraoperative_complications__cardiac_event   \n",
-       "\n",
-       "                                     column_label  \\\n",
-       "3828                      comorbidities__dementia   \n",
-       "3829                          comorbidities__aids   \n",
-       "3830  intraoperative_complications__cardiac_event   \n",
-       "\n",
-       "                                                 values  \n",
-       "3828  [{'option_code': 'False', 'option_description'...  \n",
-       "3829  [{'option_code': 'False', 'option_description'...  \n",
-       "3830  [{'option_code': 'False', 'option_description'...  "
+      "cell_type": "code",
+      "execution_count": 2,
+      "metadata": {
+        "id": "TLhEHQkNWkac",
+        "outputId": "344eca93-377c-48cf-e35f-2fd79dd7de4f",
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stdout",
+          "text": [
+            "Columns avaialable in clinical_index:\n",
+            "collection_id\n",
+            "table_name\n",
+            "short_table_name\n",
+            "column\n",
+            "column_label\n",
+            "values\n"
+          ]
+        }
       ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-ad8384cf-190c-4879-842c-66a9ba0c6c4f\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>collection_id</th>\n",
-       "      <th>table_name</th>\n",
-       "      <th>short_table_name</th>\n",
-       "      <th>column</th>\n",
-       "      <th>column_label</th>\n",
-       "      <th>values</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>3828</th>\n",
-       "      <td>c4kc_kits</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.c4kc_kit...</td>\n",
-       "      <td>c4kc_kits_clinical</td>\n",
-       "      <td>comorbidities__dementia</td>\n",
-       "      <td>comorbidities__dementia</td>\n",
-       "      <td>[{'option_code': 'False', 'option_description'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3829</th>\n",
-       "      <td>c4kc_kits</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.c4kc_kit...</td>\n",
-       "      <td>c4kc_kits_clinical</td>\n",
-       "      <td>comorbidities__aids</td>\n",
-       "      <td>comorbidities__aids</td>\n",
-       "      <td>[{'option_code': 'False', 'option_description'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3830</th>\n",
-       "      <td>c4kc_kits</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.c4kc_kit...</td>\n",
-       "      <td>c4kc_kits_clinical</td>\n",
-       "      <td>intraoperative_complications__cardiac_event</td>\n",
-       "      <td>intraoperative_complications__cardiac_event</td>\n",
-       "      <td>[{'option_code': 'False', 'option_description'...</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ad8384cf-190c-4879-842c-66a9ba0c6c4f')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-ad8384cf-190c-4879-842c-66a9ba0c6c4f button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-ad8384cf-190c-4879-842c-66a9ba0c6c4f');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "<div id=\"df-6915e002-eb06-45d4-8395-839d65258c39\">\n",
-       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-6915e002-eb06-45d4-8395-839d65258c39')\"\n",
-       "            title=\"Suggest charts\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "     width=\"24px\">\n",
-       "    <g>\n",
-       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
-       "    </g>\n",
-       "</svg>\n",
-       "  </button>\n",
-       "\n",
-       "<style>\n",
-       "  .colab-df-quickchart {\n",
-       "      --bg-color: #E8F0FE;\n",
-       "      --fill-color: #1967D2;\n",
-       "      --hover-bg-color: #E2EBFA;\n",
-       "      --hover-fill-color: #174EA6;\n",
-       "      --disabled-fill-color: #AAA;\n",
-       "      --disabled-bg-color: #DDD;\n",
-       "  }\n",
-       "\n",
-       "  [theme=dark] .colab-df-quickchart {\n",
-       "      --bg-color: #3B4455;\n",
-       "      --fill-color: #D2E3FC;\n",
-       "      --hover-bg-color: #434B5C;\n",
-       "      --hover-fill-color: #FFFFFF;\n",
-       "      --disabled-bg-color: #3B4455;\n",
-       "      --disabled-fill-color: #666;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart {\n",
-       "    background-color: var(--bg-color);\n",
-       "    border: none;\n",
-       "    border-radius: 50%;\n",
-       "    cursor: pointer;\n",
-       "    display: none;\n",
-       "    fill: var(--fill-color);\n",
-       "    height: 32px;\n",
-       "    padding: 0;\n",
-       "    width: 32px;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart:hover {\n",
-       "    background-color: var(--hover-bg-color);\n",
-       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "    fill: var(--button-hover-fill-color);\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart-complete:disabled,\n",
-       "  .colab-df-quickchart-complete:disabled:hover {\n",
-       "    background-color: var(--disabled-bg-color);\n",
-       "    fill: var(--disabled-fill-color);\n",
-       "    box-shadow: none;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-spinner {\n",
-       "    border: 2px solid var(--fill-color);\n",
-       "    border-color: transparent;\n",
-       "    border-bottom-color: var(--fill-color);\n",
-       "    animation:\n",
-       "      spin 1s steps(1) infinite;\n",
-       "  }\n",
-       "\n",
-       "  @keyframes spin {\n",
-       "    0% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "      border-left-color: var(--fill-color);\n",
-       "    }\n",
-       "    20% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    30% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    40% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    60% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    80% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "    90% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "  }\n",
-       "</style>\n",
-       "\n",
-       "  <script>\n",
-       "    async function quickchart(key) {\n",
-       "      const quickchartButtonEl =\n",
-       "        document.querySelector('#' + key + ' button');\n",
-       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
-       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
-       "      try {\n",
-       "        const charts = await google.colab.kernel.invokeFunction(\n",
-       "            'suggestCharts', [key], {});\n",
-       "      } catch (error) {\n",
-       "        console.error('Error during call to suggestCharts:', error);\n",
-       "      }\n",
-       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
-       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
-       "    }\n",
-       "    (() => {\n",
-       "      let quickchartButtonEl =\n",
-       "        document.querySelector('#df-6915e002-eb06-45d4-8395-839d65258c39 button');\n",
-       "      quickchartButtonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "    })();\n",
-       "  </script>\n",
-       "</div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
+      "source": [
+        "from idc_index import IDCClient\n",
+        "\n",
+        "c = IDCClient()\n",
+        "\n",
+        "c.fetch_index('clinical_index')\n",
+        "\n",
+        "print('Columns avaialable in clinical_index:\\n'+'\\n'.join(c.clinical_index.keys()))"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "B9Vgq-gtI9G4"
+      },
+      "source": [
+        "This table is documented in https://idc-index.readthedocs.io/en/latest/column_descriptions.html#clinical-index.\n",
+        "\n",
+        "\n",
+        "Accessing relevant clinical data for a given collection is a two-step process:\n",
+        "\n",
+        "* Step 1: use `clinical_index` table to identify relevant metadata attributes and the names of the tables where the corresponding metadata is located.\n",
+        "* Step 2: load the specific clinical data table with the selected attribute referenced from `clinical_index` and access clinical metadata for the individual patients.\n",
+        "\n",
+        "As a reminder, all of the data - including clinical metadata - in IDC is anonymized!\n",
+        "\n",
+        "For the sake of this example, we will start with identifying clinical data attributes that accompany NLST collection.\n",
+        "\n",
+        "`collection_id` column of `clinical_index` can be used to associate clinical data attribute to the collection it accompanies."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "## Understanding individual clinical data attributes\n",
+        "\n",
+        "Here's an example of the columns that correspond to the clinical metadata in one of the ACRIN collections, where `column` is not particularly helpful, but `column_label` provides human readable information to allow interpretation of the column."
       ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "summary": "{\n  \"name\": \"c\",\n  \"rows\": 3,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"c4kc_kits\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"bigquery-public-data.idc_v20_clinical.c4kc_kits_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"short_table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"c4kc_kits_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"comorbidities__dementia\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"comorbidities__dementia\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+      "metadata": {
+        "id": "CM-vI0FJClLy"
       }
-     },
-     "metadata": {},
-     "execution_count": 8
-    }
-   ],
-   "source": [
-    "c.clinical_index[c.clinical_index[\"collection_id\"] == \"c4kc_kits\"][:3]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "voowbwXCeFBF"
-   },
-   "source": [
-    "**As a general rule of thumb**:\n",
-    "* when selecting specific columns from clinical tables, use `column` values\n",
-    "* when searching for concepts of interest in `column_metadata`, use `column_label`"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "gWgLxoraN3mY"
-   },
-   "source": [
-    "# Exploring IDC clinical data\n",
-    "\n",
-    "In the following sections of the notebook we go over some use cases to demonstrate various options for navigating IDC clinical data.\n",
-    "\n",
-    "As always, if you have a use case that is not addressed here, if you have suggestions or are confused - please start a discussion thread on the [IDC User forum](https://discourse.canceridc.dev/)!"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "03rwe7OfvsnV"
-   },
-   "source": [
-    "## Collection-focused exploration\n",
-    "\n",
-    "If you used clinical data stored in TCIA, perhaps you started with a specific collection of your interest, downloaded clinical data files for that collection, parsed them into your code. Let's go over those steps the IDC way!\n",
-    "\n",
-    "First, let's see which of the collections in IDC have clinical data in BigQuery. To do that we will reuse the pandas dataframe with the results of the query we executed earlier.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "metadata": {
-    "id": "TIZq1GbQoTIC",
-    "outputId": "ef45376f-8c6e-4eed-910f-a76e0badae46",
-    "colab": {
-     "base_uri": "https://localhost:8080/"
-    }
-   },
-   "outputs": [
+    },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "['acrin_6698',\n",
-       " 'acrin_contralateral_breast_mr',\n",
-       " 'acrin_flt_breast',\n",
-       " 'acrin_nsclc_fdg_pet',\n",
-       " 'adrenal_acc_ki67_seg',\n",
-       " 'advanced_mri_breast_lesions',\n",
-       " 'anti_pd_1_lung',\n",
-       " 'b_mode_and_ceus_liver',\n",
-       " 'breast_diagnosis',\n",
-       " 'breast_mri_nact_pilot',\n",
-       " 'c4kc_kits',\n",
-       " 'cc_tumor_heterogeneity',\n",
-       " 'cmmd',\n",
-       " 'colorectal_liver_metastases',\n",
-       " 'covid_19_ar',\n",
-       " 'covid_19_ny_sbu',\n",
-       " 'cptac_brca',\n",
-       " 'cptac_ccrcc',\n",
-       " 'cptac_coad',\n",
-       " 'cptac_gbm',\n",
-       " 'cptac_hnscc',\n",
-       " 'cptac_lscc',\n",
-       " 'cptac_luad',\n",
-       " 'cptac_ov',\n",
-       " 'cptac_pda',\n",
-       " 'cptac_ucec',\n",
-       " 'ctpred_sunitinib_pannet',\n",
-       " 'duke_breast_cancer_mri',\n",
-       " 'ea1141',\n",
-       " 'hcc_tace_seg',\n",
-       " 'htan_hms',\n",
-       " 'htan_ohsu',\n",
-       " 'htan_vanderbilt',\n",
-       " 'htan_wustl',\n",
-       " 'ispy1',\n",
-       " 'ispy2',\n",
-       " 'lidc_idri',\n",
-       " 'lung_fused_ct_pathology',\n",
-       " 'lung_pet_ct_dx',\n",
-       " 'mediastinal_lymph_node_seg',\n",
-       " 'midrc_ricord_1a',\n",
-       " 'midrc_ricord_1b',\n",
-       " 'midrc_ricord_1c',\n",
-       " 'nlst',\n",
-       " 'nsclc_radiogenomics',\n",
-       " 'nsclc_radiomics',\n",
-       " 'nsclc_radiomics_genomics',\n",
-       " 'nsclc_radiomics_interobserver1',\n",
-       " 'prostate_diagnosis',\n",
-       " 'prostatex',\n",
-       " 'qin_breast',\n",
-       " 'remind',\n",
-       " 'rider_lung_pet_ct',\n",
-       " 'rms_mutation_prediction',\n",
-       " 'soft_tissue_sarcoma',\n",
-       " 'spie_aapm_lung_ct_challenge',\n",
-       " 'spine_mets_ct_seg',\n",
-       " 'tcga_acc',\n",
-       " 'tcga_blca',\n",
-       " 'tcga_brca',\n",
-       " 'tcga_cesc',\n",
-       " 'tcga_chol',\n",
-       " 'tcga_coad',\n",
-       " 'tcga_dlbc',\n",
-       " 'tcga_esca',\n",
-       " 'tcga_gbm',\n",
-       " 'tcga_hnsc',\n",
-       " 'tcga_kich',\n",
-       " 'tcga_kirc',\n",
-       " 'tcga_kirp',\n",
-       " 'tcga_lgg',\n",
-       " 'tcga_lihc',\n",
-       " 'tcga_luad',\n",
-       " 'tcga_lusc',\n",
-       " 'tcga_meso',\n",
-       " 'tcga_ov',\n",
-       " 'tcga_paad',\n",
-       " 'tcga_pcpg',\n",
-       " 'tcga_prad',\n",
-       " 'tcga_read',\n",
-       " 'tcga_sarc',\n",
-       " 'tcga_skcm',\n",
-       " 'tcga_stad',\n",
-       " 'tcga_tgct',\n",
-       " 'tcga_thca',\n",
-       " 'tcga_thym',\n",
-       " 'tcga_ucec',\n",
-       " 'tcga_ucs',\n",
-       " 'tcga_uvm',\n",
-       " 'upenn_gbm']"
+      "cell_type": "code",
+      "execution_count": 5,
+      "metadata": {
+        "id": "nGsS6SvPdrWn",
+        "outputId": "2f888f38-daff-410f-d150-96a7575b1265",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 653
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "            collection_id                                   short_table_name  \\\n",
+              "2359  acrin_nsclc_fdg_pet                             acrin_nsclc_fdg_pet_A0   \n",
+              "2360  acrin_nsclc_fdg_pet                             acrin_nsclc_fdg_pet_A0   \n",
+              "2361  acrin_nsclc_fdg_pet                             acrin_nsclc_fdg_pet_A0   \n",
+              "2362  acrin_nsclc_fdg_pet                             acrin_nsclc_fdg_pet_A0   \n",
+              "2363  acrin_nsclc_fdg_pet                             acrin_nsclc_fdg_pet_A0   \n",
+              "...                   ...                                                ...   \n",
+              "3643  acrin_nsclc_fdg_pet  acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation   \n",
+              "3644  acrin_nsclc_fdg_pet  acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation   \n",
+              "3645  acrin_nsclc_fdg_pet  acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation   \n",
+              "3646  acrin_nsclc_fdg_pet  acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation   \n",
+              "3647  acrin_nsclc_fdg_pet  acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation   \n",
+              "\n",
+              "                           column  \\\n",
+              "2359                      inst_no   \n",
+              "2360                     entryage   \n",
+              "2361                        a0e26   \n",
+              "2362                          rec   \n",
+              "2363                        a0e4d   \n",
+              "...                           ...   \n",
+              "3643             studyinstanceuid   \n",
+              "3644          ptseriesinstanceuid   \n",
+              "3645          ctseriesinstanceuid   \n",
+              "3646               aisegmentation   \n",
+              "3647  commentsaboutaisegmentation   \n",
+              "\n",
+              "                                           column_label  \\\n",
+              "2359                   De-identified Institution Number   \n",
+              "2360                        Patient age at Registration   \n",
+              "2361                            RTOG INSTITUTION NUMBER   \n",
+              "2362                      Data receipt (from base date)   \n",
+              "2363  \"Days from Base_dt to DATE THE  STUDY-SPECIFIC...   \n",
+              "...                                                 ...   \n",
+              "3643                                   StudyInstanceUID   \n",
+              "3644                                PTSeriesInstanceUID   \n",
+              "3645                                CTSeriesInstanceUID   \n",
+              "3646                                     AISegmentation   \n",
+              "3647                        CommentsAboutAISegmentation   \n",
+              "\n",
+              "                                                 values  \n",
+              "2359  [{'option_code': '\"nan\"', 'option_description'...  \n",
+              "2360  [{'option_code': '\"nan\"', 'option_description'...  \n",
+              "2361  [{'option_code': '\"nan\"', 'option_description'...  \n",
+              "2362  [{'option_code': '\"nan\"', 'option_description'...  \n",
+              "2363  [{'option_code': '\"nan\"', 'option_description'...  \n",
+              "...                                                 ...  \n",
+              "3643                                                 []  \n",
+              "3644                                                 []  \n",
+              "3645                                                 []  \n",
+              "3646                                                 []  \n",
+              "3647                                                 []  \n",
+              "\n",
+              "[1289 rows x 5 columns]"
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-c19868ab-06c0-48fd-8619-1f65ede862ed\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>collection_id</th>\n",
+              "      <th>short_table_name</th>\n",
+              "      <th>column</th>\n",
+              "      <th>column_label</th>\n",
+              "      <th>values</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>2359</th>\n",
+              "      <td>acrin_nsclc_fdg_pet</td>\n",
+              "      <td>acrin_nsclc_fdg_pet_A0</td>\n",
+              "      <td>inst_no</td>\n",
+              "      <td>De-identified Institution Number</td>\n",
+              "      <td>[{'option_code': '\"nan\"', 'option_description'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>2360</th>\n",
+              "      <td>acrin_nsclc_fdg_pet</td>\n",
+              "      <td>acrin_nsclc_fdg_pet_A0</td>\n",
+              "      <td>entryage</td>\n",
+              "      <td>Patient age at Registration</td>\n",
+              "      <td>[{'option_code': '\"nan\"', 'option_description'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>2361</th>\n",
+              "      <td>acrin_nsclc_fdg_pet</td>\n",
+              "      <td>acrin_nsclc_fdg_pet_A0</td>\n",
+              "      <td>a0e26</td>\n",
+              "      <td>RTOG INSTITUTION NUMBER</td>\n",
+              "      <td>[{'option_code': '\"nan\"', 'option_description'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>2362</th>\n",
+              "      <td>acrin_nsclc_fdg_pet</td>\n",
+              "      <td>acrin_nsclc_fdg_pet_A0</td>\n",
+              "      <td>rec</td>\n",
+              "      <td>Data receipt (from base date)</td>\n",
+              "      <td>[{'option_code': '\"nan\"', 'option_description'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>2363</th>\n",
+              "      <td>acrin_nsclc_fdg_pet</td>\n",
+              "      <td>acrin_nsclc_fdg_pet_A0</td>\n",
+              "      <td>a0e4d</td>\n",
+              "      <td>\"Days from Base_dt to DATE THE  STUDY-SPECIFIC...</td>\n",
+              "      <td>[{'option_code': '\"nan\"', 'option_description'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>...</th>\n",
+              "      <td>...</td>\n",
+              "      <td>...</td>\n",
+              "      <td>...</td>\n",
+              "      <td>...</td>\n",
+              "      <td>...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3643</th>\n",
+              "      <td>acrin_nsclc_fdg_pet</td>\n",
+              "      <td>acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation</td>\n",
+              "      <td>studyinstanceuid</td>\n",
+              "      <td>StudyInstanceUID</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3644</th>\n",
+              "      <td>acrin_nsclc_fdg_pet</td>\n",
+              "      <td>acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation</td>\n",
+              "      <td>ptseriesinstanceuid</td>\n",
+              "      <td>PTSeriesInstanceUID</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3645</th>\n",
+              "      <td>acrin_nsclc_fdg_pet</td>\n",
+              "      <td>acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation</td>\n",
+              "      <td>ctseriesinstanceuid</td>\n",
+              "      <td>CTSeriesInstanceUID</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3646</th>\n",
+              "      <td>acrin_nsclc_fdg_pet</td>\n",
+              "      <td>acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation</td>\n",
+              "      <td>aisegmentation</td>\n",
+              "      <td>AISegmentation</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3647</th>\n",
+              "      <td>acrin_nsclc_fdg_pet</td>\n",
+              "      <td>acrin_nsclc_fdg_pet_bamf_lung_pet_ct_segmentation</td>\n",
+              "      <td>commentsaboutaisegmentation</td>\n",
+              "      <td>CommentsAboutAISegmentation</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "<p>1289 rows × 5 columns</p>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c19868ab-06c0-48fd-8619-1f65ede862ed')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-c19868ab-06c0-48fd-8619-1f65ede862ed button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-c19868ab-06c0-48fd-8619-1f65ede862ed');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "<div id=\"df-eaa84ef7-34b8-443b-a32e-051b4e5b2093\">\n",
+              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-eaa84ef7-34b8-443b-a32e-051b4e5b2093')\"\n",
+              "            title=\"Suggest charts\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "     width=\"24px\">\n",
+              "    <g>\n",
+              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+              "    </g>\n",
+              "</svg>\n",
+              "  </button>\n",
+              "\n",
+              "<style>\n",
+              "  .colab-df-quickchart {\n",
+              "      --bg-color: #E8F0FE;\n",
+              "      --fill-color: #1967D2;\n",
+              "      --hover-bg-color: #E2EBFA;\n",
+              "      --hover-fill-color: #174EA6;\n",
+              "      --disabled-fill-color: #AAA;\n",
+              "      --disabled-bg-color: #DDD;\n",
+              "  }\n",
+              "\n",
+              "  [theme=dark] .colab-df-quickchart {\n",
+              "      --bg-color: #3B4455;\n",
+              "      --fill-color: #D2E3FC;\n",
+              "      --hover-bg-color: #434B5C;\n",
+              "      --hover-fill-color: #FFFFFF;\n",
+              "      --disabled-bg-color: #3B4455;\n",
+              "      --disabled-fill-color: #666;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart {\n",
+              "    background-color: var(--bg-color);\n",
+              "    border: none;\n",
+              "    border-radius: 50%;\n",
+              "    cursor: pointer;\n",
+              "    display: none;\n",
+              "    fill: var(--fill-color);\n",
+              "    height: 32px;\n",
+              "    padding: 0;\n",
+              "    width: 32px;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart:hover {\n",
+              "    background-color: var(--hover-bg-color);\n",
+              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "    fill: var(--button-hover-fill-color);\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart-complete:disabled,\n",
+              "  .colab-df-quickchart-complete:disabled:hover {\n",
+              "    background-color: var(--disabled-bg-color);\n",
+              "    fill: var(--disabled-fill-color);\n",
+              "    box-shadow: none;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-spinner {\n",
+              "    border: 2px solid var(--fill-color);\n",
+              "    border-color: transparent;\n",
+              "    border-bottom-color: var(--fill-color);\n",
+              "    animation:\n",
+              "      spin 1s steps(1) infinite;\n",
+              "  }\n",
+              "\n",
+              "  @keyframes spin {\n",
+              "    0% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "      border-left-color: var(--fill-color);\n",
+              "    }\n",
+              "    20% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    30% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    40% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    60% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    80% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "    90% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "  }\n",
+              "</style>\n",
+              "\n",
+              "  <script>\n",
+              "    async function quickchart(key) {\n",
+              "      const quickchartButtonEl =\n",
+              "        document.querySelector('#' + key + ' button');\n",
+              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+              "      try {\n",
+              "        const charts = await google.colab.kernel.invokeFunction(\n",
+              "            'suggestCharts', [key], {});\n",
+              "      } catch (error) {\n",
+              "        console.error('Error during call to suggestCharts:', error);\n",
+              "      }\n",
+              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+              "    }\n",
+              "    (() => {\n",
+              "      let quickchartButtonEl =\n",
+              "        document.querySelector('#df-eaa84ef7-34b8-443b-a32e-051b4e5b2093 button');\n",
+              "      quickchartButtonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "    })();\n",
+              "  </script>\n",
+              "</div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "summary": "{\n  \"name\": \"acrin_nsclc_fdg_pet_clinical_columns[['collection_id','short_table_name','column','column_label','values']]\",\n  \"rows\": 1289,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"acrin_nsclc_fdg_pet\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"short_table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 21,\n        \"samples\": [\n          \"acrin_nsclc_fdg_pet_A0\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1211,\n        \"samples\": [\n          \"aee34\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 789,\n        \"samples\": [\n          \"AREAS OF INTEREST WITHIN MRI SCAN - CHEST - IMAGING DATE\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 5
+        }
+      ],
+      "source": [
+        "acrin_nsclc_fdg_pet_clinical_columns = c.clinical_index[c.clinical_index['collection_id']=='acrin_nsclc_fdg_pet']\n",
+        "acrin_nsclc_fdg_pet_clinical_columns[['collection_id','short_table_name','column','column_label','values']]"
       ]
-     },
-     "metadata": {},
-     "execution_count": 9
-    }
-   ],
-   "source": [
-    "c.clinical_index[\"collection_id\"].unique().tolist()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "_KoEJ_Wvwa79"
-   },
-   "source": [
-    "If you are interested in what clinical data is available for the specific collection, you can select only the rows corresponding to that collection in the `column_metadata` table. Here we select a subset of columns to improve readability of the dataframe.\n",
-    "\n",
-    "Note that for some collections, clinical data sheets are accompanied by dictionaries, which formalize the values encountered. Examples of such collections are [ISPY1](https://wiki.cancerimagingarchive.net/display/Public/ISPY1) or ACRIN trials.\n",
-    "\n",
-    "For many other collections there are no such dictionaries available. In those situations, the values you will see in the `values` columns have been derived by examining the distinct values encountered in the clinical data sheets.\n",
-    "\n",
-    "In the following we look at the clinical data columns (\"dictionary terms\") for the [ACRIN 6698 collection](https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=50135447) mentioned earlier."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 14,
-   "metadata": {
-    "id": "zrtRWr_SwmIJ",
-    "outputId": "b16a2902-be23-4b1d-bb96-2151271eaa4d",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 1000
-    }
-   },
-   "outputs": [
+    },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "m0H78L-VdxHn"
+      },
+      "source": [
+        "For some columns, the values come from a defined set. In the example above, we can, for example, examine the values assigned to encode patient race."
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "   collection_id                                         table_name  \\\n",
-       "0     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "1     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "2     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "3     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "4     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "5     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "6     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "7     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "8     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "9     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "10    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "11    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "12    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "13    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "14    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "15    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "16    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "17    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "18    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "19    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "20    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "21    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "22    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "23    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "24    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "25    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "26    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "27    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "28    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "29    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "30    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "31    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
-       "\n",
-       "       short_table_name                column  \\\n",
-       "0   acrin_6698_clinical          source_batch   \n",
-       "1   acrin_6698_clinical                    t0   \n",
-       "2   acrin_6698_clinical                    t1   \n",
-       "3   acrin_6698_clinical                    t2   \n",
-       "4   acrin_6698_clinical                    t3   \n",
-       "5   acrin_6698_clinical           bmmr2_train   \n",
-       "6   acrin_6698_clinical            bmmr2_test   \n",
-       "7   acrin_6698_clinical        primary_aim_t0   \n",
-       "8   acrin_6698_clinical        primary_aim_t1   \n",
-       "9   acrin_6698_clinical        primary_aim_t2   \n",
-       "10  acrin_6698_clinical        primary_aim_t3   \n",
-       "11  acrin_6698_clinical       analysis_cohort   \n",
-       "12  acrin_6698_clinical            eligigible   \n",
-       "13  acrin_6698_clinical           test_retest   \n",
-       "14  acrin_6698_clinical       qc_pass_fail_t0   \n",
-       "15  acrin_6698_clinical       qc_pass_fail_t1   \n",
-       "16  acrin_6698_clinical       qc_pass_fail_t2   \n",
-       "17  acrin_6698_clinical       qc_pass_fail_t3   \n",
-       "18  acrin_6698_clinical  qc_roi_confidence_t0   \n",
-       "19  acrin_6698_clinical  qc_roi_confidence_t1   \n",
-       "20  acrin_6698_clinical  qc_roi_confidence_t2   \n",
-       "21  acrin_6698_clinical  qc_roi_confidence_t3   \n",
-       "22  acrin_6698_clinical                  race   \n",
-       "23  acrin_6698_clinical                 ltype   \n",
-       "24  acrin_6698_clinical               hrher4g   \n",
-       "25  acrin_6698_clinical              sbrgrade   \n",
-       "26  acrin_6698_clinical                   pcr   \n",
-       "27  acrin_6698_clinical      dicom_patient_id   \n",
-       "28  acrin_6698_clinical   i_spy_2_research_id   \n",
-       "29  acrin_6698_clinical       tcia_patient_id   \n",
-       "30  acrin_6698_clinical                   age   \n",
-       "31  acrin_6698_clinical                  mrld   \n",
-       "\n",
-       "                                         column_label  \\\n",
-       "0                         idc_provenance_source_batch   \n",
-       "1      T0 (baseline) MRI study included in collection   \n",
-       "2      T1 (early-Tx) MRI study included in collection   \n",
-       "3   T2 (inter-regimen) MRI study included in colle...   \n",
-       "4   T3 (pre-surgery) MRI study included in collection   \n",
-       "5   Patient included in the BMMR2 challenge traini...   \n",
-       "6   Patient included in the BMMR2 challenge test c...   \n",
-       "7   T0 MRI study included in the ACRIN-6698 primar...   \n",
-       "8   T1 MRI study included in the ACRIN-6698 primar...   \n",
-       "9   T2 MRI study included in the ACRIN-6698 primar...   \n",
-       "10  T3 MRI study included in the ACRIN-6698 primar...   \n",
-       "11         Patient in primary analysis cohort (n=242)   \n",
-       "12                                         eligigible   \n",
-       "13  Study time point that includes test/retest DWI...   \n",
-       "14             T0 MRI DWI Quality control assessment:   \n",
-       "15             T0 MRI DWI Quality control assessment:   \n",
-       "16             T0 MRI DWI Quality control assessment:   \n",
-       "17             T0 MRI DWI Quality control assessment:   \n",
-       "18  T0 quality control confidence level for whole-...   \n",
-       "19  T1 quality control confidence level for whole-...   \n",
-       "20  T2 quality control confidence level for whole-...   \n",
-       "21  T3 quality control confidence level for whole-...   \n",
-       "22                                      Patient race:   \n",
-       "23                                 Index lesion type:   \n",
-       "24             Hormone receptor (HR) and HER2 status:   \n",
-       "25                                       Tumor grade:   \n",
-       "26      Pathologic complete response (pCR) at surgery   \n",
-       "27                    idc_provenance_dicom_patient_id   \n",
-       "28      Double-blinded patient ID number from I-SPY 2   \n",
-       "29  Patient identification string in TCIA collecti...   \n",
-       "30                         Age in years at enrollment   \n",
-       "31  MRI measured longest diameter (cm) at baseline...   \n",
-       "\n",
-       "                                               values  \n",
-       "0   [{'option_code': '0', 'option_description': No...  \n",
-       "1   [{'option_code': '0', 'option_description': ' ...  \n",
-       "2   [{'option_code': '0', 'option_description': ' ...  \n",
-       "3   [{'option_code': '0', 'option_description': ' ...  \n",
-       "4   [{'option_code': '0', 'option_description': ' ...  \n",
-       "5   [{'option_code': '0', 'option_description': ' ...  \n",
-       "6   [{'option_code': '0', 'option_description': ' ...  \n",
-       "7   [{'option_code': '0', 'option_description': ' ...  \n",
-       "8   [{'option_code': '0', 'option_description': ' ...  \n",
-       "9   [{'option_code': '0', 'option_description': ' ...  \n",
-       "10  [{'option_code': '0', 'option_description': ' ...  \n",
-       "11  [{'option_code': '0', 'option_description': ' ...  \n",
-       "12  [{'option_code': '0', 'option_description': No...  \n",
-       "13  [{'option_code': 'T0', 'option_description': N...  \n",
-       "14  [{'option_code': 'PASS', 'option_description':...  \n",
-       "15  [{'option_code': 'PASS', 'option_description':...  \n",
-       "16  [{'option_code': 'PASS', 'option_description':...  \n",
-       "17  [{'option_code': 'PASS', 'option_description':...  \n",
-       "18  [{'option_code': 'HIGH', 'option_description':...  \n",
-       "19  [{'option_code': 'HIGH', 'option_description':...  \n",
-       "20  [{'option_code': 'HIGH', 'option_description':...  \n",
-       "21  [{'option_code': 'HIGH', 'option_description':...  \n",
-       "22  [{'option_code': 'White', 'option_description'...  \n",
-       "23  [{'option_code': 'Single mass', 'option_descri...  \n",
-       "24  [{'option_code': 'HR + / HER2 +', 'option_desc...  \n",
-       "25  [{'option_code': 'I (Low)', 'option_descriptio...  \n",
-       "26  [{'option_code': 'non-pCR', 'option_descriptio...  \n",
-       "27                                                 []  \n",
-       "28                                                 []  \n",
-       "29                                                 []  \n",
-       "30                                                 []  \n",
-       "31                                                 []  "
-      ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-f21ecf89-f728-408b-a5d4-82b2b9721c87\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>collection_id</th>\n",
-       "      <th>table_name</th>\n",
-       "      <th>short_table_name</th>\n",
-       "      <th>column</th>\n",
-       "      <th>column_label</th>\n",
-       "      <th>values</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>0</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>source_batch</td>\n",
-       "      <td>idc_provenance_source_batch</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': No...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>t0</td>\n",
-       "      <td>T0 (baseline) MRI study included in collection</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>t1</td>\n",
-       "      <td>T1 (early-Tx) MRI study included in collection</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>t2</td>\n",
-       "      <td>T2 (inter-regimen) MRI study included in colle...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>4</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>t3</td>\n",
-       "      <td>T3 (pre-surgery) MRI study included in collection</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>5</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>bmmr2_train</td>\n",
-       "      <td>Patient included in the BMMR2 challenge traini...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>6</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>bmmr2_test</td>\n",
-       "      <td>Patient included in the BMMR2 challenge test c...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>primary_aim_t0</td>\n",
-       "      <td>T0 MRI study included in the ACRIN-6698 primar...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>primary_aim_t1</td>\n",
-       "      <td>T1 MRI study included in the ACRIN-6698 primar...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>primary_aim_t2</td>\n",
-       "      <td>T2 MRI study included in the ACRIN-6698 primar...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>10</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>primary_aim_t3</td>\n",
-       "      <td>T3 MRI study included in the ACRIN-6698 primar...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>11</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>analysis_cohort</td>\n",
-       "      <td>Patient in primary analysis cohort (n=242)</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>12</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>eligigible</td>\n",
-       "      <td>eligigible</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': No...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>13</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>test_retest</td>\n",
-       "      <td>Study time point that includes test/retest DWI...</td>\n",
-       "      <td>[{'option_code': 'T0', 'option_description': N...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>14</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_pass_fail_t0</td>\n",
-       "      <td>T0 MRI DWI Quality control assessment:</td>\n",
-       "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>15</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_pass_fail_t1</td>\n",
-       "      <td>T0 MRI DWI Quality control assessment:</td>\n",
-       "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>16</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_pass_fail_t2</td>\n",
-       "      <td>T0 MRI DWI Quality control assessment:</td>\n",
-       "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>17</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_pass_fail_t3</td>\n",
-       "      <td>T0 MRI DWI Quality control assessment:</td>\n",
-       "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>18</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_roi_confidence_t0</td>\n",
-       "      <td>T0 quality control confidence level for whole-...</td>\n",
-       "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>19</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_roi_confidence_t1</td>\n",
-       "      <td>T1 quality control confidence level for whole-...</td>\n",
-       "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>20</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_roi_confidence_t2</td>\n",
-       "      <td>T2 quality control confidence level for whole-...</td>\n",
-       "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>21</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_roi_confidence_t3</td>\n",
-       "      <td>T3 quality control confidence level for whole-...</td>\n",
-       "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>22</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>race</td>\n",
-       "      <td>Patient race:</td>\n",
-       "      <td>[{'option_code': 'White', 'option_description'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>23</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>ltype</td>\n",
-       "      <td>Index lesion type:</td>\n",
-       "      <td>[{'option_code': 'Single mass', 'option_descri...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>24</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>hrher4g</td>\n",
-       "      <td>Hormone receptor (HR) and HER2 status:</td>\n",
-       "      <td>[{'option_code': 'HR + / HER2 +', 'option_desc...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>25</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>sbrgrade</td>\n",
-       "      <td>Tumor grade:</td>\n",
-       "      <td>[{'option_code': 'I (Low)', 'option_descriptio...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>26</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>pcr</td>\n",
-       "      <td>Pathologic complete response (pCR) at surgery</td>\n",
-       "      <td>[{'option_code': 'non-pCR', 'option_descriptio...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>27</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>dicom_patient_id</td>\n",
-       "      <td>idc_provenance_dicom_patient_id</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>28</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>i_spy_2_research_id</td>\n",
-       "      <td>Double-blinded patient ID number from I-SPY 2</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>29</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>tcia_patient_id</td>\n",
-       "      <td>Patient identification string in TCIA collecti...</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>30</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>age</td>\n",
-       "      <td>Age in years at enrollment</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>31</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>mrld</td>\n",
-       "      <td>MRI measured longest diameter (cm) at baseline...</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f21ecf89-f728-408b-a5d4-82b2b9721c87')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-f21ecf89-f728-408b-a5d4-82b2b9721c87 button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-f21ecf89-f728-408b-a5d4-82b2b9721c87');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "<div id=\"df-74b7689b-126c-49f8-8297-8f4e4d6f02c4\">\n",
-       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-74b7689b-126c-49f8-8297-8f4e4d6f02c4')\"\n",
-       "            title=\"Suggest charts\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "     width=\"24px\">\n",
-       "    <g>\n",
-       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
-       "    </g>\n",
-       "</svg>\n",
-       "  </button>\n",
-       "\n",
-       "<style>\n",
-       "  .colab-df-quickchart {\n",
-       "      --bg-color: #E8F0FE;\n",
-       "      --fill-color: #1967D2;\n",
-       "      --hover-bg-color: #E2EBFA;\n",
-       "      --hover-fill-color: #174EA6;\n",
-       "      --disabled-fill-color: #AAA;\n",
-       "      --disabled-bg-color: #DDD;\n",
-       "  }\n",
-       "\n",
-       "  [theme=dark] .colab-df-quickchart {\n",
-       "      --bg-color: #3B4455;\n",
-       "      --fill-color: #D2E3FC;\n",
-       "      --hover-bg-color: #434B5C;\n",
-       "      --hover-fill-color: #FFFFFF;\n",
-       "      --disabled-bg-color: #3B4455;\n",
-       "      --disabled-fill-color: #666;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart {\n",
-       "    background-color: var(--bg-color);\n",
-       "    border: none;\n",
-       "    border-radius: 50%;\n",
-       "    cursor: pointer;\n",
-       "    display: none;\n",
-       "    fill: var(--fill-color);\n",
-       "    height: 32px;\n",
-       "    padding: 0;\n",
-       "    width: 32px;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart:hover {\n",
-       "    background-color: var(--hover-bg-color);\n",
-       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "    fill: var(--button-hover-fill-color);\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart-complete:disabled,\n",
-       "  .colab-df-quickchart-complete:disabled:hover {\n",
-       "    background-color: var(--disabled-bg-color);\n",
-       "    fill: var(--disabled-fill-color);\n",
-       "    box-shadow: none;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-spinner {\n",
-       "    border: 2px solid var(--fill-color);\n",
-       "    border-color: transparent;\n",
-       "    border-bottom-color: var(--fill-color);\n",
-       "    animation:\n",
-       "      spin 1s steps(1) infinite;\n",
-       "  }\n",
-       "\n",
-       "  @keyframes spin {\n",
-       "    0% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "      border-left-color: var(--fill-color);\n",
-       "    }\n",
-       "    20% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    30% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    40% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    60% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    80% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "    90% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "  }\n",
-       "</style>\n",
-       "\n",
-       "  <script>\n",
-       "    async function quickchart(key) {\n",
-       "      const quickchartButtonEl =\n",
-       "        document.querySelector('#' + key + ' button');\n",
-       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
-       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
-       "      try {\n",
-       "        const charts = await google.colab.kernel.invokeFunction(\n",
-       "            'suggestCharts', [key], {});\n",
-       "      } catch (error) {\n",
-       "        console.error('Error during call to suggestCharts:', error);\n",
-       "      }\n",
-       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
-       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
-       "    }\n",
-       "    (() => {\n",
-       "      let quickchartButtonEl =\n",
-       "        document.querySelector('#df-74b7689b-126c-49f8-8297-8f4e4d6f02c4 button');\n",
-       "      quickchartButtonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "    })();\n",
-       "  </script>\n",
-       "</div>\n",
-       "\n",
-       "  <div id=\"id_6e2ef9d9-d7bb-46eb-ba22-f750713f2f77\">\n",
-       "    <style>\n",
-       "      .colab-df-generate {\n",
-       "        background-color: #E8F0FE;\n",
-       "        border: none;\n",
-       "        border-radius: 50%;\n",
-       "        cursor: pointer;\n",
-       "        display: none;\n",
-       "        fill: #1967D2;\n",
-       "        height: 32px;\n",
-       "        padding: 0 0 0 0;\n",
-       "        width: 32px;\n",
-       "      }\n",
-       "\n",
-       "      .colab-df-generate:hover {\n",
-       "        background-color: #E2EBFA;\n",
-       "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "        fill: #174EA6;\n",
-       "      }\n",
-       "\n",
-       "      [theme=dark] .colab-df-generate {\n",
-       "        background-color: #3B4455;\n",
-       "        fill: #D2E3FC;\n",
-       "      }\n",
-       "\n",
-       "      [theme=dark] .colab-df-generate:hover {\n",
-       "        background-color: #434B5C;\n",
-       "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "        fill: #FFFFFF;\n",
-       "      }\n",
-       "    </style>\n",
-       "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('acrin6698_clinical_columns')\"\n",
-       "            title=\"Generate code using this dataframe.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "       width=\"24px\">\n",
-       "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "    <script>\n",
-       "      (() => {\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#id_6e2ef9d9-d7bb-46eb-ba22-f750713f2f77 button.colab-df-generate');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      buttonEl.onclick = () => {\n",
-       "        google.colab.notebook.generateWithVariable('acrin6698_clinical_columns');\n",
-       "      }\n",
-       "      })();\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
+      "cell_type": "code",
+      "execution_count": 7,
+      "metadata": {
+        "id": "9NCP15tRd2gU",
+        "outputId": "b92fdc9a-7f19-406b-d885-0d27250154c5",
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "[array([{'option_code': '1.0', 'option_description': 'American Indian or Alaska Native'},\n",
+              "        {'option_code': '2.0', 'option_description': 'Asian'},\n",
+              "        {'option_code': '3.0', 'option_description': 'Black or African American'},\n",
+              "        {'option_code': '4.0', 'option_description': 'Native Hawaiian or other Pacific Islander'},\n",
+              "        {'option_code': '5.0', 'option_description': 'White'},\n",
+              "        {'option_code': '6.0', 'option_description': 'More than one race'},\n",
+              "        {'option_code': '9.0', 'option_description': 'Unknown'}],\n",
+              "       dtype=object)]"
+            ]
+          },
+          "metadata": {},
+          "execution_count": 7
+        }
       ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "variable_name": "acrin6698_clinical_columns",
-       "summary": "{\n  \"name\": \"acrin6698_clinical_columns\",\n  \"rows\": 32,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"acrin_6698\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"bigquery-public-data.idc_v20_clinical.acrin_6698_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"short_table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"acrin_6698_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 32,\n        \"samples\": [\n          \"tcia_patient_id\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 29,\n        \"samples\": [\n          \"Age in years at enrollment\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
-      }
-     },
-     "metadata": {},
-     "execution_count": 14
-    }
-   ],
-   "source": [
-    "acrin6698_clinical_columns = c.clinical_index[c.clinical_index[\"collection_id\"] == \"acrin_6698\"]\n",
-    "\n",
-    "acrin6698_clinical_columns"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "Dm5cAn73SQJW"
-   },
-   "source": [
-    "Here's how you can select just the specific columns in the table - this way it is easier to examine the data."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 15,
-   "metadata": {
-    "id": "4mY1gbR7ysxx",
-    "outputId": "62e65454-2faa-43f1-9155-c6779c94cec9",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 1000
-    }
-   },
-   "outputs": [
+      "source": [
+        "race_values = acrin_nsclc_fdg_pet_clinical_columns[acrin_nsclc_fdg_pet_clinical_columns[\"column_label\"] == \"RACE\"][\"values\"]\n",
+        "\n",
+        "race_values.tolist()"
+      ]
+    },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "yaoJvf2nd6wL"
+      },
+      "source": [
+        "On the other hand, if we look at the metadata available for the `c4kc_kits` collection, `column_label` and `column` are identical.\n"
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "   collection_id     short_table_name                column  \\\n",
-       "0     acrin_6698  acrin_6698_clinical          source_batch   \n",
-       "1     acrin_6698  acrin_6698_clinical                    t0   \n",
-       "2     acrin_6698  acrin_6698_clinical                    t1   \n",
-       "3     acrin_6698  acrin_6698_clinical                    t2   \n",
-       "4     acrin_6698  acrin_6698_clinical                    t3   \n",
-       "5     acrin_6698  acrin_6698_clinical           bmmr2_train   \n",
-       "6     acrin_6698  acrin_6698_clinical            bmmr2_test   \n",
-       "7     acrin_6698  acrin_6698_clinical        primary_aim_t0   \n",
-       "8     acrin_6698  acrin_6698_clinical        primary_aim_t1   \n",
-       "9     acrin_6698  acrin_6698_clinical        primary_aim_t2   \n",
-       "10    acrin_6698  acrin_6698_clinical        primary_aim_t3   \n",
-       "11    acrin_6698  acrin_6698_clinical       analysis_cohort   \n",
-       "12    acrin_6698  acrin_6698_clinical            eligigible   \n",
-       "13    acrin_6698  acrin_6698_clinical           test_retest   \n",
-       "14    acrin_6698  acrin_6698_clinical       qc_pass_fail_t0   \n",
-       "15    acrin_6698  acrin_6698_clinical       qc_pass_fail_t1   \n",
-       "16    acrin_6698  acrin_6698_clinical       qc_pass_fail_t2   \n",
-       "17    acrin_6698  acrin_6698_clinical       qc_pass_fail_t3   \n",
-       "18    acrin_6698  acrin_6698_clinical  qc_roi_confidence_t0   \n",
-       "19    acrin_6698  acrin_6698_clinical  qc_roi_confidence_t1   \n",
-       "20    acrin_6698  acrin_6698_clinical  qc_roi_confidence_t2   \n",
-       "21    acrin_6698  acrin_6698_clinical  qc_roi_confidence_t3   \n",
-       "22    acrin_6698  acrin_6698_clinical                  race   \n",
-       "23    acrin_6698  acrin_6698_clinical                 ltype   \n",
-       "24    acrin_6698  acrin_6698_clinical               hrher4g   \n",
-       "25    acrin_6698  acrin_6698_clinical              sbrgrade   \n",
-       "26    acrin_6698  acrin_6698_clinical                   pcr   \n",
-       "27    acrin_6698  acrin_6698_clinical      dicom_patient_id   \n",
-       "28    acrin_6698  acrin_6698_clinical   i_spy_2_research_id   \n",
-       "29    acrin_6698  acrin_6698_clinical       tcia_patient_id   \n",
-       "30    acrin_6698  acrin_6698_clinical                   age   \n",
-       "31    acrin_6698  acrin_6698_clinical                  mrld   \n",
-       "\n",
-       "                                         column_label  \\\n",
-       "0                         idc_provenance_source_batch   \n",
-       "1      T0 (baseline) MRI study included in collection   \n",
-       "2      T1 (early-Tx) MRI study included in collection   \n",
-       "3   T2 (inter-regimen) MRI study included in colle...   \n",
-       "4   T3 (pre-surgery) MRI study included in collection   \n",
-       "5   Patient included in the BMMR2 challenge traini...   \n",
-       "6   Patient included in the BMMR2 challenge test c...   \n",
-       "7   T0 MRI study included in the ACRIN-6698 primar...   \n",
-       "8   T1 MRI study included in the ACRIN-6698 primar...   \n",
-       "9   T2 MRI study included in the ACRIN-6698 primar...   \n",
-       "10  T3 MRI study included in the ACRIN-6698 primar...   \n",
-       "11         Patient in primary analysis cohort (n=242)   \n",
-       "12                                         eligigible   \n",
-       "13  Study time point that includes test/retest DWI...   \n",
-       "14             T0 MRI DWI Quality control assessment:   \n",
-       "15             T0 MRI DWI Quality control assessment:   \n",
-       "16             T0 MRI DWI Quality control assessment:   \n",
-       "17             T0 MRI DWI Quality control assessment:   \n",
-       "18  T0 quality control confidence level for whole-...   \n",
-       "19  T1 quality control confidence level for whole-...   \n",
-       "20  T2 quality control confidence level for whole-...   \n",
-       "21  T3 quality control confidence level for whole-...   \n",
-       "22                                      Patient race:   \n",
-       "23                                 Index lesion type:   \n",
-       "24             Hormone receptor (HR) and HER2 status:   \n",
-       "25                                       Tumor grade:   \n",
-       "26      Pathologic complete response (pCR) at surgery   \n",
-       "27                    idc_provenance_dicom_patient_id   \n",
-       "28      Double-blinded patient ID number from I-SPY 2   \n",
-       "29  Patient identification string in TCIA collecti...   \n",
-       "30                         Age in years at enrollment   \n",
-       "31  MRI measured longest diameter (cm) at baseline...   \n",
-       "\n",
-       "                                               values  \n",
-       "0   [{'option_code': '0', 'option_description': No...  \n",
-       "1   [{'option_code': '0', 'option_description': ' ...  \n",
-       "2   [{'option_code': '0', 'option_description': ' ...  \n",
-       "3   [{'option_code': '0', 'option_description': ' ...  \n",
-       "4   [{'option_code': '0', 'option_description': ' ...  \n",
-       "5   [{'option_code': '0', 'option_description': ' ...  \n",
-       "6   [{'option_code': '0', 'option_description': ' ...  \n",
-       "7   [{'option_code': '0', 'option_description': ' ...  \n",
-       "8   [{'option_code': '0', 'option_description': ' ...  \n",
-       "9   [{'option_code': '0', 'option_description': ' ...  \n",
-       "10  [{'option_code': '0', 'option_description': ' ...  \n",
-       "11  [{'option_code': '0', 'option_description': ' ...  \n",
-       "12  [{'option_code': '0', 'option_description': No...  \n",
-       "13  [{'option_code': 'T0', 'option_description': N...  \n",
-       "14  [{'option_code': 'PASS', 'option_description':...  \n",
-       "15  [{'option_code': 'PASS', 'option_description':...  \n",
-       "16  [{'option_code': 'PASS', 'option_description':...  \n",
-       "17  [{'option_code': 'PASS', 'option_description':...  \n",
-       "18  [{'option_code': 'HIGH', 'option_description':...  \n",
-       "19  [{'option_code': 'HIGH', 'option_description':...  \n",
-       "20  [{'option_code': 'HIGH', 'option_description':...  \n",
-       "21  [{'option_code': 'HIGH', 'option_description':...  \n",
-       "22  [{'option_code': 'White', 'option_description'...  \n",
-       "23  [{'option_code': 'Single mass', 'option_descri...  \n",
-       "24  [{'option_code': 'HR + / HER2 +', 'option_desc...  \n",
-       "25  [{'option_code': 'I (Low)', 'option_descriptio...  \n",
-       "26  [{'option_code': 'non-pCR', 'option_descriptio...  \n",
-       "27                                                 []  \n",
-       "28                                                 []  \n",
-       "29                                                 []  \n",
-       "30                                                 []  \n",
-       "31                                                 []  "
+      "cell_type": "code",
+      "execution_count": 8,
+      "metadata": {
+        "id": "AYMJ8kjTd9eb",
+        "outputId": "1ce1396b-66f6-4b22-e372-c3848e5a52e6",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 303
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "     collection_id                                         table_name  \\\n",
+              "3828     c4kc_kits  bigquery-public-data.idc_v20_clinical.c4kc_kit...   \n",
+              "3829     c4kc_kits  bigquery-public-data.idc_v20_clinical.c4kc_kit...   \n",
+              "3830     c4kc_kits  bigquery-public-data.idc_v20_clinical.c4kc_kit...   \n",
+              "\n",
+              "        short_table_name                                       column  \\\n",
+              "3828  c4kc_kits_clinical                      comorbidities__dementia   \n",
+              "3829  c4kc_kits_clinical                          comorbidities__aids   \n",
+              "3830  c4kc_kits_clinical  intraoperative_complications__cardiac_event   \n",
+              "\n",
+              "                                     column_label  \\\n",
+              "3828                      comorbidities__dementia   \n",
+              "3829                          comorbidities__aids   \n",
+              "3830  intraoperative_complications__cardiac_event   \n",
+              "\n",
+              "                                                 values  \n",
+              "3828  [{'option_code': 'False', 'option_description'...  \n",
+              "3829  [{'option_code': 'False', 'option_description'...  \n",
+              "3830  [{'option_code': 'False', 'option_description'...  "
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-ad8384cf-190c-4879-842c-66a9ba0c6c4f\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>collection_id</th>\n",
+              "      <th>table_name</th>\n",
+              "      <th>short_table_name</th>\n",
+              "      <th>column</th>\n",
+              "      <th>column_label</th>\n",
+              "      <th>values</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>3828</th>\n",
+              "      <td>c4kc_kits</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.c4kc_kit...</td>\n",
+              "      <td>c4kc_kits_clinical</td>\n",
+              "      <td>comorbidities__dementia</td>\n",
+              "      <td>comorbidities__dementia</td>\n",
+              "      <td>[{'option_code': 'False', 'option_description'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3829</th>\n",
+              "      <td>c4kc_kits</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.c4kc_kit...</td>\n",
+              "      <td>c4kc_kits_clinical</td>\n",
+              "      <td>comorbidities__aids</td>\n",
+              "      <td>comorbidities__aids</td>\n",
+              "      <td>[{'option_code': 'False', 'option_description'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3830</th>\n",
+              "      <td>c4kc_kits</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.c4kc_kit...</td>\n",
+              "      <td>c4kc_kits_clinical</td>\n",
+              "      <td>intraoperative_complications__cardiac_event</td>\n",
+              "      <td>intraoperative_complications__cardiac_event</td>\n",
+              "      <td>[{'option_code': 'False', 'option_description'...</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ad8384cf-190c-4879-842c-66a9ba0c6c4f')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-ad8384cf-190c-4879-842c-66a9ba0c6c4f button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-ad8384cf-190c-4879-842c-66a9ba0c6c4f');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "<div id=\"df-6915e002-eb06-45d4-8395-839d65258c39\">\n",
+              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-6915e002-eb06-45d4-8395-839d65258c39')\"\n",
+              "            title=\"Suggest charts\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "     width=\"24px\">\n",
+              "    <g>\n",
+              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+              "    </g>\n",
+              "</svg>\n",
+              "  </button>\n",
+              "\n",
+              "<style>\n",
+              "  .colab-df-quickchart {\n",
+              "      --bg-color: #E8F0FE;\n",
+              "      --fill-color: #1967D2;\n",
+              "      --hover-bg-color: #E2EBFA;\n",
+              "      --hover-fill-color: #174EA6;\n",
+              "      --disabled-fill-color: #AAA;\n",
+              "      --disabled-bg-color: #DDD;\n",
+              "  }\n",
+              "\n",
+              "  [theme=dark] .colab-df-quickchart {\n",
+              "      --bg-color: #3B4455;\n",
+              "      --fill-color: #D2E3FC;\n",
+              "      --hover-bg-color: #434B5C;\n",
+              "      --hover-fill-color: #FFFFFF;\n",
+              "      --disabled-bg-color: #3B4455;\n",
+              "      --disabled-fill-color: #666;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart {\n",
+              "    background-color: var(--bg-color);\n",
+              "    border: none;\n",
+              "    border-radius: 50%;\n",
+              "    cursor: pointer;\n",
+              "    display: none;\n",
+              "    fill: var(--fill-color);\n",
+              "    height: 32px;\n",
+              "    padding: 0;\n",
+              "    width: 32px;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart:hover {\n",
+              "    background-color: var(--hover-bg-color);\n",
+              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "    fill: var(--button-hover-fill-color);\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart-complete:disabled,\n",
+              "  .colab-df-quickchart-complete:disabled:hover {\n",
+              "    background-color: var(--disabled-bg-color);\n",
+              "    fill: var(--disabled-fill-color);\n",
+              "    box-shadow: none;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-spinner {\n",
+              "    border: 2px solid var(--fill-color);\n",
+              "    border-color: transparent;\n",
+              "    border-bottom-color: var(--fill-color);\n",
+              "    animation:\n",
+              "      spin 1s steps(1) infinite;\n",
+              "  }\n",
+              "\n",
+              "  @keyframes spin {\n",
+              "    0% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "      border-left-color: var(--fill-color);\n",
+              "    }\n",
+              "    20% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    30% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    40% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    60% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    80% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "    90% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "  }\n",
+              "</style>\n",
+              "\n",
+              "  <script>\n",
+              "    async function quickchart(key) {\n",
+              "      const quickchartButtonEl =\n",
+              "        document.querySelector('#' + key + ' button');\n",
+              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+              "      try {\n",
+              "        const charts = await google.colab.kernel.invokeFunction(\n",
+              "            'suggestCharts', [key], {});\n",
+              "      } catch (error) {\n",
+              "        console.error('Error during call to suggestCharts:', error);\n",
+              "      }\n",
+              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+              "    }\n",
+              "    (() => {\n",
+              "      let quickchartButtonEl =\n",
+              "        document.querySelector('#df-6915e002-eb06-45d4-8395-839d65258c39 button');\n",
+              "      quickchartButtonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "    })();\n",
+              "  </script>\n",
+              "</div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "summary": "{\n  \"name\": \"c\",\n  \"rows\": 3,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"c4kc_kits\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"bigquery-public-data.idc_v20_clinical.c4kc_kits_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"short_table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"c4kc_kits_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"comorbidities__dementia\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"comorbidities__dementia\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 8
+        }
       ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-a4ddfc88-bd0d-4633-9a1c-a17d90797845\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>collection_id</th>\n",
-       "      <th>short_table_name</th>\n",
-       "      <th>column</th>\n",
-       "      <th>column_label</th>\n",
-       "      <th>values</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>0</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>source_batch</td>\n",
-       "      <td>idc_provenance_source_batch</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': No...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>t0</td>\n",
-       "      <td>T0 (baseline) MRI study included in collection</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>t1</td>\n",
-       "      <td>T1 (early-Tx) MRI study included in collection</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>t2</td>\n",
-       "      <td>T2 (inter-regimen) MRI study included in colle...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>4</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>t3</td>\n",
-       "      <td>T3 (pre-surgery) MRI study included in collection</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>5</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>bmmr2_train</td>\n",
-       "      <td>Patient included in the BMMR2 challenge traini...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>6</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>bmmr2_test</td>\n",
-       "      <td>Patient included in the BMMR2 challenge test c...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>primary_aim_t0</td>\n",
-       "      <td>T0 MRI study included in the ACRIN-6698 primar...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>primary_aim_t1</td>\n",
-       "      <td>T1 MRI study included in the ACRIN-6698 primar...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>primary_aim_t2</td>\n",
-       "      <td>T2 MRI study included in the ACRIN-6698 primar...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>10</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>primary_aim_t3</td>\n",
-       "      <td>T3 MRI study included in the ACRIN-6698 primar...</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>11</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>analysis_cohort</td>\n",
-       "      <td>Patient in primary analysis cohort (n=242)</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>12</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>eligigible</td>\n",
-       "      <td>eligigible</td>\n",
-       "      <td>[{'option_code': '0', 'option_description': No...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>13</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>test_retest</td>\n",
-       "      <td>Study time point that includes test/retest DWI...</td>\n",
-       "      <td>[{'option_code': 'T0', 'option_description': N...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>14</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_pass_fail_t0</td>\n",
-       "      <td>T0 MRI DWI Quality control assessment:</td>\n",
-       "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>15</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_pass_fail_t1</td>\n",
-       "      <td>T0 MRI DWI Quality control assessment:</td>\n",
-       "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>16</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_pass_fail_t2</td>\n",
-       "      <td>T0 MRI DWI Quality control assessment:</td>\n",
-       "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>17</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_pass_fail_t3</td>\n",
-       "      <td>T0 MRI DWI Quality control assessment:</td>\n",
-       "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>18</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_roi_confidence_t0</td>\n",
-       "      <td>T0 quality control confidence level for whole-...</td>\n",
-       "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>19</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_roi_confidence_t1</td>\n",
-       "      <td>T1 quality control confidence level for whole-...</td>\n",
-       "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>20</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_roi_confidence_t2</td>\n",
-       "      <td>T2 quality control confidence level for whole-...</td>\n",
-       "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>21</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>qc_roi_confidence_t3</td>\n",
-       "      <td>T3 quality control confidence level for whole-...</td>\n",
-       "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>22</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>race</td>\n",
-       "      <td>Patient race:</td>\n",
-       "      <td>[{'option_code': 'White', 'option_description'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>23</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>ltype</td>\n",
-       "      <td>Index lesion type:</td>\n",
-       "      <td>[{'option_code': 'Single mass', 'option_descri...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>24</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>hrher4g</td>\n",
-       "      <td>Hormone receptor (HR) and HER2 status:</td>\n",
-       "      <td>[{'option_code': 'HR + / HER2 +', 'option_desc...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>25</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>sbrgrade</td>\n",
-       "      <td>Tumor grade:</td>\n",
-       "      <td>[{'option_code': 'I (Low)', 'option_descriptio...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>26</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>pcr</td>\n",
-       "      <td>Pathologic complete response (pCR) at surgery</td>\n",
-       "      <td>[{'option_code': 'non-pCR', 'option_descriptio...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>27</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>dicom_patient_id</td>\n",
-       "      <td>idc_provenance_dicom_patient_id</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>28</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>i_spy_2_research_id</td>\n",
-       "      <td>Double-blinded patient ID number from I-SPY 2</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>29</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>tcia_patient_id</td>\n",
-       "      <td>Patient identification string in TCIA collecti...</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>30</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>age</td>\n",
-       "      <td>Age in years at enrollment</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>31</th>\n",
-       "      <td>acrin_6698</td>\n",
-       "      <td>acrin_6698_clinical</td>\n",
-       "      <td>mrld</td>\n",
-       "      <td>MRI measured longest diameter (cm) at baseline...</td>\n",
-       "      <td>[]</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a4ddfc88-bd0d-4633-9a1c-a17d90797845')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-a4ddfc88-bd0d-4633-9a1c-a17d90797845 button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-a4ddfc88-bd0d-4633-9a1c-a17d90797845');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "<div id=\"df-0b18f794-9ae2-443c-8af2-991aba775f79\">\n",
-       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-0b18f794-9ae2-443c-8af2-991aba775f79')\"\n",
-       "            title=\"Suggest charts\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "     width=\"24px\">\n",
-       "    <g>\n",
-       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
-       "    </g>\n",
-       "</svg>\n",
-       "  </button>\n",
-       "\n",
-       "<style>\n",
-       "  .colab-df-quickchart {\n",
-       "      --bg-color: #E8F0FE;\n",
-       "      --fill-color: #1967D2;\n",
-       "      --hover-bg-color: #E2EBFA;\n",
-       "      --hover-fill-color: #174EA6;\n",
-       "      --disabled-fill-color: #AAA;\n",
-       "      --disabled-bg-color: #DDD;\n",
-       "  }\n",
-       "\n",
-       "  [theme=dark] .colab-df-quickchart {\n",
-       "      --bg-color: #3B4455;\n",
-       "      --fill-color: #D2E3FC;\n",
-       "      --hover-bg-color: #434B5C;\n",
-       "      --hover-fill-color: #FFFFFF;\n",
-       "      --disabled-bg-color: #3B4455;\n",
-       "      --disabled-fill-color: #666;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart {\n",
-       "    background-color: var(--bg-color);\n",
-       "    border: none;\n",
-       "    border-radius: 50%;\n",
-       "    cursor: pointer;\n",
-       "    display: none;\n",
-       "    fill: var(--fill-color);\n",
-       "    height: 32px;\n",
-       "    padding: 0;\n",
-       "    width: 32px;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart:hover {\n",
-       "    background-color: var(--hover-bg-color);\n",
-       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "    fill: var(--button-hover-fill-color);\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart-complete:disabled,\n",
-       "  .colab-df-quickchart-complete:disabled:hover {\n",
-       "    background-color: var(--disabled-bg-color);\n",
-       "    fill: var(--disabled-fill-color);\n",
-       "    box-shadow: none;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-spinner {\n",
-       "    border: 2px solid var(--fill-color);\n",
-       "    border-color: transparent;\n",
-       "    border-bottom-color: var(--fill-color);\n",
-       "    animation:\n",
-       "      spin 1s steps(1) infinite;\n",
-       "  }\n",
-       "\n",
-       "  @keyframes spin {\n",
-       "    0% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "      border-left-color: var(--fill-color);\n",
-       "    }\n",
-       "    20% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    30% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    40% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    60% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    80% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "    90% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "  }\n",
-       "</style>\n",
-       "\n",
-       "  <script>\n",
-       "    async function quickchart(key) {\n",
-       "      const quickchartButtonEl =\n",
-       "        document.querySelector('#' + key + ' button');\n",
-       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
-       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
-       "      try {\n",
-       "        const charts = await google.colab.kernel.invokeFunction(\n",
-       "            'suggestCharts', [key], {});\n",
-       "      } catch (error) {\n",
-       "        console.error('Error during call to suggestCharts:', error);\n",
-       "      }\n",
-       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
-       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
-       "    }\n",
-       "    (() => {\n",
-       "      let quickchartButtonEl =\n",
-       "        document.querySelector('#df-0b18f794-9ae2-443c-8af2-991aba775f79 button');\n",
-       "      quickchartButtonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "    })();\n",
-       "  </script>\n",
-       "</div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
+      "source": [
+        "c.clinical_index[c.clinical_index[\"collection_id\"] == \"c4kc_kits\"][:3]"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "voowbwXCeFBF"
+      },
+      "source": [
+        "**As a general rule of thumb**:\n",
+        "* when selecting specific columns from clinical tables, use `column` values\n",
+        "* when searching for concepts of interest in `column_metadata`, use `column_label`"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "gWgLxoraN3mY"
+      },
+      "source": [
+        "# Exploring IDC clinical data\n",
+        "\n",
+        "In the following sections of the notebook we go over some use cases to demonstrate various options for navigating IDC clinical data.\n",
+        "\n",
+        "As always, if you have a use case that is not addressed here, if you have suggestions or are confused - please start a discussion thread on the [IDC User forum](https://discourse.canceridc.dev/)!"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "03rwe7OfvsnV"
+      },
+      "source": [
+        "## Collection-focused exploration\n",
+        "\n",
+        "If you used clinical data stored in TCIA, perhaps you started with a specific collection of your interest, downloaded clinical data files for that collection, parsed them into your code. Let's go over those steps the IDC way!\n",
+        "\n",
+        "First, let's see which of the collections in IDC have clinical data in BigQuery. To do that we will reuse the pandas dataframe with the results of the query we executed earlier.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 9,
+      "metadata": {
+        "id": "TIZq1GbQoTIC",
+        "outputId": "ef45376f-8c6e-4eed-910f-a76e0badae46",
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "['acrin_6698',\n",
+              " 'acrin_contralateral_breast_mr',\n",
+              " 'acrin_flt_breast',\n",
+              " 'acrin_nsclc_fdg_pet',\n",
+              " 'adrenal_acc_ki67_seg',\n",
+              " 'advanced_mri_breast_lesions',\n",
+              " 'anti_pd_1_lung',\n",
+              " 'b_mode_and_ceus_liver',\n",
+              " 'breast_diagnosis',\n",
+              " 'breast_mri_nact_pilot',\n",
+              " 'c4kc_kits',\n",
+              " 'cc_tumor_heterogeneity',\n",
+              " 'cmmd',\n",
+              " 'colorectal_liver_metastases',\n",
+              " 'covid_19_ar',\n",
+              " 'covid_19_ny_sbu',\n",
+              " 'cptac_brca',\n",
+              " 'cptac_ccrcc',\n",
+              " 'cptac_coad',\n",
+              " 'cptac_gbm',\n",
+              " 'cptac_hnscc',\n",
+              " 'cptac_lscc',\n",
+              " 'cptac_luad',\n",
+              " 'cptac_ov',\n",
+              " 'cptac_pda',\n",
+              " 'cptac_ucec',\n",
+              " 'ctpred_sunitinib_pannet',\n",
+              " 'duke_breast_cancer_mri',\n",
+              " 'ea1141',\n",
+              " 'hcc_tace_seg',\n",
+              " 'htan_hms',\n",
+              " 'htan_ohsu',\n",
+              " 'htan_vanderbilt',\n",
+              " 'htan_wustl',\n",
+              " 'ispy1',\n",
+              " 'ispy2',\n",
+              " 'lidc_idri',\n",
+              " 'lung_fused_ct_pathology',\n",
+              " 'lung_pet_ct_dx',\n",
+              " 'mediastinal_lymph_node_seg',\n",
+              " 'midrc_ricord_1a',\n",
+              " 'midrc_ricord_1b',\n",
+              " 'midrc_ricord_1c',\n",
+              " 'nlst',\n",
+              " 'nsclc_radiogenomics',\n",
+              " 'nsclc_radiomics',\n",
+              " 'nsclc_radiomics_genomics',\n",
+              " 'nsclc_radiomics_interobserver1',\n",
+              " 'prostate_diagnosis',\n",
+              " 'prostatex',\n",
+              " 'qin_breast',\n",
+              " 'remind',\n",
+              " 'rider_lung_pet_ct',\n",
+              " 'rms_mutation_prediction',\n",
+              " 'soft_tissue_sarcoma',\n",
+              " 'spie_aapm_lung_ct_challenge',\n",
+              " 'spine_mets_ct_seg',\n",
+              " 'tcga_acc',\n",
+              " 'tcga_blca',\n",
+              " 'tcga_brca',\n",
+              " 'tcga_cesc',\n",
+              " 'tcga_chol',\n",
+              " 'tcga_coad',\n",
+              " 'tcga_dlbc',\n",
+              " 'tcga_esca',\n",
+              " 'tcga_gbm',\n",
+              " 'tcga_hnsc',\n",
+              " 'tcga_kich',\n",
+              " 'tcga_kirc',\n",
+              " 'tcga_kirp',\n",
+              " 'tcga_lgg',\n",
+              " 'tcga_lihc',\n",
+              " 'tcga_luad',\n",
+              " 'tcga_lusc',\n",
+              " 'tcga_meso',\n",
+              " 'tcga_ov',\n",
+              " 'tcga_paad',\n",
+              " 'tcga_pcpg',\n",
+              " 'tcga_prad',\n",
+              " 'tcga_read',\n",
+              " 'tcga_sarc',\n",
+              " 'tcga_skcm',\n",
+              " 'tcga_stad',\n",
+              " 'tcga_tgct',\n",
+              " 'tcga_thca',\n",
+              " 'tcga_thym',\n",
+              " 'tcga_ucec',\n",
+              " 'tcga_ucs',\n",
+              " 'tcga_uvm',\n",
+              " 'upenn_gbm']"
+            ]
+          },
+          "metadata": {},
+          "execution_count": 9
+        }
       ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "summary": "{\n  \"name\": \"acrin6698_clinical_columns[[\\\"collection_id\\\", \\\"short_table_name\\\", \\\"column\\\", \\\"column_label\\\", \\\"values\\\"]]\",\n  \"rows\": 32,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"acrin_6698\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"short_table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"acrin_6698_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 32,\n        \"samples\": [\n          \"tcia_patient_id\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 29,\n        \"samples\": [\n          \"Age in years at enrollment\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
-      }
-     },
-     "metadata": {},
-     "execution_count": 15
-    }
-   ],
-   "source": [
-    "acrin6698_clinical_columns[[\"collection_id\", \"short_table_name\", \"column\", \"column_label\", \"values\"]]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "CXy4mOdj428I"
-   },
-   "source": [
-    "## From attribute names to data: accessing clinical data at patient level\n",
-    "\n",
-    "`short_table_name` gives us the table stored locally that contains the values for the column described in the `column_metadata` row. All of the clinical data tables were downloaded as part of `idc-index` installation.\n",
-    "\n",
-    "Here is how we can load the clinical data from the `acrin_6698_clinical` table into a pandas dataframe."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "source": [
-    "acrin_6698_clinical_df = c.get_clinical_table(\"acrin_6698_clinical\")"
-   ],
-   "metadata": {
-    "id": "USHHDVjLDRN-"
-   },
-   "execution_count": 18,
-   "outputs": []
-  },
-  {
-   "cell_type": "markdown",
-   "source": [
-    "Given the information available in the per-collection clinical data, we can proceed with selecting a subset of patients that meet the criteria of your interest. As an example, the following query will select all of the distinct combination of patient ID and tumor grade, as defined by the `sbrgrade` column contents.\n"
-   ],
-   "metadata": {
-    "id": "KQvhHGuTP3wK"
-   }
-  },
-  {
-   "cell_type": "code",
-   "source": [
-    "# prompt: select distinct combinations of values in dicom_patient_id and sbrgrade columns from the acrin_6698_clinical_df pandas dataframe\n",
-    "acrin_6698_tumors = acrin_6698_clinical_df[['dicom_patient_id', 'sbrgrade']].drop_duplicates()\n",
-    "acrin_6698_tumors"
-   ],
-   "metadata": {
-    "id": "CKYHeu1pQK8V",
-    "outputId": "4ceb6c77-d9fe-4a3a-8647-146fdd21cd1d",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 479
-    }
-   },
-   "execution_count": 19,
-   "outputs": [
+      "source": [
+        "c.clinical_index[\"collection_id\"].unique().tolist()"
+      ]
+    },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "_KoEJ_Wvwa79"
+      },
+      "source": [
+        "If you are interested in what clinical data is available for the specific collection, you can select only the rows corresponding to that collection in the `column_metadata` table. Here we select a subset of columns to improve readability of the dataframe.\n",
+        "\n",
+        "Note that for some collections, clinical data sheets are accompanied by dictionaries, which formalize the values encountered. Examples of such collections are [ISPY1](https://wiki.cancerimagingarchive.net/display/Public/ISPY1) or ACRIN trials.\n",
+        "\n",
+        "For many other collections there are no such dictionaries available. In those situations, the values you will see in the `values` columns have been derived by examining the distinct values encountered in the clinical data sheets.\n",
+        "\n",
+        "In the following we look at the clinical data columns (\"dictionary terms\") for the [ACRIN 6698 collection](https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=50135447) mentioned earlier."
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "      dicom_patient_id           sbrgrade\n",
-       "0    ACRIN-6698-453236  II (Intermediate)\n",
-       "1    ACRIN-6698-229047                 NA\n",
-       "2    ACRIN-6698-384705  II (Intermediate)\n",
-       "3    ACRIN-6698-415631         III (High)\n",
-       "4    ACRIN-6698-793283                 NA\n",
-       "..                 ...                ...\n",
-       "380  ACRIN-6698-765671  II (Intermediate)\n",
-       "381  ACRIN-6698-962153         III (High)\n",
-       "382  ACRIN-6698-711476  II (Intermediate)\n",
-       "383  ACRIN-6698-361701  II (Intermediate)\n",
-       "384  ACRIN-6698-241998            I (Low)\n",
-       "\n",
-       "[385 rows x 2 columns]"
+      "cell_type": "code",
+      "execution_count": 14,
+      "metadata": {
+        "id": "zrtRWr_SwmIJ",
+        "outputId": "b16a2902-be23-4b1d-bb96-2151271eaa4d",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 1000
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "   collection_id                                         table_name  \\\n",
+              "0     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "1     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "2     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "3     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "4     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "5     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "6     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "7     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "8     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "9     acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "10    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "11    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "12    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "13    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "14    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "15    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "16    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "17    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "18    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "19    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "20    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "21    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "22    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "23    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "24    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "25    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "26    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "27    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "28    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "29    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "30    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "31    acrin_6698  bigquery-public-data.idc_v20_clinical.acrin_66...   \n",
+              "\n",
+              "       short_table_name                column  \\\n",
+              "0   acrin_6698_clinical          source_batch   \n",
+              "1   acrin_6698_clinical                    t0   \n",
+              "2   acrin_6698_clinical                    t1   \n",
+              "3   acrin_6698_clinical                    t2   \n",
+              "4   acrin_6698_clinical                    t3   \n",
+              "5   acrin_6698_clinical           bmmr2_train   \n",
+              "6   acrin_6698_clinical            bmmr2_test   \n",
+              "7   acrin_6698_clinical        primary_aim_t0   \n",
+              "8   acrin_6698_clinical        primary_aim_t1   \n",
+              "9   acrin_6698_clinical        primary_aim_t2   \n",
+              "10  acrin_6698_clinical        primary_aim_t3   \n",
+              "11  acrin_6698_clinical       analysis_cohort   \n",
+              "12  acrin_6698_clinical            eligigible   \n",
+              "13  acrin_6698_clinical           test_retest   \n",
+              "14  acrin_6698_clinical       qc_pass_fail_t0   \n",
+              "15  acrin_6698_clinical       qc_pass_fail_t1   \n",
+              "16  acrin_6698_clinical       qc_pass_fail_t2   \n",
+              "17  acrin_6698_clinical       qc_pass_fail_t3   \n",
+              "18  acrin_6698_clinical  qc_roi_confidence_t0   \n",
+              "19  acrin_6698_clinical  qc_roi_confidence_t1   \n",
+              "20  acrin_6698_clinical  qc_roi_confidence_t2   \n",
+              "21  acrin_6698_clinical  qc_roi_confidence_t3   \n",
+              "22  acrin_6698_clinical                  race   \n",
+              "23  acrin_6698_clinical                 ltype   \n",
+              "24  acrin_6698_clinical               hrher4g   \n",
+              "25  acrin_6698_clinical              sbrgrade   \n",
+              "26  acrin_6698_clinical                   pcr   \n",
+              "27  acrin_6698_clinical      dicom_patient_id   \n",
+              "28  acrin_6698_clinical   i_spy_2_research_id   \n",
+              "29  acrin_6698_clinical       tcia_patient_id   \n",
+              "30  acrin_6698_clinical                   age   \n",
+              "31  acrin_6698_clinical                  mrld   \n",
+              "\n",
+              "                                         column_label  \\\n",
+              "0                         idc_provenance_source_batch   \n",
+              "1      T0 (baseline) MRI study included in collection   \n",
+              "2      T1 (early-Tx) MRI study included in collection   \n",
+              "3   T2 (inter-regimen) MRI study included in colle...   \n",
+              "4   T3 (pre-surgery) MRI study included in collection   \n",
+              "5   Patient included in the BMMR2 challenge traini...   \n",
+              "6   Patient included in the BMMR2 challenge test c...   \n",
+              "7   T0 MRI study included in the ACRIN-6698 primar...   \n",
+              "8   T1 MRI study included in the ACRIN-6698 primar...   \n",
+              "9   T2 MRI study included in the ACRIN-6698 primar...   \n",
+              "10  T3 MRI study included in the ACRIN-6698 primar...   \n",
+              "11         Patient in primary analysis cohort (n=242)   \n",
+              "12                                         eligigible   \n",
+              "13  Study time point that includes test/retest DWI...   \n",
+              "14             T0 MRI DWI Quality control assessment:   \n",
+              "15             T0 MRI DWI Quality control assessment:   \n",
+              "16             T0 MRI DWI Quality control assessment:   \n",
+              "17             T0 MRI DWI Quality control assessment:   \n",
+              "18  T0 quality control confidence level for whole-...   \n",
+              "19  T1 quality control confidence level for whole-...   \n",
+              "20  T2 quality control confidence level for whole-...   \n",
+              "21  T3 quality control confidence level for whole-...   \n",
+              "22                                      Patient race:   \n",
+              "23                                 Index lesion type:   \n",
+              "24             Hormone receptor (HR) and HER2 status:   \n",
+              "25                                       Tumor grade:   \n",
+              "26      Pathologic complete response (pCR) at surgery   \n",
+              "27                    idc_provenance_dicom_patient_id   \n",
+              "28      Double-blinded patient ID number from I-SPY 2   \n",
+              "29  Patient identification string in TCIA collecti...   \n",
+              "30                         Age in years at enrollment   \n",
+              "31  MRI measured longest diameter (cm) at baseline...   \n",
+              "\n",
+              "                                               values  \n",
+              "0   [{'option_code': '0', 'option_description': No...  \n",
+              "1   [{'option_code': '0', 'option_description': ' ...  \n",
+              "2   [{'option_code': '0', 'option_description': ' ...  \n",
+              "3   [{'option_code': '0', 'option_description': ' ...  \n",
+              "4   [{'option_code': '0', 'option_description': ' ...  \n",
+              "5   [{'option_code': '0', 'option_description': ' ...  \n",
+              "6   [{'option_code': '0', 'option_description': ' ...  \n",
+              "7   [{'option_code': '0', 'option_description': ' ...  \n",
+              "8   [{'option_code': '0', 'option_description': ' ...  \n",
+              "9   [{'option_code': '0', 'option_description': ' ...  \n",
+              "10  [{'option_code': '0', 'option_description': ' ...  \n",
+              "11  [{'option_code': '0', 'option_description': ' ...  \n",
+              "12  [{'option_code': '0', 'option_description': No...  \n",
+              "13  [{'option_code': 'T0', 'option_description': N...  \n",
+              "14  [{'option_code': 'PASS', 'option_description':...  \n",
+              "15  [{'option_code': 'PASS', 'option_description':...  \n",
+              "16  [{'option_code': 'PASS', 'option_description':...  \n",
+              "17  [{'option_code': 'PASS', 'option_description':...  \n",
+              "18  [{'option_code': 'HIGH', 'option_description':...  \n",
+              "19  [{'option_code': 'HIGH', 'option_description':...  \n",
+              "20  [{'option_code': 'HIGH', 'option_description':...  \n",
+              "21  [{'option_code': 'HIGH', 'option_description':...  \n",
+              "22  [{'option_code': 'White', 'option_description'...  \n",
+              "23  [{'option_code': 'Single mass', 'option_descri...  \n",
+              "24  [{'option_code': 'HR + / HER2 +', 'option_desc...  \n",
+              "25  [{'option_code': 'I (Low)', 'option_descriptio...  \n",
+              "26  [{'option_code': 'non-pCR', 'option_descriptio...  \n",
+              "27                                                 []  \n",
+              "28                                                 []  \n",
+              "29                                                 []  \n",
+              "30                                                 []  \n",
+              "31                                                 []  "
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-f21ecf89-f728-408b-a5d4-82b2b9721c87\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>collection_id</th>\n",
+              "      <th>table_name</th>\n",
+              "      <th>short_table_name</th>\n",
+              "      <th>column</th>\n",
+              "      <th>column_label</th>\n",
+              "      <th>values</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>0</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>source_batch</td>\n",
+              "      <td>idc_provenance_source_batch</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': No...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>t0</td>\n",
+              "      <td>T0 (baseline) MRI study included in collection</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>2</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>t1</td>\n",
+              "      <td>T1 (early-Tx) MRI study included in collection</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>t2</td>\n",
+              "      <td>T2 (inter-regimen) MRI study included in colle...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>4</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>t3</td>\n",
+              "      <td>T3 (pre-surgery) MRI study included in collection</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>5</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>bmmr2_train</td>\n",
+              "      <td>Patient included in the BMMR2 challenge traini...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>6</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>bmmr2_test</td>\n",
+              "      <td>Patient included in the BMMR2 challenge test c...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>primary_aim_t0</td>\n",
+              "      <td>T0 MRI study included in the ACRIN-6698 primar...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>primary_aim_t1</td>\n",
+              "      <td>T1 MRI study included in the ACRIN-6698 primar...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>primary_aim_t2</td>\n",
+              "      <td>T2 MRI study included in the ACRIN-6698 primar...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>10</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>primary_aim_t3</td>\n",
+              "      <td>T3 MRI study included in the ACRIN-6698 primar...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>11</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>analysis_cohort</td>\n",
+              "      <td>Patient in primary analysis cohort (n=242)</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>12</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>eligigible</td>\n",
+              "      <td>eligigible</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': No...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>13</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>test_retest</td>\n",
+              "      <td>Study time point that includes test/retest DWI...</td>\n",
+              "      <td>[{'option_code': 'T0', 'option_description': N...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>14</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_pass_fail_t0</td>\n",
+              "      <td>T0 MRI DWI Quality control assessment:</td>\n",
+              "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>15</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_pass_fail_t1</td>\n",
+              "      <td>T0 MRI DWI Quality control assessment:</td>\n",
+              "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>16</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_pass_fail_t2</td>\n",
+              "      <td>T0 MRI DWI Quality control assessment:</td>\n",
+              "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>17</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_pass_fail_t3</td>\n",
+              "      <td>T0 MRI DWI Quality control assessment:</td>\n",
+              "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>18</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_roi_confidence_t0</td>\n",
+              "      <td>T0 quality control confidence level for whole-...</td>\n",
+              "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>19</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_roi_confidence_t1</td>\n",
+              "      <td>T1 quality control confidence level for whole-...</td>\n",
+              "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>20</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_roi_confidence_t2</td>\n",
+              "      <td>T2 quality control confidence level for whole-...</td>\n",
+              "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>21</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_roi_confidence_t3</td>\n",
+              "      <td>T3 quality control confidence level for whole-...</td>\n",
+              "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>22</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>race</td>\n",
+              "      <td>Patient race:</td>\n",
+              "      <td>[{'option_code': 'White', 'option_description'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>23</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>ltype</td>\n",
+              "      <td>Index lesion type:</td>\n",
+              "      <td>[{'option_code': 'Single mass', 'option_descri...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>24</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>hrher4g</td>\n",
+              "      <td>Hormone receptor (HR) and HER2 status:</td>\n",
+              "      <td>[{'option_code': 'HR + / HER2 +', 'option_desc...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>25</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>sbrgrade</td>\n",
+              "      <td>Tumor grade:</td>\n",
+              "      <td>[{'option_code': 'I (Low)', 'option_descriptio...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>26</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>pcr</td>\n",
+              "      <td>Pathologic complete response (pCR) at surgery</td>\n",
+              "      <td>[{'option_code': 'non-pCR', 'option_descriptio...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>27</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>dicom_patient_id</td>\n",
+              "      <td>idc_provenance_dicom_patient_id</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>28</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>i_spy_2_research_id</td>\n",
+              "      <td>Double-blinded patient ID number from I-SPY 2</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>29</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>tcia_patient_id</td>\n",
+              "      <td>Patient identification string in TCIA collecti...</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>30</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>age</td>\n",
+              "      <td>Age in years at enrollment</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>31</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.acrin_66...</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>mrld</td>\n",
+              "      <td>MRI measured longest diameter (cm) at baseline...</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f21ecf89-f728-408b-a5d4-82b2b9721c87')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-f21ecf89-f728-408b-a5d4-82b2b9721c87 button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-f21ecf89-f728-408b-a5d4-82b2b9721c87');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "<div id=\"df-74b7689b-126c-49f8-8297-8f4e4d6f02c4\">\n",
+              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-74b7689b-126c-49f8-8297-8f4e4d6f02c4')\"\n",
+              "            title=\"Suggest charts\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "     width=\"24px\">\n",
+              "    <g>\n",
+              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+              "    </g>\n",
+              "</svg>\n",
+              "  </button>\n",
+              "\n",
+              "<style>\n",
+              "  .colab-df-quickchart {\n",
+              "      --bg-color: #E8F0FE;\n",
+              "      --fill-color: #1967D2;\n",
+              "      --hover-bg-color: #E2EBFA;\n",
+              "      --hover-fill-color: #174EA6;\n",
+              "      --disabled-fill-color: #AAA;\n",
+              "      --disabled-bg-color: #DDD;\n",
+              "  }\n",
+              "\n",
+              "  [theme=dark] .colab-df-quickchart {\n",
+              "      --bg-color: #3B4455;\n",
+              "      --fill-color: #D2E3FC;\n",
+              "      --hover-bg-color: #434B5C;\n",
+              "      --hover-fill-color: #FFFFFF;\n",
+              "      --disabled-bg-color: #3B4455;\n",
+              "      --disabled-fill-color: #666;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart {\n",
+              "    background-color: var(--bg-color);\n",
+              "    border: none;\n",
+              "    border-radius: 50%;\n",
+              "    cursor: pointer;\n",
+              "    display: none;\n",
+              "    fill: var(--fill-color);\n",
+              "    height: 32px;\n",
+              "    padding: 0;\n",
+              "    width: 32px;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart:hover {\n",
+              "    background-color: var(--hover-bg-color);\n",
+              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "    fill: var(--button-hover-fill-color);\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart-complete:disabled,\n",
+              "  .colab-df-quickchart-complete:disabled:hover {\n",
+              "    background-color: var(--disabled-bg-color);\n",
+              "    fill: var(--disabled-fill-color);\n",
+              "    box-shadow: none;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-spinner {\n",
+              "    border: 2px solid var(--fill-color);\n",
+              "    border-color: transparent;\n",
+              "    border-bottom-color: var(--fill-color);\n",
+              "    animation:\n",
+              "      spin 1s steps(1) infinite;\n",
+              "  }\n",
+              "\n",
+              "  @keyframes spin {\n",
+              "    0% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "      border-left-color: var(--fill-color);\n",
+              "    }\n",
+              "    20% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    30% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    40% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    60% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    80% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "    90% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "  }\n",
+              "</style>\n",
+              "\n",
+              "  <script>\n",
+              "    async function quickchart(key) {\n",
+              "      const quickchartButtonEl =\n",
+              "        document.querySelector('#' + key + ' button');\n",
+              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+              "      try {\n",
+              "        const charts = await google.colab.kernel.invokeFunction(\n",
+              "            'suggestCharts', [key], {});\n",
+              "      } catch (error) {\n",
+              "        console.error('Error during call to suggestCharts:', error);\n",
+              "      }\n",
+              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+              "    }\n",
+              "    (() => {\n",
+              "      let quickchartButtonEl =\n",
+              "        document.querySelector('#df-74b7689b-126c-49f8-8297-8f4e4d6f02c4 button');\n",
+              "      quickchartButtonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "    })();\n",
+              "  </script>\n",
+              "</div>\n",
+              "\n",
+              "  <div id=\"id_6e2ef9d9-d7bb-46eb-ba22-f750713f2f77\">\n",
+              "    <style>\n",
+              "      .colab-df-generate {\n",
+              "        background-color: #E8F0FE;\n",
+              "        border: none;\n",
+              "        border-radius: 50%;\n",
+              "        cursor: pointer;\n",
+              "        display: none;\n",
+              "        fill: #1967D2;\n",
+              "        height: 32px;\n",
+              "        padding: 0 0 0 0;\n",
+              "        width: 32px;\n",
+              "      }\n",
+              "\n",
+              "      .colab-df-generate:hover {\n",
+              "        background-color: #E2EBFA;\n",
+              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "        fill: #174EA6;\n",
+              "      }\n",
+              "\n",
+              "      [theme=dark] .colab-df-generate {\n",
+              "        background-color: #3B4455;\n",
+              "        fill: #D2E3FC;\n",
+              "      }\n",
+              "\n",
+              "      [theme=dark] .colab-df-generate:hover {\n",
+              "        background-color: #434B5C;\n",
+              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "        fill: #FFFFFF;\n",
+              "      }\n",
+              "    </style>\n",
+              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('acrin6698_clinical_columns')\"\n",
+              "            title=\"Generate code using this dataframe.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "       width=\"24px\">\n",
+              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "    <script>\n",
+              "      (() => {\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#id_6e2ef9d9-d7bb-46eb-ba22-f750713f2f77 button.colab-df-generate');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      buttonEl.onclick = () => {\n",
+              "        google.colab.notebook.generateWithVariable('acrin6698_clinical_columns');\n",
+              "      }\n",
+              "      })();\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "variable_name": "acrin6698_clinical_columns",
+              "summary": "{\n  \"name\": \"acrin6698_clinical_columns\",\n  \"rows\": 32,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"acrin_6698\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"bigquery-public-data.idc_v20_clinical.acrin_6698_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"short_table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"acrin_6698_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 32,\n        \"samples\": [\n          \"tcia_patient_id\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 29,\n        \"samples\": [\n          \"Age in years at enrollment\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 14
+        }
       ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-c8d4f416-aa84-4c49-a260-f295bebf58a2\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>dicom_patient_id</th>\n",
-       "      <th>sbrgrade</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>0</th>\n",
-       "      <td>ACRIN-6698-453236</td>\n",
-       "      <td>II (Intermediate)</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1</th>\n",
-       "      <td>ACRIN-6698-229047</td>\n",
-       "      <td>NA</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2</th>\n",
-       "      <td>ACRIN-6698-384705</td>\n",
-       "      <td>II (Intermediate)</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3</th>\n",
-       "      <td>ACRIN-6698-415631</td>\n",
-       "      <td>III (High)</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>4</th>\n",
-       "      <td>ACRIN-6698-793283</td>\n",
-       "      <td>NA</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>...</th>\n",
-       "      <td>...</td>\n",
-       "      <td>...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>380</th>\n",
-       "      <td>ACRIN-6698-765671</td>\n",
-       "      <td>II (Intermediate)</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>381</th>\n",
-       "      <td>ACRIN-6698-962153</td>\n",
-       "      <td>III (High)</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>382</th>\n",
-       "      <td>ACRIN-6698-711476</td>\n",
-       "      <td>II (Intermediate)</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>383</th>\n",
-       "      <td>ACRIN-6698-361701</td>\n",
-       "      <td>II (Intermediate)</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>384</th>\n",
-       "      <td>ACRIN-6698-241998</td>\n",
-       "      <td>I (Low)</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "<p>385 rows × 2 columns</p>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c8d4f416-aa84-4c49-a260-f295bebf58a2')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-c8d4f416-aa84-4c49-a260-f295bebf58a2 button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-c8d4f416-aa84-4c49-a260-f295bebf58a2');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "<div id=\"df-7e3c6a5b-97f9-4779-83ab-cc4658119bb7\">\n",
-       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-7e3c6a5b-97f9-4779-83ab-cc4658119bb7')\"\n",
-       "            title=\"Suggest charts\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "     width=\"24px\">\n",
-       "    <g>\n",
-       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
-       "    </g>\n",
-       "</svg>\n",
-       "  </button>\n",
-       "\n",
-       "<style>\n",
-       "  .colab-df-quickchart {\n",
-       "      --bg-color: #E8F0FE;\n",
-       "      --fill-color: #1967D2;\n",
-       "      --hover-bg-color: #E2EBFA;\n",
-       "      --hover-fill-color: #174EA6;\n",
-       "      --disabled-fill-color: #AAA;\n",
-       "      --disabled-bg-color: #DDD;\n",
-       "  }\n",
-       "\n",
-       "  [theme=dark] .colab-df-quickchart {\n",
-       "      --bg-color: #3B4455;\n",
-       "      --fill-color: #D2E3FC;\n",
-       "      --hover-bg-color: #434B5C;\n",
-       "      --hover-fill-color: #FFFFFF;\n",
-       "      --disabled-bg-color: #3B4455;\n",
-       "      --disabled-fill-color: #666;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart {\n",
-       "    background-color: var(--bg-color);\n",
-       "    border: none;\n",
-       "    border-radius: 50%;\n",
-       "    cursor: pointer;\n",
-       "    display: none;\n",
-       "    fill: var(--fill-color);\n",
-       "    height: 32px;\n",
-       "    padding: 0;\n",
-       "    width: 32px;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart:hover {\n",
-       "    background-color: var(--hover-bg-color);\n",
-       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "    fill: var(--button-hover-fill-color);\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart-complete:disabled,\n",
-       "  .colab-df-quickchart-complete:disabled:hover {\n",
-       "    background-color: var(--disabled-bg-color);\n",
-       "    fill: var(--disabled-fill-color);\n",
-       "    box-shadow: none;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-spinner {\n",
-       "    border: 2px solid var(--fill-color);\n",
-       "    border-color: transparent;\n",
-       "    border-bottom-color: var(--fill-color);\n",
-       "    animation:\n",
-       "      spin 1s steps(1) infinite;\n",
-       "  }\n",
-       "\n",
-       "  @keyframes spin {\n",
-       "    0% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "      border-left-color: var(--fill-color);\n",
-       "    }\n",
-       "    20% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    30% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    40% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    60% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    80% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "    90% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "  }\n",
-       "</style>\n",
-       "\n",
-       "  <script>\n",
-       "    async function quickchart(key) {\n",
-       "      const quickchartButtonEl =\n",
-       "        document.querySelector('#' + key + ' button');\n",
-       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
-       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
-       "      try {\n",
-       "        const charts = await google.colab.kernel.invokeFunction(\n",
-       "            'suggestCharts', [key], {});\n",
-       "      } catch (error) {\n",
-       "        console.error('Error during call to suggestCharts:', error);\n",
-       "      }\n",
-       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
-       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
-       "    }\n",
-       "    (() => {\n",
-       "      let quickchartButtonEl =\n",
-       "        document.querySelector('#df-7e3c6a5b-97f9-4779-83ab-cc4658119bb7 button');\n",
-       "      quickchartButtonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "    })();\n",
-       "  </script>\n",
-       "</div>\n",
-       "\n",
-       "  <div id=\"id_1cb558c1-92b9-4f6e-908f-b9e6b7cfa343\">\n",
-       "    <style>\n",
-       "      .colab-df-generate {\n",
-       "        background-color: #E8F0FE;\n",
-       "        border: none;\n",
-       "        border-radius: 50%;\n",
-       "        cursor: pointer;\n",
-       "        display: none;\n",
-       "        fill: #1967D2;\n",
-       "        height: 32px;\n",
-       "        padding: 0 0 0 0;\n",
-       "        width: 32px;\n",
-       "      }\n",
-       "\n",
-       "      .colab-df-generate:hover {\n",
-       "        background-color: #E2EBFA;\n",
-       "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "        fill: #174EA6;\n",
-       "      }\n",
-       "\n",
-       "      [theme=dark] .colab-df-generate {\n",
-       "        background-color: #3B4455;\n",
-       "        fill: #D2E3FC;\n",
-       "      }\n",
-       "\n",
-       "      [theme=dark] .colab-df-generate:hover {\n",
-       "        background-color: #434B5C;\n",
-       "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "        fill: #FFFFFF;\n",
-       "      }\n",
-       "    </style>\n",
-       "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('distinct_combinations')\"\n",
-       "            title=\"Generate code using this dataframe.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "       width=\"24px\">\n",
-       "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "    <script>\n",
-       "      (() => {\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#id_1cb558c1-92b9-4f6e-908f-b9e6b7cfa343 button.colab-df-generate');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      buttonEl.onclick = () => {\n",
-       "        google.colab.notebook.generateWithVariable('distinct_combinations');\n",
-       "      }\n",
-       "      })();\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
+      "source": [
+        "acrin6698_clinical_columns = c.clinical_index[c.clinical_index[\"collection_id\"] == \"acrin_6698\"]\n",
+        "\n",
+        "acrin6698_clinical_columns"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "Dm5cAn73SQJW"
+      },
+      "source": [
+        "Here's how you can select just the specific columns in the table - this way it is easier to examine the data."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 15,
+      "metadata": {
+        "id": "4mY1gbR7ysxx",
+        "outputId": "62e65454-2faa-43f1-9155-c6779c94cec9",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 1000
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "   collection_id     short_table_name                column  \\\n",
+              "0     acrin_6698  acrin_6698_clinical          source_batch   \n",
+              "1     acrin_6698  acrin_6698_clinical                    t0   \n",
+              "2     acrin_6698  acrin_6698_clinical                    t1   \n",
+              "3     acrin_6698  acrin_6698_clinical                    t2   \n",
+              "4     acrin_6698  acrin_6698_clinical                    t3   \n",
+              "5     acrin_6698  acrin_6698_clinical           bmmr2_train   \n",
+              "6     acrin_6698  acrin_6698_clinical            bmmr2_test   \n",
+              "7     acrin_6698  acrin_6698_clinical        primary_aim_t0   \n",
+              "8     acrin_6698  acrin_6698_clinical        primary_aim_t1   \n",
+              "9     acrin_6698  acrin_6698_clinical        primary_aim_t2   \n",
+              "10    acrin_6698  acrin_6698_clinical        primary_aim_t3   \n",
+              "11    acrin_6698  acrin_6698_clinical       analysis_cohort   \n",
+              "12    acrin_6698  acrin_6698_clinical            eligigible   \n",
+              "13    acrin_6698  acrin_6698_clinical           test_retest   \n",
+              "14    acrin_6698  acrin_6698_clinical       qc_pass_fail_t0   \n",
+              "15    acrin_6698  acrin_6698_clinical       qc_pass_fail_t1   \n",
+              "16    acrin_6698  acrin_6698_clinical       qc_pass_fail_t2   \n",
+              "17    acrin_6698  acrin_6698_clinical       qc_pass_fail_t3   \n",
+              "18    acrin_6698  acrin_6698_clinical  qc_roi_confidence_t0   \n",
+              "19    acrin_6698  acrin_6698_clinical  qc_roi_confidence_t1   \n",
+              "20    acrin_6698  acrin_6698_clinical  qc_roi_confidence_t2   \n",
+              "21    acrin_6698  acrin_6698_clinical  qc_roi_confidence_t3   \n",
+              "22    acrin_6698  acrin_6698_clinical                  race   \n",
+              "23    acrin_6698  acrin_6698_clinical                 ltype   \n",
+              "24    acrin_6698  acrin_6698_clinical               hrher4g   \n",
+              "25    acrin_6698  acrin_6698_clinical              sbrgrade   \n",
+              "26    acrin_6698  acrin_6698_clinical                   pcr   \n",
+              "27    acrin_6698  acrin_6698_clinical      dicom_patient_id   \n",
+              "28    acrin_6698  acrin_6698_clinical   i_spy_2_research_id   \n",
+              "29    acrin_6698  acrin_6698_clinical       tcia_patient_id   \n",
+              "30    acrin_6698  acrin_6698_clinical                   age   \n",
+              "31    acrin_6698  acrin_6698_clinical                  mrld   \n",
+              "\n",
+              "                                         column_label  \\\n",
+              "0                         idc_provenance_source_batch   \n",
+              "1      T0 (baseline) MRI study included in collection   \n",
+              "2      T1 (early-Tx) MRI study included in collection   \n",
+              "3   T2 (inter-regimen) MRI study included in colle...   \n",
+              "4   T3 (pre-surgery) MRI study included in collection   \n",
+              "5   Patient included in the BMMR2 challenge traini...   \n",
+              "6   Patient included in the BMMR2 challenge test c...   \n",
+              "7   T0 MRI study included in the ACRIN-6698 primar...   \n",
+              "8   T1 MRI study included in the ACRIN-6698 primar...   \n",
+              "9   T2 MRI study included in the ACRIN-6698 primar...   \n",
+              "10  T3 MRI study included in the ACRIN-6698 primar...   \n",
+              "11         Patient in primary analysis cohort (n=242)   \n",
+              "12                                         eligigible   \n",
+              "13  Study time point that includes test/retest DWI...   \n",
+              "14             T0 MRI DWI Quality control assessment:   \n",
+              "15             T0 MRI DWI Quality control assessment:   \n",
+              "16             T0 MRI DWI Quality control assessment:   \n",
+              "17             T0 MRI DWI Quality control assessment:   \n",
+              "18  T0 quality control confidence level for whole-...   \n",
+              "19  T1 quality control confidence level for whole-...   \n",
+              "20  T2 quality control confidence level for whole-...   \n",
+              "21  T3 quality control confidence level for whole-...   \n",
+              "22                                      Patient race:   \n",
+              "23                                 Index lesion type:   \n",
+              "24             Hormone receptor (HR) and HER2 status:   \n",
+              "25                                       Tumor grade:   \n",
+              "26      Pathologic complete response (pCR) at surgery   \n",
+              "27                    idc_provenance_dicom_patient_id   \n",
+              "28      Double-blinded patient ID number from I-SPY 2   \n",
+              "29  Patient identification string in TCIA collecti...   \n",
+              "30                         Age in years at enrollment   \n",
+              "31  MRI measured longest diameter (cm) at baseline...   \n",
+              "\n",
+              "                                               values  \n",
+              "0   [{'option_code': '0', 'option_description': No...  \n",
+              "1   [{'option_code': '0', 'option_description': ' ...  \n",
+              "2   [{'option_code': '0', 'option_description': ' ...  \n",
+              "3   [{'option_code': '0', 'option_description': ' ...  \n",
+              "4   [{'option_code': '0', 'option_description': ' ...  \n",
+              "5   [{'option_code': '0', 'option_description': ' ...  \n",
+              "6   [{'option_code': '0', 'option_description': ' ...  \n",
+              "7   [{'option_code': '0', 'option_description': ' ...  \n",
+              "8   [{'option_code': '0', 'option_description': ' ...  \n",
+              "9   [{'option_code': '0', 'option_description': ' ...  \n",
+              "10  [{'option_code': '0', 'option_description': ' ...  \n",
+              "11  [{'option_code': '0', 'option_description': ' ...  \n",
+              "12  [{'option_code': '0', 'option_description': No...  \n",
+              "13  [{'option_code': 'T0', 'option_description': N...  \n",
+              "14  [{'option_code': 'PASS', 'option_description':...  \n",
+              "15  [{'option_code': 'PASS', 'option_description':...  \n",
+              "16  [{'option_code': 'PASS', 'option_description':...  \n",
+              "17  [{'option_code': 'PASS', 'option_description':...  \n",
+              "18  [{'option_code': 'HIGH', 'option_description':...  \n",
+              "19  [{'option_code': 'HIGH', 'option_description':...  \n",
+              "20  [{'option_code': 'HIGH', 'option_description':...  \n",
+              "21  [{'option_code': 'HIGH', 'option_description':...  \n",
+              "22  [{'option_code': 'White', 'option_description'...  \n",
+              "23  [{'option_code': 'Single mass', 'option_descri...  \n",
+              "24  [{'option_code': 'HR + / HER2 +', 'option_desc...  \n",
+              "25  [{'option_code': 'I (Low)', 'option_descriptio...  \n",
+              "26  [{'option_code': 'non-pCR', 'option_descriptio...  \n",
+              "27                                                 []  \n",
+              "28                                                 []  \n",
+              "29                                                 []  \n",
+              "30                                                 []  \n",
+              "31                                                 []  "
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-a4ddfc88-bd0d-4633-9a1c-a17d90797845\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>collection_id</th>\n",
+              "      <th>short_table_name</th>\n",
+              "      <th>column</th>\n",
+              "      <th>column_label</th>\n",
+              "      <th>values</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>0</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>source_batch</td>\n",
+              "      <td>idc_provenance_source_batch</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': No...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>t0</td>\n",
+              "      <td>T0 (baseline) MRI study included in collection</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>2</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>t1</td>\n",
+              "      <td>T1 (early-Tx) MRI study included in collection</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>t2</td>\n",
+              "      <td>T2 (inter-regimen) MRI study included in colle...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>4</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>t3</td>\n",
+              "      <td>T3 (pre-surgery) MRI study included in collection</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>5</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>bmmr2_train</td>\n",
+              "      <td>Patient included in the BMMR2 challenge traini...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>6</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>bmmr2_test</td>\n",
+              "      <td>Patient included in the BMMR2 challenge test c...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>primary_aim_t0</td>\n",
+              "      <td>T0 MRI study included in the ACRIN-6698 primar...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>primary_aim_t1</td>\n",
+              "      <td>T1 MRI study included in the ACRIN-6698 primar...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>primary_aim_t2</td>\n",
+              "      <td>T2 MRI study included in the ACRIN-6698 primar...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>10</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>primary_aim_t3</td>\n",
+              "      <td>T3 MRI study included in the ACRIN-6698 primar...</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>11</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>analysis_cohort</td>\n",
+              "      <td>Patient in primary analysis cohort (n=242)</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': ' ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>12</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>eligigible</td>\n",
+              "      <td>eligigible</td>\n",
+              "      <td>[{'option_code': '0', 'option_description': No...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>13</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>test_retest</td>\n",
+              "      <td>Study time point that includes test/retest DWI...</td>\n",
+              "      <td>[{'option_code': 'T0', 'option_description': N...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>14</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_pass_fail_t0</td>\n",
+              "      <td>T0 MRI DWI Quality control assessment:</td>\n",
+              "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>15</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_pass_fail_t1</td>\n",
+              "      <td>T0 MRI DWI Quality control assessment:</td>\n",
+              "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>16</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_pass_fail_t2</td>\n",
+              "      <td>T0 MRI DWI Quality control assessment:</td>\n",
+              "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>17</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_pass_fail_t3</td>\n",
+              "      <td>T0 MRI DWI Quality control assessment:</td>\n",
+              "      <td>[{'option_code': 'PASS', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>18</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_roi_confidence_t0</td>\n",
+              "      <td>T0 quality control confidence level for whole-...</td>\n",
+              "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>19</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_roi_confidence_t1</td>\n",
+              "      <td>T1 quality control confidence level for whole-...</td>\n",
+              "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>20</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_roi_confidence_t2</td>\n",
+              "      <td>T2 quality control confidence level for whole-...</td>\n",
+              "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>21</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>qc_roi_confidence_t3</td>\n",
+              "      <td>T3 quality control confidence level for whole-...</td>\n",
+              "      <td>[{'option_code': 'HIGH', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>22</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>race</td>\n",
+              "      <td>Patient race:</td>\n",
+              "      <td>[{'option_code': 'White', 'option_description'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>23</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>ltype</td>\n",
+              "      <td>Index lesion type:</td>\n",
+              "      <td>[{'option_code': 'Single mass', 'option_descri...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>24</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>hrher4g</td>\n",
+              "      <td>Hormone receptor (HR) and HER2 status:</td>\n",
+              "      <td>[{'option_code': 'HR + / HER2 +', 'option_desc...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>25</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>sbrgrade</td>\n",
+              "      <td>Tumor grade:</td>\n",
+              "      <td>[{'option_code': 'I (Low)', 'option_descriptio...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>26</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>pcr</td>\n",
+              "      <td>Pathologic complete response (pCR) at surgery</td>\n",
+              "      <td>[{'option_code': 'non-pCR', 'option_descriptio...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>27</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>dicom_patient_id</td>\n",
+              "      <td>idc_provenance_dicom_patient_id</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>28</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>i_spy_2_research_id</td>\n",
+              "      <td>Double-blinded patient ID number from I-SPY 2</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>29</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>tcia_patient_id</td>\n",
+              "      <td>Patient identification string in TCIA collecti...</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>30</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>age</td>\n",
+              "      <td>Age in years at enrollment</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>31</th>\n",
+              "      <td>acrin_6698</td>\n",
+              "      <td>acrin_6698_clinical</td>\n",
+              "      <td>mrld</td>\n",
+              "      <td>MRI measured longest diameter (cm) at baseline...</td>\n",
+              "      <td>[]</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a4ddfc88-bd0d-4633-9a1c-a17d90797845')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-a4ddfc88-bd0d-4633-9a1c-a17d90797845 button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-a4ddfc88-bd0d-4633-9a1c-a17d90797845');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "<div id=\"df-0b18f794-9ae2-443c-8af2-991aba775f79\">\n",
+              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-0b18f794-9ae2-443c-8af2-991aba775f79')\"\n",
+              "            title=\"Suggest charts\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "     width=\"24px\">\n",
+              "    <g>\n",
+              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+              "    </g>\n",
+              "</svg>\n",
+              "  </button>\n",
+              "\n",
+              "<style>\n",
+              "  .colab-df-quickchart {\n",
+              "      --bg-color: #E8F0FE;\n",
+              "      --fill-color: #1967D2;\n",
+              "      --hover-bg-color: #E2EBFA;\n",
+              "      --hover-fill-color: #174EA6;\n",
+              "      --disabled-fill-color: #AAA;\n",
+              "      --disabled-bg-color: #DDD;\n",
+              "  }\n",
+              "\n",
+              "  [theme=dark] .colab-df-quickchart {\n",
+              "      --bg-color: #3B4455;\n",
+              "      --fill-color: #D2E3FC;\n",
+              "      --hover-bg-color: #434B5C;\n",
+              "      --hover-fill-color: #FFFFFF;\n",
+              "      --disabled-bg-color: #3B4455;\n",
+              "      --disabled-fill-color: #666;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart {\n",
+              "    background-color: var(--bg-color);\n",
+              "    border: none;\n",
+              "    border-radius: 50%;\n",
+              "    cursor: pointer;\n",
+              "    display: none;\n",
+              "    fill: var(--fill-color);\n",
+              "    height: 32px;\n",
+              "    padding: 0;\n",
+              "    width: 32px;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart:hover {\n",
+              "    background-color: var(--hover-bg-color);\n",
+              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "    fill: var(--button-hover-fill-color);\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart-complete:disabled,\n",
+              "  .colab-df-quickchart-complete:disabled:hover {\n",
+              "    background-color: var(--disabled-bg-color);\n",
+              "    fill: var(--disabled-fill-color);\n",
+              "    box-shadow: none;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-spinner {\n",
+              "    border: 2px solid var(--fill-color);\n",
+              "    border-color: transparent;\n",
+              "    border-bottom-color: var(--fill-color);\n",
+              "    animation:\n",
+              "      spin 1s steps(1) infinite;\n",
+              "  }\n",
+              "\n",
+              "  @keyframes spin {\n",
+              "    0% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "      border-left-color: var(--fill-color);\n",
+              "    }\n",
+              "    20% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    30% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    40% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    60% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    80% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "    90% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "  }\n",
+              "</style>\n",
+              "\n",
+              "  <script>\n",
+              "    async function quickchart(key) {\n",
+              "      const quickchartButtonEl =\n",
+              "        document.querySelector('#' + key + ' button');\n",
+              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+              "      try {\n",
+              "        const charts = await google.colab.kernel.invokeFunction(\n",
+              "            'suggestCharts', [key], {});\n",
+              "      } catch (error) {\n",
+              "        console.error('Error during call to suggestCharts:', error);\n",
+              "      }\n",
+              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+              "    }\n",
+              "    (() => {\n",
+              "      let quickchartButtonEl =\n",
+              "        document.querySelector('#df-0b18f794-9ae2-443c-8af2-991aba775f79 button');\n",
+              "      quickchartButtonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "    })();\n",
+              "  </script>\n",
+              "</div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "summary": "{\n  \"name\": \"acrin6698_clinical_columns[[\\\"collection_id\\\", \\\"short_table_name\\\", \\\"column\\\", \\\"column_label\\\", \\\"values\\\"]]\",\n  \"rows\": 32,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"acrin_6698\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"short_table_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"acrin_6698_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 32,\n        \"samples\": [\n          \"tcia_patient_id\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 29,\n        \"samples\": [\n          \"Age in years at enrollment\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 15
+        }
       ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "variable_name": "distinct_combinations",
-       "summary": "{\n  \"name\": \"distinct_combinations\",\n  \"rows\": 385,\n  \"fields\": [\n    {\n      \"column\": \"dicom_patient_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 385,\n        \"samples\": [\n          \"ACRIN-6698-227055\",\n          \"ACRIN-6698-522140\",\n          \"ACRIN-6698-712785\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"sbrgrade\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 4,\n        \"samples\": [\n          \"NA\",\n          \"I (Low)\",\n          \"II (Intermediate)\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
-      }
-     },
-     "metadata": {},
-     "execution_count": 19
-    }
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "EE5N8y7RUw6a"
-   },
-   "source": [
-    "We can next use `dicom_patient_id` to link clinical data with the imaging studies available for the given patient (which is available in the `index` table included in `idc-index`). The query becomes a bit more complex, since we need to join data across two tables."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "source": "query = \"\"\"\nSELECT\n  ANY_VALUE(PatientID) AS PatientID,\n  STRING_AGG(DISTINCT(acrin_6698_clinical_df.sbrgrade)) as tumor_grade,\n  STRING_AGG(DISTINCT(Modality)) AS modalities,\n  COUNT(DISTINCT(SeriesInstanceUID)) AS num_series,\n  ANY_VALUE(CONCAT('https://viewer.imaging.datacommons.cancer.gov/viewer/', StudyInstanceUID)) AS viewer_url\nFROM\n  index\nJOIN\n  acrin_6698_clinical_df\nON\n  index.PatientID = acrin_6698_clinical_df.dicom_patient_id\nGROUP BY\n  StudyInstanceUID\nORDER BY\n  PatientID\n\"\"\"\n\n# sql_query() only auto-registers built-in tables (index, clinical_index, etc.).\n# User-created DataFrames must be manually registered with the underlying DuckDB\n# connection before they can be referenced in SQL queries.\n# In the future, idc-index may provide a public API to register user DataFrames.\nc._duckdb_conn.register(\"acrin_6698_clinical_df\", acrin_6698_clinical_df)\nacrin_6698_viewable = c.sql_query(query)",
-   "metadata": {
-    "id": "lITbSQ2jQ17K"
-   },
-   "execution_count": null,
-   "outputs": []
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 26,
-   "metadata": {
-    "id": "tqb7ONv5a_6_",
-    "outputId": "0505b6d3-38fd-4cb4-f1f9-03880b88c682",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 1620
-    }
-   },
-   "outputs": [
+      "source": [
+        "acrin6698_clinical_columns[[\"collection_id\", \"short_table_name\", \"column\", \"column_label\", \"values\"]]"
+      ]
+    },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "CXy4mOdj428I"
+      },
+      "source": [
+        "## From attribute names to data: accessing clinical data at patient level\n",
+        "\n",
+        "`short_table_name` gives us the table stored locally that contains the values for the column described in the `column_metadata` row. All of the clinical data tables were downloaded as part of `idc-index` installation.\n",
+        "\n",
+        "Here is how we can load the clinical data from the `acrin_6698_clinical` table into a pandas dataframe."
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "              PatientID        tumor_grade modalities  num_series  \\\n",
-       "0     ACRIN-6698-102212         III (High)     MR,SEG          19   \n",
-       "1     ACRIN-6698-102212         III (High)     SEG,MR          19   \n",
-       "2     ACRIN-6698-102212         III (High)     SEG,MR          19   \n",
-       "3     ACRIN-6698-102212         III (High)     MR,SEG          15   \n",
-       "4     ACRIN-6698-103939         III (High)     SEG,MR          18   \n",
-       "...                 ...                ...        ...         ...   \n",
-       "1118  ACRIN-6698-995480         III (High)     SEG,MR          18   \n",
-       "1119  ACRIN-6698-995480         III (High)     SEG,MR          18   \n",
-       "1120  ACRIN-6698-995480         III (High)     SEG,MR          18   \n",
-       "1121  ACRIN-6698-995480         III (High)     MR,SEG          14   \n",
-       "1122  ACRIN-6698-999368  II (Intermediate)     SEG,MR          18   \n",
-       "\n",
-       "                                             viewer_url  \n",
-       "0     https://viewer.imaging.datacommons.cancer.gov/...  \n",
-       "1     https://viewer.imaging.datacommons.cancer.gov/...  \n",
-       "2     https://viewer.imaging.datacommons.cancer.gov/...  \n",
-       "3     https://viewer.imaging.datacommons.cancer.gov/...  \n",
-       "4     https://viewer.imaging.datacommons.cancer.gov/...  \n",
-       "...                                                 ...  \n",
-       "1118  https://viewer.imaging.datacommons.cancer.gov/...  \n",
-       "1119  https://viewer.imaging.datacommons.cancer.gov/...  \n",
-       "1120  https://viewer.imaging.datacommons.cancer.gov/...  \n",
-       "1121  https://viewer.imaging.datacommons.cancer.gov/...  \n",
-       "1122  https://viewer.imaging.datacommons.cancer.gov/...  \n",
-       "\n",
-       "[1123 rows x 5 columns]"
+      "cell_type": "code",
+      "source": [
+        "acrin_6698_clinical_df = c.get_clinical_table(\"acrin_6698_clinical\")"
       ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-3f21ee63-893c-45c4-a051-cb87b998defd\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>PatientID</th>\n",
-       "      <th>tumor_grade</th>\n",
-       "      <th>modalities</th>\n",
-       "      <th>num_series</th>\n",
-       "      <th>viewer_url</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>0</th>\n",
-       "      <td>ACRIN-6698-102212</td>\n",
-       "      <td>III (High)</td>\n",
-       "      <td>MR,SEG</td>\n",
-       "      <td>19</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1</th>\n",
-       "      <td>ACRIN-6698-102212</td>\n",
-       "      <td>III (High)</td>\n",
-       "      <td>SEG,MR</td>\n",
-       "      <td>19</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2</th>\n",
-       "      <td>ACRIN-6698-102212</td>\n",
-       "      <td>III (High)</td>\n",
-       "      <td>SEG,MR</td>\n",
-       "      <td>19</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3</th>\n",
-       "      <td>ACRIN-6698-102212</td>\n",
-       "      <td>III (High)</td>\n",
-       "      <td>MR,SEG</td>\n",
-       "      <td>15</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>4</th>\n",
-       "      <td>ACRIN-6698-103939</td>\n",
-       "      <td>III (High)</td>\n",
-       "      <td>SEG,MR</td>\n",
-       "      <td>18</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>...</th>\n",
-       "      <td>...</td>\n",
-       "      <td>...</td>\n",
-       "      <td>...</td>\n",
-       "      <td>...</td>\n",
-       "      <td>...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1118</th>\n",
-       "      <td>ACRIN-6698-995480</td>\n",
-       "      <td>III (High)</td>\n",
-       "      <td>SEG,MR</td>\n",
-       "      <td>18</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1119</th>\n",
-       "      <td>ACRIN-6698-995480</td>\n",
-       "      <td>III (High)</td>\n",
-       "      <td>SEG,MR</td>\n",
-       "      <td>18</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1120</th>\n",
-       "      <td>ACRIN-6698-995480</td>\n",
-       "      <td>III (High)</td>\n",
-       "      <td>SEG,MR</td>\n",
-       "      <td>18</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1121</th>\n",
-       "      <td>ACRIN-6698-995480</td>\n",
-       "      <td>III (High)</td>\n",
-       "      <td>MR,SEG</td>\n",
-       "      <td>14</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1122</th>\n",
-       "      <td>ACRIN-6698-999368</td>\n",
-       "      <td>II (Intermediate)</td>\n",
-       "      <td>SEG,MR</td>\n",
-       "      <td>18</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "<p>1123 rows × 5 columns</p>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3f21ee63-893c-45c4-a051-cb87b998defd')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-3f21ee63-893c-45c4-a051-cb87b998defd button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-3f21ee63-893c-45c4-a051-cb87b998defd');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "<div id=\"df-3c186072-c51f-4a36-b05f-693af61a6a7b\">\n",
-       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-3c186072-c51f-4a36-b05f-693af61a6a7b')\"\n",
-       "            title=\"Suggest charts\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "     width=\"24px\">\n",
-       "    <g>\n",
-       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
-       "    </g>\n",
-       "</svg>\n",
-       "  </button>\n",
-       "\n",
-       "<style>\n",
-       "  .colab-df-quickchart {\n",
-       "      --bg-color: #E8F0FE;\n",
-       "      --fill-color: #1967D2;\n",
-       "      --hover-bg-color: #E2EBFA;\n",
-       "      --hover-fill-color: #174EA6;\n",
-       "      --disabled-fill-color: #AAA;\n",
-       "      --disabled-bg-color: #DDD;\n",
-       "  }\n",
-       "\n",
-       "  [theme=dark] .colab-df-quickchart {\n",
-       "      --bg-color: #3B4455;\n",
-       "      --fill-color: #D2E3FC;\n",
-       "      --hover-bg-color: #434B5C;\n",
-       "      --hover-fill-color: #FFFFFF;\n",
-       "      --disabled-bg-color: #3B4455;\n",
-       "      --disabled-fill-color: #666;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart {\n",
-       "    background-color: var(--bg-color);\n",
-       "    border: none;\n",
-       "    border-radius: 50%;\n",
-       "    cursor: pointer;\n",
-       "    display: none;\n",
-       "    fill: var(--fill-color);\n",
-       "    height: 32px;\n",
-       "    padding: 0;\n",
-       "    width: 32px;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart:hover {\n",
-       "    background-color: var(--hover-bg-color);\n",
-       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "    fill: var(--button-hover-fill-color);\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart-complete:disabled,\n",
-       "  .colab-df-quickchart-complete:disabled:hover {\n",
-       "    background-color: var(--disabled-bg-color);\n",
-       "    fill: var(--disabled-fill-color);\n",
-       "    box-shadow: none;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-spinner {\n",
-       "    border: 2px solid var(--fill-color);\n",
-       "    border-color: transparent;\n",
-       "    border-bottom-color: var(--fill-color);\n",
-       "    animation:\n",
-       "      spin 1s steps(1) infinite;\n",
-       "  }\n",
-       "\n",
-       "  @keyframes spin {\n",
-       "    0% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "      border-left-color: var(--fill-color);\n",
-       "    }\n",
-       "    20% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    30% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    40% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    60% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    80% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "    90% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "  }\n",
-       "</style>\n",
-       "\n",
-       "  <script>\n",
-       "    async function quickchart(key) {\n",
-       "      const quickchartButtonEl =\n",
-       "        document.querySelector('#' + key + ' button');\n",
-       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
-       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
-       "      try {\n",
-       "        const charts = await google.colab.kernel.invokeFunction(\n",
-       "            'suggestCharts', [key], {});\n",
-       "      } catch (error) {\n",
-       "        console.error('Error during call to suggestCharts:', error);\n",
-       "      }\n",
-       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
-       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
-       "    }\n",
-       "    (() => {\n",
-       "      let quickchartButtonEl =\n",
-       "        document.querySelector('#df-3c186072-c51f-4a36-b05f-693af61a6a7b button');\n",
-       "      quickchartButtonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "    })();\n",
-       "  </script>\n",
-       "</div>\n",
-       "\n",
-       "  <div id=\"id_7853b0f0-1a49-4a3c-9305-bd445b45cc8a\">\n",
-       "    <style>\n",
-       "      .colab-df-generate {\n",
-       "        background-color: #E8F0FE;\n",
-       "        border: none;\n",
-       "        border-radius: 50%;\n",
-       "        cursor: pointer;\n",
-       "        display: none;\n",
-       "        fill: #1967D2;\n",
-       "        height: 32px;\n",
-       "        padding: 0 0 0 0;\n",
-       "        width: 32px;\n",
-       "      }\n",
-       "\n",
-       "      .colab-df-generate:hover {\n",
-       "        background-color: #E2EBFA;\n",
-       "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "        fill: #174EA6;\n",
-       "      }\n",
-       "\n",
-       "      [theme=dark] .colab-df-generate {\n",
-       "        background-color: #3B4455;\n",
-       "        fill: #D2E3FC;\n",
-       "      }\n",
-       "\n",
-       "      [theme=dark] .colab-df-generate:hover {\n",
-       "        background-color: #434B5C;\n",
-       "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "        fill: #FFFFFF;\n",
-       "      }\n",
-       "    </style>\n",
-       "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('acrin_6698_viewable')\"\n",
-       "            title=\"Generate code using this dataframe.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "       width=\"24px\">\n",
-       "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "    <script>\n",
-       "      (() => {\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#id_7853b0f0-1a49-4a3c-9305-bd445b45cc8a button.colab-df-generate');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      buttonEl.onclick = () => {\n",
-       "        google.colab.notebook.generateWithVariable('acrin_6698_viewable');\n",
-       "      }\n",
-       "      })();\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
+      "metadata": {
+        "id": "USHHDVjLDRN-"
+      },
+      "execution_count": 18,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "Given the information available in the per-collection clinical data, we can proceed with selecting a subset of patients that meet the criteria of your interest. As an example, the following query will select all of the distinct combination of patient ID and tumor grade, as defined by the `sbrgrade` column contents.\n"
       ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "variable_name": "acrin_6698_viewable",
-       "summary": "{\n  \"name\": \"acrin_6698_viewable\",\n  \"rows\": 1123,\n  \"fields\": [\n    {\n      \"column\": \"PatientID\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 385,\n        \"samples\": [\n          \"ACRIN-6698-721099\",\n          \"ACRIN-6698-687913\",\n          \"ACRIN-6698-945948\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"tumor_grade\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 4,\n        \"samples\": [\n          \"NA\",\n          \"I (Low)\",\n          \"III (High)\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"modalities\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"MR,SEG\",\n          \"SEG,MR\",\n          \"MR\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"num_series\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3,\n        \"min\": 2,\n        \"max\": 32,\n        \"num_unique_values\": 28,\n        \"samples\": [\n          2,\n          27,\n          14\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"viewer_url\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1123,\n        \"samples\": [\n          \"https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.7695.4164.102691710349159895369529927942\",\n          \"https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.7695.4164.314494137701007936430926991782\",\n          \"https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.7695.4164.179120149527078018237837173947\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+      "metadata": {
+        "id": "KQvhHGuTP3wK"
       }
-     },
-     "metadata": {},
-     "execution_count": 26
     },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/data_table.py:200: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  return self._dataframe._repr_html_()  # pylint: disable=protected-access\n"
-     ]
-    }
-   ],
-   "source": [
-    "acrin_6698_viewable"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "-38R6Z_3veHG"
-   },
-   "source": [
-    "## Discovery mode\n",
-    "\n",
-    "Sometime you may want to find whether specific clinical attribute is available for the imaging data you can find in IDC.\n",
-    "\n",
-    "We can start by looking at the distinct values of `column_label` (which in the general case will be either more descriptive, or identical to `column`).\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 29,
-   "metadata": {
-    "id": "tt3htrbxr6KC",
-    "outputId": "bbceac91-6c1d-4479-e7f7-c9ed930e9a5e",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 479
-    }
-   },
-   "outputs": [
+      "cell_type": "code",
+      "source": [
+        "# prompt: select distinct combinations of values in dicom_patient_id and sbrgrade columns from the acrin_6698_clinical_df pandas dataframe\n",
+        "acrin_6698_tumors = acrin_6698_clinical_df[['dicom_patient_id', 'sbrgrade']].drop_duplicates()\n",
+        "acrin_6698_tumors"
+      ],
+      "metadata": {
+        "id": "CKYHeu1pQK8V",
+        "outputId": "4ceb6c77-d9fe-4a3a-8647-146fdd21cd1d",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 479
+        }
+      },
+      "execution_count": 19,
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "      dicom_patient_id           sbrgrade\n",
+              "0    ACRIN-6698-453236  II (Intermediate)\n",
+              "1    ACRIN-6698-229047                 NA\n",
+              "2    ACRIN-6698-384705  II (Intermediate)\n",
+              "3    ACRIN-6698-415631         III (High)\n",
+              "4    ACRIN-6698-793283                 NA\n",
+              "..                 ...                ...\n",
+              "380  ACRIN-6698-765671  II (Intermediate)\n",
+              "381  ACRIN-6698-962153         III (High)\n",
+              "382  ACRIN-6698-711476  II (Intermediate)\n",
+              "383  ACRIN-6698-361701  II (Intermediate)\n",
+              "384  ACRIN-6698-241998            I (Low)\n",
+              "\n",
+              "[385 rows x 2 columns]"
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-c8d4f416-aa84-4c49-a260-f295bebf58a2\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>dicom_patient_id</th>\n",
+              "      <th>sbrgrade</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>0</th>\n",
+              "      <td>ACRIN-6698-453236</td>\n",
+              "      <td>II (Intermediate)</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1</th>\n",
+              "      <td>ACRIN-6698-229047</td>\n",
+              "      <td>NA</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>2</th>\n",
+              "      <td>ACRIN-6698-384705</td>\n",
+              "      <td>II (Intermediate)</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3</th>\n",
+              "      <td>ACRIN-6698-415631</td>\n",
+              "      <td>III (High)</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>4</th>\n",
+              "      <td>ACRIN-6698-793283</td>\n",
+              "      <td>NA</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>...</th>\n",
+              "      <td>...</td>\n",
+              "      <td>...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>380</th>\n",
+              "      <td>ACRIN-6698-765671</td>\n",
+              "      <td>II (Intermediate)</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>381</th>\n",
+              "      <td>ACRIN-6698-962153</td>\n",
+              "      <td>III (High)</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>382</th>\n",
+              "      <td>ACRIN-6698-711476</td>\n",
+              "      <td>II (Intermediate)</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>383</th>\n",
+              "      <td>ACRIN-6698-361701</td>\n",
+              "      <td>II (Intermediate)</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>384</th>\n",
+              "      <td>ACRIN-6698-241998</td>\n",
+              "      <td>I (Low)</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "<p>385 rows × 2 columns</p>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c8d4f416-aa84-4c49-a260-f295bebf58a2')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-c8d4f416-aa84-4c49-a260-f295bebf58a2 button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-c8d4f416-aa84-4c49-a260-f295bebf58a2');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "<div id=\"df-7e3c6a5b-97f9-4779-83ab-cc4658119bb7\">\n",
+              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-7e3c6a5b-97f9-4779-83ab-cc4658119bb7')\"\n",
+              "            title=\"Suggest charts\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "     width=\"24px\">\n",
+              "    <g>\n",
+              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+              "    </g>\n",
+              "</svg>\n",
+              "  </button>\n",
+              "\n",
+              "<style>\n",
+              "  .colab-df-quickchart {\n",
+              "      --bg-color: #E8F0FE;\n",
+              "      --fill-color: #1967D2;\n",
+              "      --hover-bg-color: #E2EBFA;\n",
+              "      --hover-fill-color: #174EA6;\n",
+              "      --disabled-fill-color: #AAA;\n",
+              "      --disabled-bg-color: #DDD;\n",
+              "  }\n",
+              "\n",
+              "  [theme=dark] .colab-df-quickchart {\n",
+              "      --bg-color: #3B4455;\n",
+              "      --fill-color: #D2E3FC;\n",
+              "      --hover-bg-color: #434B5C;\n",
+              "      --hover-fill-color: #FFFFFF;\n",
+              "      --disabled-bg-color: #3B4455;\n",
+              "      --disabled-fill-color: #666;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart {\n",
+              "    background-color: var(--bg-color);\n",
+              "    border: none;\n",
+              "    border-radius: 50%;\n",
+              "    cursor: pointer;\n",
+              "    display: none;\n",
+              "    fill: var(--fill-color);\n",
+              "    height: 32px;\n",
+              "    padding: 0;\n",
+              "    width: 32px;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart:hover {\n",
+              "    background-color: var(--hover-bg-color);\n",
+              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "    fill: var(--button-hover-fill-color);\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart-complete:disabled,\n",
+              "  .colab-df-quickchart-complete:disabled:hover {\n",
+              "    background-color: var(--disabled-bg-color);\n",
+              "    fill: var(--disabled-fill-color);\n",
+              "    box-shadow: none;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-spinner {\n",
+              "    border: 2px solid var(--fill-color);\n",
+              "    border-color: transparent;\n",
+              "    border-bottom-color: var(--fill-color);\n",
+              "    animation:\n",
+              "      spin 1s steps(1) infinite;\n",
+              "  }\n",
+              "\n",
+              "  @keyframes spin {\n",
+              "    0% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "      border-left-color: var(--fill-color);\n",
+              "    }\n",
+              "    20% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    30% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    40% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    60% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    80% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "    90% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "  }\n",
+              "</style>\n",
+              "\n",
+              "  <script>\n",
+              "    async function quickchart(key) {\n",
+              "      const quickchartButtonEl =\n",
+              "        document.querySelector('#' + key + ' button');\n",
+              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+              "      try {\n",
+              "        const charts = await google.colab.kernel.invokeFunction(\n",
+              "            'suggestCharts', [key], {});\n",
+              "      } catch (error) {\n",
+              "        console.error('Error during call to suggestCharts:', error);\n",
+              "      }\n",
+              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+              "    }\n",
+              "    (() => {\n",
+              "      let quickchartButtonEl =\n",
+              "        document.querySelector('#df-7e3c6a5b-97f9-4779-83ab-cc4658119bb7 button');\n",
+              "      quickchartButtonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "    })();\n",
+              "  </script>\n",
+              "</div>\n",
+              "\n",
+              "  <div id=\"id_1cb558c1-92b9-4f6e-908f-b9e6b7cfa343\">\n",
+              "    <style>\n",
+              "      .colab-df-generate {\n",
+              "        background-color: #E8F0FE;\n",
+              "        border: none;\n",
+              "        border-radius: 50%;\n",
+              "        cursor: pointer;\n",
+              "        display: none;\n",
+              "        fill: #1967D2;\n",
+              "        height: 32px;\n",
+              "        padding: 0 0 0 0;\n",
+              "        width: 32px;\n",
+              "      }\n",
+              "\n",
+              "      .colab-df-generate:hover {\n",
+              "        background-color: #E2EBFA;\n",
+              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "        fill: #174EA6;\n",
+              "      }\n",
+              "\n",
+              "      [theme=dark] .colab-df-generate {\n",
+              "        background-color: #3B4455;\n",
+              "        fill: #D2E3FC;\n",
+              "      }\n",
+              "\n",
+              "      [theme=dark] .colab-df-generate:hover {\n",
+              "        background-color: #434B5C;\n",
+              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "        fill: #FFFFFF;\n",
+              "      }\n",
+              "    </style>\n",
+              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('distinct_combinations')\"\n",
+              "            title=\"Generate code using this dataframe.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "       width=\"24px\">\n",
+              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "    <script>\n",
+              "      (() => {\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#id_1cb558c1-92b9-4f6e-908f-b9e6b7cfa343 button.colab-df-generate');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      buttonEl.onclick = () => {\n",
+              "        google.colab.notebook.generateWithVariable('distinct_combinations');\n",
+              "      }\n",
+              "      })();\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "variable_name": "distinct_combinations",
+              "summary": "{\n  \"name\": \"distinct_combinations\",\n  \"rows\": 385,\n  \"fields\": [\n    {\n      \"column\": \"dicom_patient_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 385,\n        \"samples\": [\n          \"ACRIN-6698-227055\",\n          \"ACRIN-6698-522140\",\n          \"ACRIN-6698-712785\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"sbrgrade\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 4,\n        \"samples\": [\n          \"NA\",\n          \"I (Low)\",\n          \"II (Intermediate)\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 19
+        }
+      ]
+    },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "EE5N8y7RUw6a"
+      },
+      "source": [
+        "We can next use `dicom_patient_id` to link clinical data with the imaging studies available for the given patient (which is available in the `index` table included in `idc-index`). The query becomes a bit more complex, since we need to join data across two tables."
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "                                           column_label\n",
-       "0                           idc_provenance_source_batch\n",
-       "1        T0 (baseline) MRI study included in collection\n",
-       "2        T1 (early-Tx) MRI study included in collection\n",
-       "3     T2 (inter-regimen) MRI study included in colle...\n",
-       "4     T3 (pre-surgery) MRI study included in collection\n",
-       "...                                                 ...\n",
-       "3707                                                 ID\n",
-       "3708                 Survival_from_surgery_days_UPDATED\n",
-       "3709                                    Survival_Censor\n",
-       "3710                          Time_since_baseline_preop\n",
-       "3711                                  Age_at_scan_years\n",
-       "\n",
-       "[3712 rows x 1 columns]"
+      "cell_type": "code",
+      "source": [
+        "query = \"\"\"\n",
+        "SELECT\n",
+        "  ANY_VALUE(PatientID) AS PatientID,\n",
+        "  STRING_AGG(DISTINCT(acrin_6698_clinical_df.sbrgrade)) as tumor_grade,\n",
+        "  STRING_AGG(DISTINCT(Modality)) AS modalities,\n",
+        "  COUNT(DISTINCT(SeriesInstanceUID)) AS num_series,\n",
+        "  ANY_VALUE(CONCAT('https://viewer.imaging.datacommons.cancer.gov/viewer/', StudyInstanceUID)) AS viewer_url\n",
+        "FROM\n",
+        "  index\n",
+        "JOIN\n",
+        "  acrin_6698_clinical_df\n",
+        "ON\n",
+        "  index.PatientID = acrin_6698_clinical_df.dicom_patient_id\n",
+        "GROUP BY\n",
+        "  StudyInstanceUID\n",
+        "ORDER BY\n",
+        "  PatientID\n",
+        "\"\"\"\n",
+        "\n",
+        "acrin_6698_viewable = c.sql_query(query)"
       ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-2020b699-f599-4c35-956e-bf344abc3aa3\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>column_label</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>0</th>\n",
-       "      <td>idc_provenance_source_batch</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1</th>\n",
-       "      <td>T0 (baseline) MRI study included in collection</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2</th>\n",
-       "      <td>T1 (early-Tx) MRI study included in collection</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3</th>\n",
-       "      <td>T2 (inter-regimen) MRI study included in colle...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>4</th>\n",
-       "      <td>T3 (pre-surgery) MRI study included in collection</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>...</th>\n",
-       "      <td>...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3707</th>\n",
-       "      <td>ID</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3708</th>\n",
-       "      <td>Survival_from_surgery_days_UPDATED</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3709</th>\n",
-       "      <td>Survival_Censor</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3710</th>\n",
-       "      <td>Time_since_baseline_preop</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3711</th>\n",
-       "      <td>Age_at_scan_years</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "<p>3712 rows × 1 columns</p>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2020b699-f599-4c35-956e-bf344abc3aa3')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-2020b699-f599-4c35-956e-bf344abc3aa3 button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-2020b699-f599-4c35-956e-bf344abc3aa3');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "<div id=\"df-8ff08438-cc9c-4303-a321-1f2a80b9269c\">\n",
-       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-8ff08438-cc9c-4303-a321-1f2a80b9269c')\"\n",
-       "            title=\"Suggest charts\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "     width=\"24px\">\n",
-       "    <g>\n",
-       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
-       "    </g>\n",
-       "</svg>\n",
-       "  </button>\n",
-       "\n",
-       "<style>\n",
-       "  .colab-df-quickchart {\n",
-       "      --bg-color: #E8F0FE;\n",
-       "      --fill-color: #1967D2;\n",
-       "      --hover-bg-color: #E2EBFA;\n",
-       "      --hover-fill-color: #174EA6;\n",
-       "      --disabled-fill-color: #AAA;\n",
-       "      --disabled-bg-color: #DDD;\n",
-       "  }\n",
-       "\n",
-       "  [theme=dark] .colab-df-quickchart {\n",
-       "      --bg-color: #3B4455;\n",
-       "      --fill-color: #D2E3FC;\n",
-       "      --hover-bg-color: #434B5C;\n",
-       "      --hover-fill-color: #FFFFFF;\n",
-       "      --disabled-bg-color: #3B4455;\n",
-       "      --disabled-fill-color: #666;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart {\n",
-       "    background-color: var(--bg-color);\n",
-       "    border: none;\n",
-       "    border-radius: 50%;\n",
-       "    cursor: pointer;\n",
-       "    display: none;\n",
-       "    fill: var(--fill-color);\n",
-       "    height: 32px;\n",
-       "    padding: 0;\n",
-       "    width: 32px;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart:hover {\n",
-       "    background-color: var(--hover-bg-color);\n",
-       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "    fill: var(--button-hover-fill-color);\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart-complete:disabled,\n",
-       "  .colab-df-quickchart-complete:disabled:hover {\n",
-       "    background-color: var(--disabled-bg-color);\n",
-       "    fill: var(--disabled-fill-color);\n",
-       "    box-shadow: none;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-spinner {\n",
-       "    border: 2px solid var(--fill-color);\n",
-       "    border-color: transparent;\n",
-       "    border-bottom-color: var(--fill-color);\n",
-       "    animation:\n",
-       "      spin 1s steps(1) infinite;\n",
-       "  }\n",
-       "\n",
-       "  @keyframes spin {\n",
-       "    0% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "      border-left-color: var(--fill-color);\n",
-       "    }\n",
-       "    20% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    30% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    40% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    60% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    80% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "    90% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "  }\n",
-       "</style>\n",
-       "\n",
-       "  <script>\n",
-       "    async function quickchart(key) {\n",
-       "      const quickchartButtonEl =\n",
-       "        document.querySelector('#' + key + ' button');\n",
-       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
-       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
-       "      try {\n",
-       "        const charts = await google.colab.kernel.invokeFunction(\n",
-       "            'suggestCharts', [key], {});\n",
-       "      } catch (error) {\n",
-       "        console.error('Error during call to suggestCharts:', error);\n",
-       "      }\n",
-       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
-       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
-       "    }\n",
-       "    (() => {\n",
-       "      let quickchartButtonEl =\n",
-       "        document.querySelector('#df-8ff08438-cc9c-4303-a321-1f2a80b9269c button');\n",
-       "      quickchartButtonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "    })();\n",
-       "  </script>\n",
-       "</div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
+      "metadata": {
+        "id": "lITbSQ2jQ17K"
+      },
+      "execution_count": 25,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 26,
+      "metadata": {
+        "id": "tqb7ONv5a_6_",
+        "outputId": "0505b6d3-38fd-4cb4-f1f9-03880b88c682",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 1620
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "              PatientID        tumor_grade modalities  num_series  \\\n",
+              "0     ACRIN-6698-102212         III (High)     MR,SEG          19   \n",
+              "1     ACRIN-6698-102212         III (High)     SEG,MR          19   \n",
+              "2     ACRIN-6698-102212         III (High)     SEG,MR          19   \n",
+              "3     ACRIN-6698-102212         III (High)     MR,SEG          15   \n",
+              "4     ACRIN-6698-103939         III (High)     SEG,MR          18   \n",
+              "...                 ...                ...        ...         ...   \n",
+              "1118  ACRIN-6698-995480         III (High)     SEG,MR          18   \n",
+              "1119  ACRIN-6698-995480         III (High)     SEG,MR          18   \n",
+              "1120  ACRIN-6698-995480         III (High)     SEG,MR          18   \n",
+              "1121  ACRIN-6698-995480         III (High)     MR,SEG          14   \n",
+              "1122  ACRIN-6698-999368  II (Intermediate)     SEG,MR          18   \n",
+              "\n",
+              "                                             viewer_url  \n",
+              "0     https://viewer.imaging.datacommons.cancer.gov/...  \n",
+              "1     https://viewer.imaging.datacommons.cancer.gov/...  \n",
+              "2     https://viewer.imaging.datacommons.cancer.gov/...  \n",
+              "3     https://viewer.imaging.datacommons.cancer.gov/...  \n",
+              "4     https://viewer.imaging.datacommons.cancer.gov/...  \n",
+              "...                                                 ...  \n",
+              "1118  https://viewer.imaging.datacommons.cancer.gov/...  \n",
+              "1119  https://viewer.imaging.datacommons.cancer.gov/...  \n",
+              "1120  https://viewer.imaging.datacommons.cancer.gov/...  \n",
+              "1121  https://viewer.imaging.datacommons.cancer.gov/...  \n",
+              "1122  https://viewer.imaging.datacommons.cancer.gov/...  \n",
+              "\n",
+              "[1123 rows x 5 columns]"
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-3f21ee63-893c-45c4-a051-cb87b998defd\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>PatientID</th>\n",
+              "      <th>tumor_grade</th>\n",
+              "      <th>modalities</th>\n",
+              "      <th>num_series</th>\n",
+              "      <th>viewer_url</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>0</th>\n",
+              "      <td>ACRIN-6698-102212</td>\n",
+              "      <td>III (High)</td>\n",
+              "      <td>MR,SEG</td>\n",
+              "      <td>19</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1</th>\n",
+              "      <td>ACRIN-6698-102212</td>\n",
+              "      <td>III (High)</td>\n",
+              "      <td>SEG,MR</td>\n",
+              "      <td>19</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>2</th>\n",
+              "      <td>ACRIN-6698-102212</td>\n",
+              "      <td>III (High)</td>\n",
+              "      <td>SEG,MR</td>\n",
+              "      <td>19</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3</th>\n",
+              "      <td>ACRIN-6698-102212</td>\n",
+              "      <td>III (High)</td>\n",
+              "      <td>MR,SEG</td>\n",
+              "      <td>15</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>4</th>\n",
+              "      <td>ACRIN-6698-103939</td>\n",
+              "      <td>III (High)</td>\n",
+              "      <td>SEG,MR</td>\n",
+              "      <td>18</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>...</th>\n",
+              "      <td>...</td>\n",
+              "      <td>...</td>\n",
+              "      <td>...</td>\n",
+              "      <td>...</td>\n",
+              "      <td>...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1118</th>\n",
+              "      <td>ACRIN-6698-995480</td>\n",
+              "      <td>III (High)</td>\n",
+              "      <td>SEG,MR</td>\n",
+              "      <td>18</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1119</th>\n",
+              "      <td>ACRIN-6698-995480</td>\n",
+              "      <td>III (High)</td>\n",
+              "      <td>SEG,MR</td>\n",
+              "      <td>18</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1120</th>\n",
+              "      <td>ACRIN-6698-995480</td>\n",
+              "      <td>III (High)</td>\n",
+              "      <td>SEG,MR</td>\n",
+              "      <td>18</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1121</th>\n",
+              "      <td>ACRIN-6698-995480</td>\n",
+              "      <td>III (High)</td>\n",
+              "      <td>MR,SEG</td>\n",
+              "      <td>14</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1122</th>\n",
+              "      <td>ACRIN-6698-999368</td>\n",
+              "      <td>II (Intermediate)</td>\n",
+              "      <td>SEG,MR</td>\n",
+              "      <td>18</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "<p>1123 rows × 5 columns</p>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3f21ee63-893c-45c4-a051-cb87b998defd')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-3f21ee63-893c-45c4-a051-cb87b998defd button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-3f21ee63-893c-45c4-a051-cb87b998defd');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "<div id=\"df-3c186072-c51f-4a36-b05f-693af61a6a7b\">\n",
+              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-3c186072-c51f-4a36-b05f-693af61a6a7b')\"\n",
+              "            title=\"Suggest charts\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "     width=\"24px\">\n",
+              "    <g>\n",
+              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+              "    </g>\n",
+              "</svg>\n",
+              "  </button>\n",
+              "\n",
+              "<style>\n",
+              "  .colab-df-quickchart {\n",
+              "      --bg-color: #E8F0FE;\n",
+              "      --fill-color: #1967D2;\n",
+              "      --hover-bg-color: #E2EBFA;\n",
+              "      --hover-fill-color: #174EA6;\n",
+              "      --disabled-fill-color: #AAA;\n",
+              "      --disabled-bg-color: #DDD;\n",
+              "  }\n",
+              "\n",
+              "  [theme=dark] .colab-df-quickchart {\n",
+              "      --bg-color: #3B4455;\n",
+              "      --fill-color: #D2E3FC;\n",
+              "      --hover-bg-color: #434B5C;\n",
+              "      --hover-fill-color: #FFFFFF;\n",
+              "      --disabled-bg-color: #3B4455;\n",
+              "      --disabled-fill-color: #666;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart {\n",
+              "    background-color: var(--bg-color);\n",
+              "    border: none;\n",
+              "    border-radius: 50%;\n",
+              "    cursor: pointer;\n",
+              "    display: none;\n",
+              "    fill: var(--fill-color);\n",
+              "    height: 32px;\n",
+              "    padding: 0;\n",
+              "    width: 32px;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart:hover {\n",
+              "    background-color: var(--hover-bg-color);\n",
+              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "    fill: var(--button-hover-fill-color);\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart-complete:disabled,\n",
+              "  .colab-df-quickchart-complete:disabled:hover {\n",
+              "    background-color: var(--disabled-bg-color);\n",
+              "    fill: var(--disabled-fill-color);\n",
+              "    box-shadow: none;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-spinner {\n",
+              "    border: 2px solid var(--fill-color);\n",
+              "    border-color: transparent;\n",
+              "    border-bottom-color: var(--fill-color);\n",
+              "    animation:\n",
+              "      spin 1s steps(1) infinite;\n",
+              "  }\n",
+              "\n",
+              "  @keyframes spin {\n",
+              "    0% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "      border-left-color: var(--fill-color);\n",
+              "    }\n",
+              "    20% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    30% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    40% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    60% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    80% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "    90% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "  }\n",
+              "</style>\n",
+              "\n",
+              "  <script>\n",
+              "    async function quickchart(key) {\n",
+              "      const quickchartButtonEl =\n",
+              "        document.querySelector('#' + key + ' button');\n",
+              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+              "      try {\n",
+              "        const charts = await google.colab.kernel.invokeFunction(\n",
+              "            'suggestCharts', [key], {});\n",
+              "      } catch (error) {\n",
+              "        console.error('Error during call to suggestCharts:', error);\n",
+              "      }\n",
+              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+              "    }\n",
+              "    (() => {\n",
+              "      let quickchartButtonEl =\n",
+              "        document.querySelector('#df-3c186072-c51f-4a36-b05f-693af61a6a7b button');\n",
+              "      quickchartButtonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "    })();\n",
+              "  </script>\n",
+              "</div>\n",
+              "\n",
+              "  <div id=\"id_7853b0f0-1a49-4a3c-9305-bd445b45cc8a\">\n",
+              "    <style>\n",
+              "      .colab-df-generate {\n",
+              "        background-color: #E8F0FE;\n",
+              "        border: none;\n",
+              "        border-radius: 50%;\n",
+              "        cursor: pointer;\n",
+              "        display: none;\n",
+              "        fill: #1967D2;\n",
+              "        height: 32px;\n",
+              "        padding: 0 0 0 0;\n",
+              "        width: 32px;\n",
+              "      }\n",
+              "\n",
+              "      .colab-df-generate:hover {\n",
+              "        background-color: #E2EBFA;\n",
+              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "        fill: #174EA6;\n",
+              "      }\n",
+              "\n",
+              "      [theme=dark] .colab-df-generate {\n",
+              "        background-color: #3B4455;\n",
+              "        fill: #D2E3FC;\n",
+              "      }\n",
+              "\n",
+              "      [theme=dark] .colab-df-generate:hover {\n",
+              "        background-color: #434B5C;\n",
+              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "        fill: #FFFFFF;\n",
+              "      }\n",
+              "    </style>\n",
+              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('acrin_6698_viewable')\"\n",
+              "            title=\"Generate code using this dataframe.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "       width=\"24px\">\n",
+              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "    <script>\n",
+              "      (() => {\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#id_7853b0f0-1a49-4a3c-9305-bd445b45cc8a button.colab-df-generate');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      buttonEl.onclick = () => {\n",
+              "        google.colab.notebook.generateWithVariable('acrin_6698_viewable');\n",
+              "      }\n",
+              "      })();\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "variable_name": "acrin_6698_viewable",
+              "summary": "{\n  \"name\": \"acrin_6698_viewable\",\n  \"rows\": 1123,\n  \"fields\": [\n    {\n      \"column\": \"PatientID\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 385,\n        \"samples\": [\n          \"ACRIN-6698-721099\",\n          \"ACRIN-6698-687913\",\n          \"ACRIN-6698-945948\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"tumor_grade\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 4,\n        \"samples\": [\n          \"NA\",\n          \"I (Low)\",\n          \"III (High)\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"modalities\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"MR,SEG\",\n          \"SEG,MR\",\n          \"MR\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"num_series\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3,\n        \"min\": 2,\n        \"max\": 32,\n        \"num_unique_values\": 28,\n        \"samples\": [\n          2,\n          27,\n          14\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"viewer_url\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1123,\n        \"samples\": [\n          \"https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.7695.4164.102691710349159895369529927942\",\n          \"https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.7695.4164.314494137701007936430926991782\",\n          \"https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.7695.4164.179120149527078018237837173947\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 26
+        },
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/data_table.py:200: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  return self._dataframe._repr_html_()  # pylint: disable=protected-access\n"
+          ]
+        }
       ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "summary": "{\n  \"name\": \"pd\",\n  \"rows\": 3712,\n  \"fields\": [\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 3712,\n        \"samples\": [\n          \"TNM staging \",\n          \"BASELINE SYMPTOMS - SYMPTOM GRADE 19\",\n          \"DATE FORM COMPLETED (MM/DD/YYYY)\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
-      }
-     },
-     "metadata": {},
-     "execution_count": 29
-    }
-   ],
-   "source": [
-    "import pandas as pd\n",
-    "\n",
-    "pd.DataFrame({\"column_label\":c.clinical_index[\"column_label\"].unique()})"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "7iZpB1Foi1hd"
-   },
-   "source": [
-    "Let's say we want to know which of the cases have information related to therapy. We can search column metadata for the presence of word \"therapy\" (since the terms in clinical data are not harmonized, we need to account for the variability in capitalization)."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 31,
-   "metadata": {
-    "id": "qCTFR-QpjJSw",
-    "outputId": "9a3e6dfb-497c-4e4d-d12e-f0dbe6490208",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 1000
-    }
-   },
-   "outputs": [
+      "source": [
+        "acrin_6698_viewable"
+      ]
+    },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "-38R6Z_3veHG"
+      },
+      "source": [
+        "## Discovery mode\n",
+        "\n",
+        "Sometime you may want to find whether specific clinical attribute is available for the imaging data you can find in IDC.\n",
+        "\n",
+        "We can start by looking at the distinct values of `column_label` (which in the general case will be either more descriptive, or identical to `column`).\n"
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "                                           column_label  \\\n",
-       "59    current or recent hx (6 months prior to MRI) c...   \n",
-       "201         current use of estrogen replacement therapy   \n",
-       "202               current use of tamoxifen/serm therapy   \n",
-       "203             current use aromatase inhibitor therapy   \n",
-       "205            past use of estrogen replacement therapy   \n",
-       "206                  past use of tamoxifen/serm therapy   \n",
-       "207               prior use aromatase inhibitor therapy   \n",
-       "3891           Early response (1-2 months post-therapy)   \n",
-       "4000  Did the patient have documented renal replacem...   \n",
-       "4004  Patient had other anticoagulation therapy as l...   \n",
-       "5570              chemotherapy used for TACE procedure    \n",
-       "5888                         Regimen_or_Line_of_Therapy   \n",
-       "5892                               Treatment_or_Therapy   \n",
-       "6194                         Regimen_or_Line_of_Therapy   \n",
-       "6209                               Treatment_or_Therapy   \n",
-       "6515                         Regimen_or_Line_of_Therapy   \n",
-       "6522                               Treatment_or_Therapy   \n",
-       "6766                               Treatment_or_Therapy   \n",
-       "6768                         Regimen_or_Line_of_Therapy   \n",
-       "7141                                       Chemotherapy   \n",
-       "7457                    primary_therapy_outcome_success   \n",
-       "7534                    primary_therapy_outcome_success   \n",
-       "7635                    primary_therapy_outcome_success   \n",
-       "7684                    primary_therapy_outcome_success   \n",
-       "7784                    primary_therapy_outcome_success   \n",
-       "7830                    primary_therapy_outcome_success   \n",
-       "7894                    primary_therapy_outcome_success   \n",
-       "7984                    primary_therapy_outcome_success   \n",
-       "8045                    primary_therapy_outcome_success   \n",
-       "8129                    primary_therapy_outcome_success   \n",
-       "8198                    primary_therapy_outcome_success   \n",
-       "8290                    primary_therapy_outcome_success   \n",
-       "8367                    primary_therapy_outcome_success   \n",
-       "8435                    primary_therapy_outcome_success   \n",
-       "8573                    primary_therapy_outcome_success   \n",
-       "8647                    primary_therapy_outcome_success   \n",
-       "8721                    primary_therapy_outcome_success   \n",
-       "8857                    primary_therapy_outcome_success   \n",
-       "8900                    primary_therapy_outcome_success   \n",
-       "8978                    primary_therapy_outcome_success   \n",
-       "9043                    primary_therapy_outcome_success   \n",
-       "9125                    primary_therapy_outcome_success   \n",
-       "9201                    primary_therapy_outcome_success   \n",
-       "9301                    primary_therapy_outcome_success   \n",
-       "9374                    primary_therapy_outcome_success   \n",
-       "9423                    primary_therapy_outcome_success   \n",
-       "9497                    primary_therapy_outcome_success   \n",
-       "9597                    primary_therapy_outcome_success   \n",
-       "9670                    primary_therapy_outcome_success   \n",
-       "9715                    primary_therapy_outcome_success   \n",
-       "9790                    primary_therapy_outcome_success   \n",
-       "9892                    primary_therapy_outcome_success   \n",
-       "\n",
-       "                      collection_id  \\\n",
-       "59    acrin_contralateral_breast_mr   \n",
-       "201   acrin_contralateral_breast_mr   \n",
-       "202   acrin_contralateral_breast_mr   \n",
-       "203   acrin_contralateral_breast_mr   \n",
-       "205   acrin_contralateral_breast_mr   \n",
-       "206   acrin_contralateral_breast_mr   \n",
-       "207   acrin_contralateral_breast_mr   \n",
-       "3891         cc_tumor_heterogeneity   \n",
-       "4000                covid_19_ny_sbu   \n",
-       "4004                covid_19_ny_sbu   \n",
-       "5570                   hcc_tace_seg   \n",
-       "5888                       htan_hms   \n",
-       "5892                       htan_hms   \n",
-       "6194                      htan_ohsu   \n",
-       "6209                      htan_ohsu   \n",
-       "6515                htan_vanderbilt   \n",
-       "6522                htan_vanderbilt   \n",
-       "6766                     htan_wustl   \n",
-       "6768                     htan_wustl   \n",
-       "7141            nsclc_radiogenomics   \n",
-       "7457                       tcga_acc   \n",
-       "7534                      tcga_blca   \n",
-       "7635                      tcga_brca   \n",
-       "7684                      tcga_cesc   \n",
-       "7784                      tcga_chol   \n",
-       "7830                      tcga_coad   \n",
-       "7894                      tcga_dlbc   \n",
-       "7984                      tcga_esca   \n",
-       "8045                       tcga_gbm   \n",
-       "8129                      tcga_hnsc   \n",
-       "8198                      tcga_kich   \n",
-       "8290                      tcga_kirc   \n",
-       "8367                      tcga_kirp   \n",
-       "8435                       tcga_lgg   \n",
-       "8573                      tcga_lihc   \n",
-       "8647                      tcga_luad   \n",
-       "8721                      tcga_lusc   \n",
-       "8857                      tcga_meso   \n",
-       "8900                        tcga_ov   \n",
-       "8978                      tcga_paad   \n",
-       "9043                      tcga_pcpg   \n",
-       "9125                      tcga_prad   \n",
-       "9201                      tcga_read   \n",
-       "9301                      tcga_sarc   \n",
-       "9374                      tcga_skcm   \n",
-       "9423                      tcga_stad   \n",
-       "9497                      tcga_tgct   \n",
-       "9597                      tcga_thca   \n",
-       "9670                      tcga_thym   \n",
-       "9715                      tcga_ucec   \n",
-       "9790                       tcga_ucs   \n",
-       "9892                       tcga_uvm   \n",
-       "\n",
-       "                                                 values  \n",
-       "59    [{'option_code': '1.0', 'option_description': ...  \n",
-       "201   [{'option_code': '1.0', 'option_description': ...  \n",
-       "202   [{'option_code': '1.0', 'option_description': ...  \n",
-       "203   [{'option_code': '1.0', 'option_description': ...  \n",
-       "205   [{'option_code': '1.0', 'option_description': ...  \n",
-       "206   [{'option_code': '1.0', 'option_description': ...  \n",
-       "207   [{'option_code': '1.0', 'option_description': ...  \n",
-       "3891  [{'option_code': 'Non-responder', 'option_desc...  \n",
-       "4000  [{'option_code': 'Yes', 'option_description': ...  \n",
-       "4004  [{'option_code': 'apixaban', 'option_descripti...  \n",
-       "5570  [{'option_code': 'Cisplastin', 'option_descrip...  \n",
-       "5888  [{'option_code': 'FOLFOX', 'option_description...  \n",
-       "5892  [{'option_code': 'No', 'option_description': N...  \n",
-       "6194  [{'option_code': 'Abemaciclib', 'option_descri...  \n",
-       "6209  [{'option_code': 'None', 'option_description':...  \n",
-       "6515  [{'option_code': 'None', 'option_description':...  \n",
-       "6522  [{'option_code': 'Not Reported', 'option_descr...  \n",
-       "6766  [{'option_code': 'Yes', 'option_description': ...  \n",
-       "6768  [{'option_code': 'Fifth', 'option_description'...  \n",
-       "7141  [{'option_code': 'No', 'option_description': N...  \n",
-       "7457  [{'option_code': 'Complete Remission/Response'...  \n",
-       "7534  [{'option_code': 'Complete Remission/Response'...  \n",
-       "7635  [{'option_code': 'None', 'option_description':...  \n",
-       "7684  [{'option_code': 'Complete Remission/Response'...  \n",
-       "7784  [{'option_code': 'None', 'option_description':...  \n",
-       "7830  [{'option_code': 'Complete Remission/Response'...  \n",
-       "7894  [{'option_code': 'Complete Remission/Response'...  \n",
-       "7984  [{'option_code': 'Complete Remission/Response'...  \n",
-       "8045  [{'option_code': 'Complete Remission/Response'...  \n",
-       "8129  [{'option_code': 'Complete Remission/Response'...  \n",
-       "8198  [{'option_code': 'Complete Remission/Response'...  \n",
-       "8290  [{'option_code': 'Complete Remission/Response'...  \n",
-       "8367  [{'option_code': 'Complete Remission/Response'...  \n",
-       "8435  [{'option_code': 'Complete Remission/Response'...  \n",
-       "8573  [{'option_code': 'None', 'option_description':...  \n",
-       "8647  [{'option_code': 'Complete Remission/Response'...  \n",
-       "8721  [{'option_code': 'Complete Remission/Response'...  \n",
-       "8857  [{'option_code': 'None', 'option_description':...  \n",
-       "8900  [{'option_code': 'Complete Remission/Response'...  \n",
-       "8978  [{'option_code': 'Complete Remission/Response'...  \n",
-       "9043  [{'option_code': 'Complete Remission/Response'...  \n",
-       "9125  [{'option_code': 'Complete Remission/Response'...  \n",
-       "9201  [{'option_code': 'Complete Remission/Response'...  \n",
-       "9301  [{'option_code': 'None', 'option_description':...  \n",
-       "9374  [{'option_code': 'None', 'option_description':...  \n",
-       "9423  [{'option_code': 'Complete Remission/Response'...  \n",
-       "9497  [{'option_code': 'Complete Remission/Response'...  \n",
-       "9597  [{'option_code': 'None', 'option_description':...  \n",
-       "9670  [{'option_code': 'None', 'option_description':...  \n",
-       "9715  [{'option_code': 'Complete Remission/Response'...  \n",
-       "9790  [{'option_code': 'Complete Remission/Response'...  \n",
-       "9892  [{'option_code': 'None', 'option_description':...  "
-      ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-b9608398-c631-4d08-abcc-199ec352f902\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>column_label</th>\n",
-       "      <th>collection_id</th>\n",
-       "      <th>values</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>59</th>\n",
-       "      <td>current or recent hx (6 months prior to MRI) c...</td>\n",
-       "      <td>acrin_contralateral_breast_mr</td>\n",
-       "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>201</th>\n",
-       "      <td>current use of estrogen replacement therapy</td>\n",
-       "      <td>acrin_contralateral_breast_mr</td>\n",
-       "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>202</th>\n",
-       "      <td>current use of tamoxifen/serm therapy</td>\n",
-       "      <td>acrin_contralateral_breast_mr</td>\n",
-       "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>203</th>\n",
-       "      <td>current use aromatase inhibitor therapy</td>\n",
-       "      <td>acrin_contralateral_breast_mr</td>\n",
-       "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>205</th>\n",
-       "      <td>past use of estrogen replacement therapy</td>\n",
-       "      <td>acrin_contralateral_breast_mr</td>\n",
-       "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>206</th>\n",
-       "      <td>past use of tamoxifen/serm therapy</td>\n",
-       "      <td>acrin_contralateral_breast_mr</td>\n",
-       "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>207</th>\n",
-       "      <td>prior use aromatase inhibitor therapy</td>\n",
-       "      <td>acrin_contralateral_breast_mr</td>\n",
-       "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3891</th>\n",
-       "      <td>Early response (1-2 months post-therapy)</td>\n",
-       "      <td>cc_tumor_heterogeneity</td>\n",
-       "      <td>[{'option_code': 'Non-responder', 'option_desc...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>4000</th>\n",
-       "      <td>Did the patient have documented renal replacem...</td>\n",
-       "      <td>covid_19_ny_sbu</td>\n",
-       "      <td>[{'option_code': 'Yes', 'option_description': ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>4004</th>\n",
-       "      <td>Patient had other anticoagulation therapy as l...</td>\n",
-       "      <td>covid_19_ny_sbu</td>\n",
-       "      <td>[{'option_code': 'apixaban', 'option_descripti...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>5570</th>\n",
-       "      <td>chemotherapy used for TACE procedure</td>\n",
-       "      <td>hcc_tace_seg</td>\n",
-       "      <td>[{'option_code': 'Cisplastin', 'option_descrip...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>5888</th>\n",
-       "      <td>Regimen_or_Line_of_Therapy</td>\n",
-       "      <td>htan_hms</td>\n",
-       "      <td>[{'option_code': 'FOLFOX', 'option_description...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>5892</th>\n",
-       "      <td>Treatment_or_Therapy</td>\n",
-       "      <td>htan_hms</td>\n",
-       "      <td>[{'option_code': 'No', 'option_description': N...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>6194</th>\n",
-       "      <td>Regimen_or_Line_of_Therapy</td>\n",
-       "      <td>htan_ohsu</td>\n",
-       "      <td>[{'option_code': 'Abemaciclib', 'option_descri...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>6209</th>\n",
-       "      <td>Treatment_or_Therapy</td>\n",
-       "      <td>htan_ohsu</td>\n",
-       "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>6515</th>\n",
-       "      <td>Regimen_or_Line_of_Therapy</td>\n",
-       "      <td>htan_vanderbilt</td>\n",
-       "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>6522</th>\n",
-       "      <td>Treatment_or_Therapy</td>\n",
-       "      <td>htan_vanderbilt</td>\n",
-       "      <td>[{'option_code': 'Not Reported', 'option_descr...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>6766</th>\n",
-       "      <td>Treatment_or_Therapy</td>\n",
-       "      <td>htan_wustl</td>\n",
-       "      <td>[{'option_code': 'Yes', 'option_description': ...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>6768</th>\n",
-       "      <td>Regimen_or_Line_of_Therapy</td>\n",
-       "      <td>htan_wustl</td>\n",
-       "      <td>[{'option_code': 'Fifth', 'option_description'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7141</th>\n",
-       "      <td>Chemotherapy</td>\n",
-       "      <td>nsclc_radiogenomics</td>\n",
-       "      <td>[{'option_code': 'No', 'option_description': N...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7457</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_acc</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7534</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_blca</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7635</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_brca</td>\n",
-       "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7684</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_cesc</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7784</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_chol</td>\n",
-       "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7830</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_coad</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7894</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_dlbc</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7984</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_esca</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8045</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_gbm</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8129</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_hnsc</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8198</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_kich</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8290</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_kirc</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8367</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_kirp</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8435</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_lgg</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8573</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_lihc</td>\n",
-       "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8647</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_luad</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8721</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_lusc</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8857</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_meso</td>\n",
-       "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8900</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_ov</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>8978</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_paad</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9043</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_pcpg</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9125</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_prad</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9201</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_read</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9301</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_sarc</td>\n",
-       "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9374</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_skcm</td>\n",
-       "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9423</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_stad</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9497</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_tgct</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9597</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_thca</td>\n",
-       "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9670</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_thym</td>\n",
-       "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9715</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_ucec</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9790</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_ucs</td>\n",
-       "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>9892</th>\n",
-       "      <td>primary_therapy_outcome_success</td>\n",
-       "      <td>tcga_uvm</td>\n",
-       "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b9608398-c631-4d08-abcc-199ec352f902')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-b9608398-c631-4d08-abcc-199ec352f902 button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-b9608398-c631-4d08-abcc-199ec352f902');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "<div id=\"df-dd67b513-98bf-4711-a26d-2d2e7acdc226\">\n",
-       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-dd67b513-98bf-4711-a26d-2d2e7acdc226')\"\n",
-       "            title=\"Suggest charts\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "     width=\"24px\">\n",
-       "    <g>\n",
-       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
-       "    </g>\n",
-       "</svg>\n",
-       "  </button>\n",
-       "\n",
-       "<style>\n",
-       "  .colab-df-quickchart {\n",
-       "      --bg-color: #E8F0FE;\n",
-       "      --fill-color: #1967D2;\n",
-       "      --hover-bg-color: #E2EBFA;\n",
-       "      --hover-fill-color: #174EA6;\n",
-       "      --disabled-fill-color: #AAA;\n",
-       "      --disabled-bg-color: #DDD;\n",
-       "  }\n",
-       "\n",
-       "  [theme=dark] .colab-df-quickchart {\n",
-       "      --bg-color: #3B4455;\n",
-       "      --fill-color: #D2E3FC;\n",
-       "      --hover-bg-color: #434B5C;\n",
-       "      --hover-fill-color: #FFFFFF;\n",
-       "      --disabled-bg-color: #3B4455;\n",
-       "      --disabled-fill-color: #666;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart {\n",
-       "    background-color: var(--bg-color);\n",
-       "    border: none;\n",
-       "    border-radius: 50%;\n",
-       "    cursor: pointer;\n",
-       "    display: none;\n",
-       "    fill: var(--fill-color);\n",
-       "    height: 32px;\n",
-       "    padding: 0;\n",
-       "    width: 32px;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart:hover {\n",
-       "    background-color: var(--hover-bg-color);\n",
-       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "    fill: var(--button-hover-fill-color);\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart-complete:disabled,\n",
-       "  .colab-df-quickchart-complete:disabled:hover {\n",
-       "    background-color: var(--disabled-bg-color);\n",
-       "    fill: var(--disabled-fill-color);\n",
-       "    box-shadow: none;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-spinner {\n",
-       "    border: 2px solid var(--fill-color);\n",
-       "    border-color: transparent;\n",
-       "    border-bottom-color: var(--fill-color);\n",
-       "    animation:\n",
-       "      spin 1s steps(1) infinite;\n",
-       "  }\n",
-       "\n",
-       "  @keyframes spin {\n",
-       "    0% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "      border-left-color: var(--fill-color);\n",
-       "    }\n",
-       "    20% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    30% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    40% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    60% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    80% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "    90% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "  }\n",
-       "</style>\n",
-       "\n",
-       "  <script>\n",
-       "    async function quickchart(key) {\n",
-       "      const quickchartButtonEl =\n",
-       "        document.querySelector('#' + key + ' button');\n",
-       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
-       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
-       "      try {\n",
-       "        const charts = await google.colab.kernel.invokeFunction(\n",
-       "            'suggestCharts', [key], {});\n",
-       "      } catch (error) {\n",
-       "        console.error('Error during call to suggestCharts:', error);\n",
-       "      }\n",
-       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
-       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
-       "    }\n",
-       "    (() => {\n",
-       "      let quickchartButtonEl =\n",
-       "        document.querySelector('#df-dd67b513-98bf-4711-a26d-2d2e7acdc226 button');\n",
-       "      quickchartButtonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "    })();\n",
-       "  </script>\n",
-       "</div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
+      "cell_type": "code",
+      "execution_count": 29,
+      "metadata": {
+        "id": "tt3htrbxr6KC",
+        "outputId": "bbceac91-6c1d-4479-e7f7-c9ed930e9a5e",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 479
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "                                           column_label\n",
+              "0                           idc_provenance_source_batch\n",
+              "1        T0 (baseline) MRI study included in collection\n",
+              "2        T1 (early-Tx) MRI study included in collection\n",
+              "3     T2 (inter-regimen) MRI study included in colle...\n",
+              "4     T3 (pre-surgery) MRI study included in collection\n",
+              "...                                                 ...\n",
+              "3707                                                 ID\n",
+              "3708                 Survival_from_surgery_days_UPDATED\n",
+              "3709                                    Survival_Censor\n",
+              "3710                          Time_since_baseline_preop\n",
+              "3711                                  Age_at_scan_years\n",
+              "\n",
+              "[3712 rows x 1 columns]"
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-2020b699-f599-4c35-956e-bf344abc3aa3\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>column_label</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>0</th>\n",
+              "      <td>idc_provenance_source_batch</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1</th>\n",
+              "      <td>T0 (baseline) MRI study included in collection</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>2</th>\n",
+              "      <td>T1 (early-Tx) MRI study included in collection</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3</th>\n",
+              "      <td>T2 (inter-regimen) MRI study included in colle...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>4</th>\n",
+              "      <td>T3 (pre-surgery) MRI study included in collection</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>...</th>\n",
+              "      <td>...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3707</th>\n",
+              "      <td>ID</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3708</th>\n",
+              "      <td>Survival_from_surgery_days_UPDATED</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3709</th>\n",
+              "      <td>Survival_Censor</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3710</th>\n",
+              "      <td>Time_since_baseline_preop</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3711</th>\n",
+              "      <td>Age_at_scan_years</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "<p>3712 rows × 1 columns</p>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2020b699-f599-4c35-956e-bf344abc3aa3')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-2020b699-f599-4c35-956e-bf344abc3aa3 button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-2020b699-f599-4c35-956e-bf344abc3aa3');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "<div id=\"df-8ff08438-cc9c-4303-a321-1f2a80b9269c\">\n",
+              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-8ff08438-cc9c-4303-a321-1f2a80b9269c')\"\n",
+              "            title=\"Suggest charts\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "     width=\"24px\">\n",
+              "    <g>\n",
+              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+              "    </g>\n",
+              "</svg>\n",
+              "  </button>\n",
+              "\n",
+              "<style>\n",
+              "  .colab-df-quickchart {\n",
+              "      --bg-color: #E8F0FE;\n",
+              "      --fill-color: #1967D2;\n",
+              "      --hover-bg-color: #E2EBFA;\n",
+              "      --hover-fill-color: #174EA6;\n",
+              "      --disabled-fill-color: #AAA;\n",
+              "      --disabled-bg-color: #DDD;\n",
+              "  }\n",
+              "\n",
+              "  [theme=dark] .colab-df-quickchart {\n",
+              "      --bg-color: #3B4455;\n",
+              "      --fill-color: #D2E3FC;\n",
+              "      --hover-bg-color: #434B5C;\n",
+              "      --hover-fill-color: #FFFFFF;\n",
+              "      --disabled-bg-color: #3B4455;\n",
+              "      --disabled-fill-color: #666;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart {\n",
+              "    background-color: var(--bg-color);\n",
+              "    border: none;\n",
+              "    border-radius: 50%;\n",
+              "    cursor: pointer;\n",
+              "    display: none;\n",
+              "    fill: var(--fill-color);\n",
+              "    height: 32px;\n",
+              "    padding: 0;\n",
+              "    width: 32px;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart:hover {\n",
+              "    background-color: var(--hover-bg-color);\n",
+              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "    fill: var(--button-hover-fill-color);\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart-complete:disabled,\n",
+              "  .colab-df-quickchart-complete:disabled:hover {\n",
+              "    background-color: var(--disabled-bg-color);\n",
+              "    fill: var(--disabled-fill-color);\n",
+              "    box-shadow: none;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-spinner {\n",
+              "    border: 2px solid var(--fill-color);\n",
+              "    border-color: transparent;\n",
+              "    border-bottom-color: var(--fill-color);\n",
+              "    animation:\n",
+              "      spin 1s steps(1) infinite;\n",
+              "  }\n",
+              "\n",
+              "  @keyframes spin {\n",
+              "    0% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "      border-left-color: var(--fill-color);\n",
+              "    }\n",
+              "    20% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    30% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    40% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    60% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    80% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "    90% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "  }\n",
+              "</style>\n",
+              "\n",
+              "  <script>\n",
+              "    async function quickchart(key) {\n",
+              "      const quickchartButtonEl =\n",
+              "        document.querySelector('#' + key + ' button');\n",
+              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+              "      try {\n",
+              "        const charts = await google.colab.kernel.invokeFunction(\n",
+              "            'suggestCharts', [key], {});\n",
+              "      } catch (error) {\n",
+              "        console.error('Error during call to suggestCharts:', error);\n",
+              "      }\n",
+              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+              "    }\n",
+              "    (() => {\n",
+              "      let quickchartButtonEl =\n",
+              "        document.querySelector('#df-8ff08438-cc9c-4303-a321-1f2a80b9269c button');\n",
+              "      quickchartButtonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "    })();\n",
+              "  </script>\n",
+              "</div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "summary": "{\n  \"name\": \"pd\",\n  \"rows\": 3712,\n  \"fields\": [\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 3712,\n        \"samples\": [\n          \"TNM staging \",\n          \"BASELINE SYMPTOMS - SYMPTOM GRADE 19\",\n          \"DATE FORM COMPLETED (MM/DD/YYYY)\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 29
+        }
       ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "summary": "{\n  \"name\": \"c\",\n  \"rows\": 52,\n  \"fields\": [\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 15,\n        \"samples\": [\n          \"Patient had other anticoagulation therapy as listed\",\n          \"Regimen_or_Line_of_Therapy\",\n          \"current or recent hx (6 months prior to MRI) chemo therapy\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 41,\n        \"samples\": [\n          \"tcga_luad\",\n          \"tcga_chol\",\n          \"nsclc_radiogenomics\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
-      }
-     },
-     "metadata": {},
-     "execution_count": 31
-    }
-   ],
-   "source": [
-    "c.clinical_index[c.clinical_index[\"column_label\"].str.contains(\"[tT]herapy\", na=False)][[\"column_label\", \"collection_id\", \"values\"]]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "V23hIu1jo-We"
-   },
-   "source": [
-    "We observe that there are several collections that contain column named \"Chemotherapy\". Let's filter these values further, in order to identify subjects that underwent chemotherapy."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 32,
-   "metadata": {
-    "id": "uACJ3N_GFHFG",
-    "outputId": "54733fc0-defa-4374-e88d-d24909034628",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 571
-    }
-   },
-   "outputs": [
+      "source": [
+        "import pandas as pd\n",
+        "\n",
+        "pd.DataFrame({\"column_label\":c.clinical_index[\"column_label\"].unique()})"
+      ]
+    },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "7iZpB1Foi1hd"
+      },
+      "source": [
+        "Let's say we want to know which of the cases have information related to therapy. We can search column metadata for the presence of word \"therapy\" (since the terms in clinical data are not harmonized, we need to account for the variability in capitalization)."
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "            collection_id                                         table_name  \\\n",
-       "5570         hcc_tace_seg  bigquery-public-data.idc_v20_clinical.hcc_tace...   \n",
-       "7141  nsclc_radiogenomics  bigquery-public-data.idc_v20_clinical.nsclc_ra...   \n",
-       "\n",
-       "            column                           column_label  \\\n",
-       "5570  chemotherapy  chemotherapy used for TACE procedure    \n",
-       "7141  chemotherapy                           Chemotherapy   \n",
-       "\n",
-       "                                                 values  \n",
-       "5570  [{'option_code': 'Cisplastin', 'option_descrip...  \n",
-       "7141  [{'option_code': 'No', 'option_description': N...  "
-      ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-8e270772-8c27-4911-8d96-27fdfaf1e4a1\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>collection_id</th>\n",
-       "      <th>table_name</th>\n",
-       "      <th>column</th>\n",
-       "      <th>column_label</th>\n",
-       "      <th>values</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>5570</th>\n",
-       "      <td>hcc_tace_seg</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.hcc_tace...</td>\n",
-       "      <td>chemotherapy</td>\n",
-       "      <td>chemotherapy used for TACE procedure</td>\n",
-       "      <td>[{'option_code': 'Cisplastin', 'option_descrip...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>7141</th>\n",
-       "      <td>nsclc_radiogenomics</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.nsclc_ra...</td>\n",
-       "      <td>chemotherapy</td>\n",
-       "      <td>Chemotherapy</td>\n",
-       "      <td>[{'option_code': 'No', 'option_description': N...</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8e270772-8c27-4911-8d96-27fdfaf1e4a1')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-8e270772-8c27-4911-8d96-27fdfaf1e4a1 button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-8e270772-8c27-4911-8d96-27fdfaf1e4a1');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "<div id=\"df-b3c61081-60b8-4f72-bace-11c58c51131c\">\n",
-       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b3c61081-60b8-4f72-bace-11c58c51131c')\"\n",
-       "            title=\"Suggest charts\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "     width=\"24px\">\n",
-       "    <g>\n",
-       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
-       "    </g>\n",
-       "</svg>\n",
-       "  </button>\n",
-       "\n",
-       "<style>\n",
-       "  .colab-df-quickchart {\n",
-       "      --bg-color: #E8F0FE;\n",
-       "      --fill-color: #1967D2;\n",
-       "      --hover-bg-color: #E2EBFA;\n",
-       "      --hover-fill-color: #174EA6;\n",
-       "      --disabled-fill-color: #AAA;\n",
-       "      --disabled-bg-color: #DDD;\n",
-       "  }\n",
-       "\n",
-       "  [theme=dark] .colab-df-quickchart {\n",
-       "      --bg-color: #3B4455;\n",
-       "      --fill-color: #D2E3FC;\n",
-       "      --hover-bg-color: #434B5C;\n",
-       "      --hover-fill-color: #FFFFFF;\n",
-       "      --disabled-bg-color: #3B4455;\n",
-       "      --disabled-fill-color: #666;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart {\n",
-       "    background-color: var(--bg-color);\n",
-       "    border: none;\n",
-       "    border-radius: 50%;\n",
-       "    cursor: pointer;\n",
-       "    display: none;\n",
-       "    fill: var(--fill-color);\n",
-       "    height: 32px;\n",
-       "    padding: 0;\n",
-       "    width: 32px;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart:hover {\n",
-       "    background-color: var(--hover-bg-color);\n",
-       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "    fill: var(--button-hover-fill-color);\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart-complete:disabled,\n",
-       "  .colab-df-quickchart-complete:disabled:hover {\n",
-       "    background-color: var(--disabled-bg-color);\n",
-       "    fill: var(--disabled-fill-color);\n",
-       "    box-shadow: none;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-spinner {\n",
-       "    border: 2px solid var(--fill-color);\n",
-       "    border-color: transparent;\n",
-       "    border-bottom-color: var(--fill-color);\n",
-       "    animation:\n",
-       "      spin 1s steps(1) infinite;\n",
-       "  }\n",
-       "\n",
-       "  @keyframes spin {\n",
-       "    0% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "      border-left-color: var(--fill-color);\n",
-       "    }\n",
-       "    20% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    30% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    40% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    60% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    80% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "    90% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "  }\n",
-       "</style>\n",
-       "\n",
-       "  <script>\n",
-       "    async function quickchart(key) {\n",
-       "      const quickchartButtonEl =\n",
-       "        document.querySelector('#' + key + ' button');\n",
-       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
-       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
-       "      try {\n",
-       "        const charts = await google.colab.kernel.invokeFunction(\n",
-       "            'suggestCharts', [key], {});\n",
-       "      } catch (error) {\n",
-       "        console.error('Error during call to suggestCharts:', error);\n",
-       "      }\n",
-       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
-       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
-       "    }\n",
-       "    (() => {\n",
-       "      let quickchartButtonEl =\n",
-       "        document.querySelector('#df-b3c61081-60b8-4f72-bace-11c58c51131c button');\n",
-       "      quickchartButtonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "    })();\n",
-       "  </script>\n",
-       "</div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
+      "cell_type": "code",
+      "execution_count": 31,
+      "metadata": {
+        "id": "qCTFR-QpjJSw",
+        "outputId": "9a3e6dfb-497c-4e4d-d12e-f0dbe6490208",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 1000
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "                                           column_label  \\\n",
+              "59    current or recent hx (6 months prior to MRI) c...   \n",
+              "201         current use of estrogen replacement therapy   \n",
+              "202               current use of tamoxifen/serm therapy   \n",
+              "203             current use aromatase inhibitor therapy   \n",
+              "205            past use of estrogen replacement therapy   \n",
+              "206                  past use of tamoxifen/serm therapy   \n",
+              "207               prior use aromatase inhibitor therapy   \n",
+              "3891           Early response (1-2 months post-therapy)   \n",
+              "4000  Did the patient have documented renal replacem...   \n",
+              "4004  Patient had other anticoagulation therapy as l...   \n",
+              "5570              chemotherapy used for TACE procedure    \n",
+              "5888                         Regimen_or_Line_of_Therapy   \n",
+              "5892                               Treatment_or_Therapy   \n",
+              "6194                         Regimen_or_Line_of_Therapy   \n",
+              "6209                               Treatment_or_Therapy   \n",
+              "6515                         Regimen_or_Line_of_Therapy   \n",
+              "6522                               Treatment_or_Therapy   \n",
+              "6766                               Treatment_or_Therapy   \n",
+              "6768                         Regimen_or_Line_of_Therapy   \n",
+              "7141                                       Chemotherapy   \n",
+              "7457                    primary_therapy_outcome_success   \n",
+              "7534                    primary_therapy_outcome_success   \n",
+              "7635                    primary_therapy_outcome_success   \n",
+              "7684                    primary_therapy_outcome_success   \n",
+              "7784                    primary_therapy_outcome_success   \n",
+              "7830                    primary_therapy_outcome_success   \n",
+              "7894                    primary_therapy_outcome_success   \n",
+              "7984                    primary_therapy_outcome_success   \n",
+              "8045                    primary_therapy_outcome_success   \n",
+              "8129                    primary_therapy_outcome_success   \n",
+              "8198                    primary_therapy_outcome_success   \n",
+              "8290                    primary_therapy_outcome_success   \n",
+              "8367                    primary_therapy_outcome_success   \n",
+              "8435                    primary_therapy_outcome_success   \n",
+              "8573                    primary_therapy_outcome_success   \n",
+              "8647                    primary_therapy_outcome_success   \n",
+              "8721                    primary_therapy_outcome_success   \n",
+              "8857                    primary_therapy_outcome_success   \n",
+              "8900                    primary_therapy_outcome_success   \n",
+              "8978                    primary_therapy_outcome_success   \n",
+              "9043                    primary_therapy_outcome_success   \n",
+              "9125                    primary_therapy_outcome_success   \n",
+              "9201                    primary_therapy_outcome_success   \n",
+              "9301                    primary_therapy_outcome_success   \n",
+              "9374                    primary_therapy_outcome_success   \n",
+              "9423                    primary_therapy_outcome_success   \n",
+              "9497                    primary_therapy_outcome_success   \n",
+              "9597                    primary_therapy_outcome_success   \n",
+              "9670                    primary_therapy_outcome_success   \n",
+              "9715                    primary_therapy_outcome_success   \n",
+              "9790                    primary_therapy_outcome_success   \n",
+              "9892                    primary_therapy_outcome_success   \n",
+              "\n",
+              "                      collection_id  \\\n",
+              "59    acrin_contralateral_breast_mr   \n",
+              "201   acrin_contralateral_breast_mr   \n",
+              "202   acrin_contralateral_breast_mr   \n",
+              "203   acrin_contralateral_breast_mr   \n",
+              "205   acrin_contralateral_breast_mr   \n",
+              "206   acrin_contralateral_breast_mr   \n",
+              "207   acrin_contralateral_breast_mr   \n",
+              "3891         cc_tumor_heterogeneity   \n",
+              "4000                covid_19_ny_sbu   \n",
+              "4004                covid_19_ny_sbu   \n",
+              "5570                   hcc_tace_seg   \n",
+              "5888                       htan_hms   \n",
+              "5892                       htan_hms   \n",
+              "6194                      htan_ohsu   \n",
+              "6209                      htan_ohsu   \n",
+              "6515                htan_vanderbilt   \n",
+              "6522                htan_vanderbilt   \n",
+              "6766                     htan_wustl   \n",
+              "6768                     htan_wustl   \n",
+              "7141            nsclc_radiogenomics   \n",
+              "7457                       tcga_acc   \n",
+              "7534                      tcga_blca   \n",
+              "7635                      tcga_brca   \n",
+              "7684                      tcga_cesc   \n",
+              "7784                      tcga_chol   \n",
+              "7830                      tcga_coad   \n",
+              "7894                      tcga_dlbc   \n",
+              "7984                      tcga_esca   \n",
+              "8045                       tcga_gbm   \n",
+              "8129                      tcga_hnsc   \n",
+              "8198                      tcga_kich   \n",
+              "8290                      tcga_kirc   \n",
+              "8367                      tcga_kirp   \n",
+              "8435                       tcga_lgg   \n",
+              "8573                      tcga_lihc   \n",
+              "8647                      tcga_luad   \n",
+              "8721                      tcga_lusc   \n",
+              "8857                      tcga_meso   \n",
+              "8900                        tcga_ov   \n",
+              "8978                      tcga_paad   \n",
+              "9043                      tcga_pcpg   \n",
+              "9125                      tcga_prad   \n",
+              "9201                      tcga_read   \n",
+              "9301                      tcga_sarc   \n",
+              "9374                      tcga_skcm   \n",
+              "9423                      tcga_stad   \n",
+              "9497                      tcga_tgct   \n",
+              "9597                      tcga_thca   \n",
+              "9670                      tcga_thym   \n",
+              "9715                      tcga_ucec   \n",
+              "9790                       tcga_ucs   \n",
+              "9892                       tcga_uvm   \n",
+              "\n",
+              "                                                 values  \n",
+              "59    [{'option_code': '1.0', 'option_description': ...  \n",
+              "201   [{'option_code': '1.0', 'option_description': ...  \n",
+              "202   [{'option_code': '1.0', 'option_description': ...  \n",
+              "203   [{'option_code': '1.0', 'option_description': ...  \n",
+              "205   [{'option_code': '1.0', 'option_description': ...  \n",
+              "206   [{'option_code': '1.0', 'option_description': ...  \n",
+              "207   [{'option_code': '1.0', 'option_description': ...  \n",
+              "3891  [{'option_code': 'Non-responder', 'option_desc...  \n",
+              "4000  [{'option_code': 'Yes', 'option_description': ...  \n",
+              "4004  [{'option_code': 'apixaban', 'option_descripti...  \n",
+              "5570  [{'option_code': 'Cisplastin', 'option_descrip...  \n",
+              "5888  [{'option_code': 'FOLFOX', 'option_description...  \n",
+              "5892  [{'option_code': 'No', 'option_description': N...  \n",
+              "6194  [{'option_code': 'Abemaciclib', 'option_descri...  \n",
+              "6209  [{'option_code': 'None', 'option_description':...  \n",
+              "6515  [{'option_code': 'None', 'option_description':...  \n",
+              "6522  [{'option_code': 'Not Reported', 'option_descr...  \n",
+              "6766  [{'option_code': 'Yes', 'option_description': ...  \n",
+              "6768  [{'option_code': 'Fifth', 'option_description'...  \n",
+              "7141  [{'option_code': 'No', 'option_description': N...  \n",
+              "7457  [{'option_code': 'Complete Remission/Response'...  \n",
+              "7534  [{'option_code': 'Complete Remission/Response'...  \n",
+              "7635  [{'option_code': 'None', 'option_description':...  \n",
+              "7684  [{'option_code': 'Complete Remission/Response'...  \n",
+              "7784  [{'option_code': 'None', 'option_description':...  \n",
+              "7830  [{'option_code': 'Complete Remission/Response'...  \n",
+              "7894  [{'option_code': 'Complete Remission/Response'...  \n",
+              "7984  [{'option_code': 'Complete Remission/Response'...  \n",
+              "8045  [{'option_code': 'Complete Remission/Response'...  \n",
+              "8129  [{'option_code': 'Complete Remission/Response'...  \n",
+              "8198  [{'option_code': 'Complete Remission/Response'...  \n",
+              "8290  [{'option_code': 'Complete Remission/Response'...  \n",
+              "8367  [{'option_code': 'Complete Remission/Response'...  \n",
+              "8435  [{'option_code': 'Complete Remission/Response'...  \n",
+              "8573  [{'option_code': 'None', 'option_description':...  \n",
+              "8647  [{'option_code': 'Complete Remission/Response'...  \n",
+              "8721  [{'option_code': 'Complete Remission/Response'...  \n",
+              "8857  [{'option_code': 'None', 'option_description':...  \n",
+              "8900  [{'option_code': 'Complete Remission/Response'...  \n",
+              "8978  [{'option_code': 'Complete Remission/Response'...  \n",
+              "9043  [{'option_code': 'Complete Remission/Response'...  \n",
+              "9125  [{'option_code': 'Complete Remission/Response'...  \n",
+              "9201  [{'option_code': 'Complete Remission/Response'...  \n",
+              "9301  [{'option_code': 'None', 'option_description':...  \n",
+              "9374  [{'option_code': 'None', 'option_description':...  \n",
+              "9423  [{'option_code': 'Complete Remission/Response'...  \n",
+              "9497  [{'option_code': 'Complete Remission/Response'...  \n",
+              "9597  [{'option_code': 'None', 'option_description':...  \n",
+              "9670  [{'option_code': 'None', 'option_description':...  \n",
+              "9715  [{'option_code': 'Complete Remission/Response'...  \n",
+              "9790  [{'option_code': 'Complete Remission/Response'...  \n",
+              "9892  [{'option_code': 'None', 'option_description':...  "
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-b9608398-c631-4d08-abcc-199ec352f902\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>column_label</th>\n",
+              "      <th>collection_id</th>\n",
+              "      <th>values</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>59</th>\n",
+              "      <td>current or recent hx (6 months prior to MRI) c...</td>\n",
+              "      <td>acrin_contralateral_breast_mr</td>\n",
+              "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>201</th>\n",
+              "      <td>current use of estrogen replacement therapy</td>\n",
+              "      <td>acrin_contralateral_breast_mr</td>\n",
+              "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>202</th>\n",
+              "      <td>current use of tamoxifen/serm therapy</td>\n",
+              "      <td>acrin_contralateral_breast_mr</td>\n",
+              "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>203</th>\n",
+              "      <td>current use aromatase inhibitor therapy</td>\n",
+              "      <td>acrin_contralateral_breast_mr</td>\n",
+              "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>205</th>\n",
+              "      <td>past use of estrogen replacement therapy</td>\n",
+              "      <td>acrin_contralateral_breast_mr</td>\n",
+              "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>206</th>\n",
+              "      <td>past use of tamoxifen/serm therapy</td>\n",
+              "      <td>acrin_contralateral_breast_mr</td>\n",
+              "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>207</th>\n",
+              "      <td>prior use aromatase inhibitor therapy</td>\n",
+              "      <td>acrin_contralateral_breast_mr</td>\n",
+              "      <td>[{'option_code': '1.0', 'option_description': ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3891</th>\n",
+              "      <td>Early response (1-2 months post-therapy)</td>\n",
+              "      <td>cc_tumor_heterogeneity</td>\n",
+              "      <td>[{'option_code': 'Non-responder', 'option_desc...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>4000</th>\n",
+              "      <td>Did the patient have documented renal replacem...</td>\n",
+              "      <td>covid_19_ny_sbu</td>\n",
+              "      <td>[{'option_code': 'Yes', 'option_description': ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>4004</th>\n",
+              "      <td>Patient had other anticoagulation therapy as l...</td>\n",
+              "      <td>covid_19_ny_sbu</td>\n",
+              "      <td>[{'option_code': 'apixaban', 'option_descripti...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>5570</th>\n",
+              "      <td>chemotherapy used for TACE procedure</td>\n",
+              "      <td>hcc_tace_seg</td>\n",
+              "      <td>[{'option_code': 'Cisplastin', 'option_descrip...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>5888</th>\n",
+              "      <td>Regimen_or_Line_of_Therapy</td>\n",
+              "      <td>htan_hms</td>\n",
+              "      <td>[{'option_code': 'FOLFOX', 'option_description...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>5892</th>\n",
+              "      <td>Treatment_or_Therapy</td>\n",
+              "      <td>htan_hms</td>\n",
+              "      <td>[{'option_code': 'No', 'option_description': N...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>6194</th>\n",
+              "      <td>Regimen_or_Line_of_Therapy</td>\n",
+              "      <td>htan_ohsu</td>\n",
+              "      <td>[{'option_code': 'Abemaciclib', 'option_descri...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>6209</th>\n",
+              "      <td>Treatment_or_Therapy</td>\n",
+              "      <td>htan_ohsu</td>\n",
+              "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>6515</th>\n",
+              "      <td>Regimen_or_Line_of_Therapy</td>\n",
+              "      <td>htan_vanderbilt</td>\n",
+              "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>6522</th>\n",
+              "      <td>Treatment_or_Therapy</td>\n",
+              "      <td>htan_vanderbilt</td>\n",
+              "      <td>[{'option_code': 'Not Reported', 'option_descr...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>6766</th>\n",
+              "      <td>Treatment_or_Therapy</td>\n",
+              "      <td>htan_wustl</td>\n",
+              "      <td>[{'option_code': 'Yes', 'option_description': ...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>6768</th>\n",
+              "      <td>Regimen_or_Line_of_Therapy</td>\n",
+              "      <td>htan_wustl</td>\n",
+              "      <td>[{'option_code': 'Fifth', 'option_description'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7141</th>\n",
+              "      <td>Chemotherapy</td>\n",
+              "      <td>nsclc_radiogenomics</td>\n",
+              "      <td>[{'option_code': 'No', 'option_description': N...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7457</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_acc</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7534</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_blca</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7635</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_brca</td>\n",
+              "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7684</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_cesc</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7784</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_chol</td>\n",
+              "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7830</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_coad</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7894</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_dlbc</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7984</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_esca</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8045</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_gbm</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8129</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_hnsc</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8198</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_kich</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8290</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_kirc</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8367</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_kirp</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8435</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_lgg</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8573</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_lihc</td>\n",
+              "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8647</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_luad</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8721</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_lusc</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8857</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_meso</td>\n",
+              "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8900</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_ov</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>8978</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_paad</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9043</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_pcpg</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9125</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_prad</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9201</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_read</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9301</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_sarc</td>\n",
+              "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9374</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_skcm</td>\n",
+              "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9423</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_stad</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9497</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_tgct</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9597</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_thca</td>\n",
+              "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9670</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_thym</td>\n",
+              "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9715</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_ucec</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9790</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_ucs</td>\n",
+              "      <td>[{'option_code': 'Complete Remission/Response'...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>9892</th>\n",
+              "      <td>primary_therapy_outcome_success</td>\n",
+              "      <td>tcga_uvm</td>\n",
+              "      <td>[{'option_code': 'None', 'option_description':...</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b9608398-c631-4d08-abcc-199ec352f902')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-b9608398-c631-4d08-abcc-199ec352f902 button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-b9608398-c631-4d08-abcc-199ec352f902');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "<div id=\"df-dd67b513-98bf-4711-a26d-2d2e7acdc226\">\n",
+              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-dd67b513-98bf-4711-a26d-2d2e7acdc226')\"\n",
+              "            title=\"Suggest charts\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "     width=\"24px\">\n",
+              "    <g>\n",
+              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+              "    </g>\n",
+              "</svg>\n",
+              "  </button>\n",
+              "\n",
+              "<style>\n",
+              "  .colab-df-quickchart {\n",
+              "      --bg-color: #E8F0FE;\n",
+              "      --fill-color: #1967D2;\n",
+              "      --hover-bg-color: #E2EBFA;\n",
+              "      --hover-fill-color: #174EA6;\n",
+              "      --disabled-fill-color: #AAA;\n",
+              "      --disabled-bg-color: #DDD;\n",
+              "  }\n",
+              "\n",
+              "  [theme=dark] .colab-df-quickchart {\n",
+              "      --bg-color: #3B4455;\n",
+              "      --fill-color: #D2E3FC;\n",
+              "      --hover-bg-color: #434B5C;\n",
+              "      --hover-fill-color: #FFFFFF;\n",
+              "      --disabled-bg-color: #3B4455;\n",
+              "      --disabled-fill-color: #666;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart {\n",
+              "    background-color: var(--bg-color);\n",
+              "    border: none;\n",
+              "    border-radius: 50%;\n",
+              "    cursor: pointer;\n",
+              "    display: none;\n",
+              "    fill: var(--fill-color);\n",
+              "    height: 32px;\n",
+              "    padding: 0;\n",
+              "    width: 32px;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart:hover {\n",
+              "    background-color: var(--hover-bg-color);\n",
+              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "    fill: var(--button-hover-fill-color);\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart-complete:disabled,\n",
+              "  .colab-df-quickchart-complete:disabled:hover {\n",
+              "    background-color: var(--disabled-bg-color);\n",
+              "    fill: var(--disabled-fill-color);\n",
+              "    box-shadow: none;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-spinner {\n",
+              "    border: 2px solid var(--fill-color);\n",
+              "    border-color: transparent;\n",
+              "    border-bottom-color: var(--fill-color);\n",
+              "    animation:\n",
+              "      spin 1s steps(1) infinite;\n",
+              "  }\n",
+              "\n",
+              "  @keyframes spin {\n",
+              "    0% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "      border-left-color: var(--fill-color);\n",
+              "    }\n",
+              "    20% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    30% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    40% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    60% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    80% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "    90% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "  }\n",
+              "</style>\n",
+              "\n",
+              "  <script>\n",
+              "    async function quickchart(key) {\n",
+              "      const quickchartButtonEl =\n",
+              "        document.querySelector('#' + key + ' button');\n",
+              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+              "      try {\n",
+              "        const charts = await google.colab.kernel.invokeFunction(\n",
+              "            'suggestCharts', [key], {});\n",
+              "      } catch (error) {\n",
+              "        console.error('Error during call to suggestCharts:', error);\n",
+              "      }\n",
+              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+              "    }\n",
+              "    (() => {\n",
+              "      let quickchartButtonEl =\n",
+              "        document.querySelector('#df-dd67b513-98bf-4711-a26d-2d2e7acdc226 button');\n",
+              "      quickchartButtonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "    })();\n",
+              "  </script>\n",
+              "</div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "summary": "{\n  \"name\": \"c\",\n  \"rows\": 52,\n  \"fields\": [\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 15,\n        \"samples\": [\n          \"Patient had other anticoagulation therapy as listed\",\n          \"Regimen_or_Line_of_Therapy\",\n          \"current or recent hx (6 months prior to MRI) chemo therapy\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 41,\n        \"samples\": [\n          \"tcga_luad\",\n          \"tcga_chol\",\n          \"nsclc_radiogenomics\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 31
+        }
       ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "summary": "{\n  \"name\": \"c\",\n  \"rows\": 2,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"nsclc_radiogenomics\",\n          \"hcc_tace_seg\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"table_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"bigquery-public-data.idc_v20_clinical.nsclc_radiogenomics_clinical\",\n          \"bigquery-public-data.idc_v20_clinical.hcc_tace_seg_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"chemotherapy\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"Chemotherapy\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
-      }
-     },
-     "metadata": {},
-     "execution_count": 32
+      "source": [
+        "c.clinical_index[c.clinical_index[\"column_label\"].str.contains(\"[tT]herapy\", na=False)][[\"column_label\", \"collection_id\", \"values\"]]"
+      ]
     },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/data_table.py:200: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  return self._dataframe._repr_html_()  # pylint: disable=protected-access\n"
-     ]
-    }
-   ],
-   "source": [
-    "c.clinical_index[c.clinical_index[\"column_label\"].str.contains(\"[Cc]hemotherapy\", na=False)][[ \"collection_id\", \"table_name\", \"column\", \"column_label\",\"values\"]]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "kke1DZgwGHuA"
-   },
-   "source": [
-    "From the table above we can observe that there are several collections that have clinical metadata related to chemotherapy regimen of the subject.\n",
-    "\n",
-    "Looking at the value sets for the collections/columns that have those, we can observe that subjects that had any chemotherapy could be selected as follows for the respective collections (non-exhaustive list):\n",
-    "* `nsclc_radiogenomics`: subjects that have value `Yes` in table `nsclc_radiogenomics_clinical` column `chemotherapy`\n",
-    "* `hcc_tace_seg` column `chemotherapy` in the `hcc_tace_seg_clinical` table\n",
-    "\n",
-    "Let's focus on the clinical data related to chemotherapy for the collection `hcc_tace_seg`."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 33,
-   "metadata": {
-    "id": "EhrEq84tezyL",
-    "outputId": "b1982490-a95f-458b-a8a1-a47aec802ed4",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 171
-    }
-   },
-   "outputs": [
-    {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "V23hIu1jo-We"
+      },
+      "source": [
+        "We observe that there are several collections that contain column named \"Chemotherapy\". Let's filter these values further, in order to identify subjects that underwent chemotherapy."
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "     collection_id                                         table_name  \\\n",
-       "5570  hcc_tace_seg  bigquery-public-data.idc_v20_clinical.hcc_tace...   \n",
-       "\n",
-       "            column                           column_label  \\\n",
-       "5570  chemotherapy  chemotherapy used for TACE procedure    \n",
-       "\n",
-       "                                                 values  \n",
-       "5570  [{'option_code': 'Cisplastin', 'option_descrip...  "
+      "cell_type": "code",
+      "execution_count": 32,
+      "metadata": {
+        "id": "uACJ3N_GFHFG",
+        "outputId": "54733fc0-defa-4374-e88d-d24909034628",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 571
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "            collection_id                                         table_name  \\\n",
+              "5570         hcc_tace_seg  bigquery-public-data.idc_v20_clinical.hcc_tace...   \n",
+              "7141  nsclc_radiogenomics  bigquery-public-data.idc_v20_clinical.nsclc_ra...   \n",
+              "\n",
+              "            column                           column_label  \\\n",
+              "5570  chemotherapy  chemotherapy used for TACE procedure    \n",
+              "7141  chemotherapy                           Chemotherapy   \n",
+              "\n",
+              "                                                 values  \n",
+              "5570  [{'option_code': 'Cisplastin', 'option_descrip...  \n",
+              "7141  [{'option_code': 'No', 'option_description': N...  "
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-8e270772-8c27-4911-8d96-27fdfaf1e4a1\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>collection_id</th>\n",
+              "      <th>table_name</th>\n",
+              "      <th>column</th>\n",
+              "      <th>column_label</th>\n",
+              "      <th>values</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>5570</th>\n",
+              "      <td>hcc_tace_seg</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.hcc_tace...</td>\n",
+              "      <td>chemotherapy</td>\n",
+              "      <td>chemotherapy used for TACE procedure</td>\n",
+              "      <td>[{'option_code': 'Cisplastin', 'option_descrip...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>7141</th>\n",
+              "      <td>nsclc_radiogenomics</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.nsclc_ra...</td>\n",
+              "      <td>chemotherapy</td>\n",
+              "      <td>Chemotherapy</td>\n",
+              "      <td>[{'option_code': 'No', 'option_description': N...</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8e270772-8c27-4911-8d96-27fdfaf1e4a1')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-8e270772-8c27-4911-8d96-27fdfaf1e4a1 button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-8e270772-8c27-4911-8d96-27fdfaf1e4a1');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "<div id=\"df-b3c61081-60b8-4f72-bace-11c58c51131c\">\n",
+              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b3c61081-60b8-4f72-bace-11c58c51131c')\"\n",
+              "            title=\"Suggest charts\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "     width=\"24px\">\n",
+              "    <g>\n",
+              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+              "    </g>\n",
+              "</svg>\n",
+              "  </button>\n",
+              "\n",
+              "<style>\n",
+              "  .colab-df-quickchart {\n",
+              "      --bg-color: #E8F0FE;\n",
+              "      --fill-color: #1967D2;\n",
+              "      --hover-bg-color: #E2EBFA;\n",
+              "      --hover-fill-color: #174EA6;\n",
+              "      --disabled-fill-color: #AAA;\n",
+              "      --disabled-bg-color: #DDD;\n",
+              "  }\n",
+              "\n",
+              "  [theme=dark] .colab-df-quickchart {\n",
+              "      --bg-color: #3B4455;\n",
+              "      --fill-color: #D2E3FC;\n",
+              "      --hover-bg-color: #434B5C;\n",
+              "      --hover-fill-color: #FFFFFF;\n",
+              "      --disabled-bg-color: #3B4455;\n",
+              "      --disabled-fill-color: #666;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart {\n",
+              "    background-color: var(--bg-color);\n",
+              "    border: none;\n",
+              "    border-radius: 50%;\n",
+              "    cursor: pointer;\n",
+              "    display: none;\n",
+              "    fill: var(--fill-color);\n",
+              "    height: 32px;\n",
+              "    padding: 0;\n",
+              "    width: 32px;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart:hover {\n",
+              "    background-color: var(--hover-bg-color);\n",
+              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "    fill: var(--button-hover-fill-color);\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart-complete:disabled,\n",
+              "  .colab-df-quickchart-complete:disabled:hover {\n",
+              "    background-color: var(--disabled-bg-color);\n",
+              "    fill: var(--disabled-fill-color);\n",
+              "    box-shadow: none;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-spinner {\n",
+              "    border: 2px solid var(--fill-color);\n",
+              "    border-color: transparent;\n",
+              "    border-bottom-color: var(--fill-color);\n",
+              "    animation:\n",
+              "      spin 1s steps(1) infinite;\n",
+              "  }\n",
+              "\n",
+              "  @keyframes spin {\n",
+              "    0% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "      border-left-color: var(--fill-color);\n",
+              "    }\n",
+              "    20% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    30% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    40% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    60% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    80% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "    90% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "  }\n",
+              "</style>\n",
+              "\n",
+              "  <script>\n",
+              "    async function quickchart(key) {\n",
+              "      const quickchartButtonEl =\n",
+              "        document.querySelector('#' + key + ' button');\n",
+              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+              "      try {\n",
+              "        const charts = await google.colab.kernel.invokeFunction(\n",
+              "            'suggestCharts', [key], {});\n",
+              "      } catch (error) {\n",
+              "        console.error('Error during call to suggestCharts:', error);\n",
+              "      }\n",
+              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+              "    }\n",
+              "    (() => {\n",
+              "      let quickchartButtonEl =\n",
+              "        document.querySelector('#df-b3c61081-60b8-4f72-bace-11c58c51131c button');\n",
+              "      quickchartButtonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "    })();\n",
+              "  </script>\n",
+              "</div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "summary": "{\n  \"name\": \"c\",\n  \"rows\": 2,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"nsclc_radiogenomics\",\n          \"hcc_tace_seg\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"table_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"bigquery-public-data.idc_v20_clinical.nsclc_radiogenomics_clinical\",\n          \"bigquery-public-data.idc_v20_clinical.hcc_tace_seg_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"chemotherapy\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"Chemotherapy\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 32
+        },
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/data_table.py:200: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  return self._dataframe._repr_html_()  # pylint: disable=protected-access\n"
+          ]
+        }
       ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-ee45441e-e416-4bdb-bd04-0dcd62935a39\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>collection_id</th>\n",
-       "      <th>table_name</th>\n",
-       "      <th>column</th>\n",
-       "      <th>column_label</th>\n",
-       "      <th>values</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>5570</th>\n",
-       "      <td>hcc_tace_seg</td>\n",
-       "      <td>bigquery-public-data.idc_v20_clinical.hcc_tace...</td>\n",
-       "      <td>chemotherapy</td>\n",
-       "      <td>chemotherapy used for TACE procedure</td>\n",
-       "      <td>[{'option_code': 'Cisplastin', 'option_descrip...</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ee45441e-e416-4bdb-bd04-0dcd62935a39')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-ee45441e-e416-4bdb-bd04-0dcd62935a39 button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-ee45441e-e416-4bdb-bd04-0dcd62935a39');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "  <div id=\"id_2844e6e8-4263-47eb-951f-4393f532e115\">\n",
-       "    <style>\n",
-       "      .colab-df-generate {\n",
-       "        background-color: #E8F0FE;\n",
-       "        border: none;\n",
-       "        border-radius: 50%;\n",
-       "        cursor: pointer;\n",
-       "        display: none;\n",
-       "        fill: #1967D2;\n",
-       "        height: 32px;\n",
-       "        padding: 0 0 0 0;\n",
-       "        width: 32px;\n",
-       "      }\n",
-       "\n",
-       "      .colab-df-generate:hover {\n",
-       "        background-color: #E2EBFA;\n",
-       "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "        fill: #174EA6;\n",
-       "      }\n",
-       "\n",
-       "      [theme=dark] .colab-df-generate {\n",
-       "        background-color: #3B4455;\n",
-       "        fill: #D2E3FC;\n",
-       "      }\n",
-       "\n",
-       "      [theme=dark] .colab-df-generate:hover {\n",
-       "        background-color: #434B5C;\n",
-       "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "        fill: #FFFFFF;\n",
-       "      }\n",
-       "    </style>\n",
-       "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('chemotherapy_subset')\"\n",
-       "            title=\"Generate code using this dataframe.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "       width=\"24px\">\n",
-       "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "    <script>\n",
-       "      (() => {\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#id_2844e6e8-4263-47eb-951f-4393f532e115 button.colab-df-generate');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      buttonEl.onclick = () => {\n",
-       "        google.colab.notebook.generateWithVariable('chemotherapy_subset');\n",
-       "      }\n",
-       "      })();\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
-      ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "variable_name": "chemotherapy_subset",
-       "summary": "{\n  \"name\": \"chemotherapy_subset\",\n  \"rows\": 1,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"hcc_tace_seg\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"table_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"bigquery-public-data.idc_v20_clinical.hcc_tace_seg_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"chemotherapy\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"chemotherapy used for TACE procedure \"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
-      }
-     },
-     "metadata": {},
-     "execution_count": 33
-    }
-   ],
-   "source": [
-    "chemotherapy_subset = c.clinical_index[c.clinical_index[\"column_label\"].str.contains(\"[Cc]hemotherapy\", na=False)][[ \"collection_id\", \"table_name\", \"column\", \"column_label\",\"values\"]]\n",
-    "\n",
-    "chemotherapy_subset = chemotherapy_subset[chemotherapy_subset[\"collection_id\"] == \"hcc_tace_seg\"]\n",
-    "\n",
-    "chemotherapy_subset"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "NGu6dVggelsQ"
-   },
-   "source": [
-    "Before we select subjects that meet the criteria defined above, let's confirm  the values encountered in table `hcc_tace_seg_clinical` column `chemotherapy` match the value set in the `column_metadata` table. Here are the values of the dictionary we observe from the `column_metadata` inventory."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 34,
-   "metadata": {
-    "id": "pcJeHJrifjIG",
-    "outputId": "e4916bb9-80f1-4d56-9bc2-c273e393752a",
-    "colab": {
-     "base_uri": "https://localhost:8080/"
-    }
-   },
-   "outputs": [
+      "source": [
+        "c.clinical_index[c.clinical_index[\"column_label\"].str.contains(\"[Cc]hemotherapy\", na=False)][[ \"collection_id\", \"table_name\", \"column\", \"column_label\",\"values\"]]"
+      ]
+    },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "[array([{'option_code': 'Cisplastin', 'option_description': None},\n",
-       "        {'option_code': 'Cisplatin, Mitomycin-C', 'option_description': None},\n",
-       "        {'option_code': 'Cisplatin, doxorubicin, Mitomycin-C', 'option_description': None},\n",
-       "        {'option_code': 'NA', 'option_description': None},\n",
-       "        {'option_code': 'doxorubicin LC beads', 'option_description': None}],\n",
-       "       dtype=object)]"
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "kke1DZgwGHuA"
+      },
+      "source": [
+        "From the table above we can observe that there are several collections that have clinical metadata related to chemotherapy regimen of the subject.\n",
+        "\n",
+        "Looking at the value sets for the collections/columns that have those, we can observe that subjects that had any chemotherapy could be selected as follows for the respective collections (non-exhaustive list):\n",
+        "* `nsclc_radiogenomics`: subjects that have value `Yes` in table `nsclc_radiogenomics_clinical` column `chemotherapy`\n",
+        "* `hcc_tace_seg` column `chemotherapy` in the `hcc_tace_seg_clinical` table\n",
+        "\n",
+        "Let's focus on the clinical data related to chemotherapy for the collection `hcc_tace_seg`."
       ]
-     },
-     "metadata": {},
-     "execution_count": 34
-    }
-   ],
-   "source": [
-    "chemotherapy_subset[\"values\"].tolist()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "GEKd7J4Dfa46"
-   },
-   "source": [
-    "The query below will select the distinct values encountered in the `chemotherapy` column of the `hcc_tace_seg_clinical` table, which we confirm matches those in the dictionary."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 35,
-   "metadata": {
-    "id": "sUdb9uOoJg7p",
-    "outputId": "b6e54a83-9803-40b0-922b-24b79e0247d4",
-    "colab": {
-     "base_uri": "https://localhost:8080/"
-    }
-   },
-   "outputs": [
+    },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "array(['Cisplatin, doxorubicin, Mitomycin-C', 'doxorubicin LC beads',\n",
-       "       'NA', 'Cisplastin', 'Cisplatin, Mitomycin-C'], dtype=object)"
+      "cell_type": "code",
+      "execution_count": 33,
+      "metadata": {
+        "id": "EhrEq84tezyL",
+        "outputId": "b1982490-a95f-458b-a8a1-a47aec802ed4",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 171
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "     collection_id                                         table_name  \\\n",
+              "5570  hcc_tace_seg  bigquery-public-data.idc_v20_clinical.hcc_tace...   \n",
+              "\n",
+              "            column                           column_label  \\\n",
+              "5570  chemotherapy  chemotherapy used for TACE procedure    \n",
+              "\n",
+              "                                                 values  \n",
+              "5570  [{'option_code': 'Cisplastin', 'option_descrip...  "
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-ee45441e-e416-4bdb-bd04-0dcd62935a39\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>collection_id</th>\n",
+              "      <th>table_name</th>\n",
+              "      <th>column</th>\n",
+              "      <th>column_label</th>\n",
+              "      <th>values</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>5570</th>\n",
+              "      <td>hcc_tace_seg</td>\n",
+              "      <td>bigquery-public-data.idc_v20_clinical.hcc_tace...</td>\n",
+              "      <td>chemotherapy</td>\n",
+              "      <td>chemotherapy used for TACE procedure</td>\n",
+              "      <td>[{'option_code': 'Cisplastin', 'option_descrip...</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ee45441e-e416-4bdb-bd04-0dcd62935a39')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-ee45441e-e416-4bdb-bd04-0dcd62935a39 button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-ee45441e-e416-4bdb-bd04-0dcd62935a39');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "  <div id=\"id_2844e6e8-4263-47eb-951f-4393f532e115\">\n",
+              "    <style>\n",
+              "      .colab-df-generate {\n",
+              "        background-color: #E8F0FE;\n",
+              "        border: none;\n",
+              "        border-radius: 50%;\n",
+              "        cursor: pointer;\n",
+              "        display: none;\n",
+              "        fill: #1967D2;\n",
+              "        height: 32px;\n",
+              "        padding: 0 0 0 0;\n",
+              "        width: 32px;\n",
+              "      }\n",
+              "\n",
+              "      .colab-df-generate:hover {\n",
+              "        background-color: #E2EBFA;\n",
+              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "        fill: #174EA6;\n",
+              "      }\n",
+              "\n",
+              "      [theme=dark] .colab-df-generate {\n",
+              "        background-color: #3B4455;\n",
+              "        fill: #D2E3FC;\n",
+              "      }\n",
+              "\n",
+              "      [theme=dark] .colab-df-generate:hover {\n",
+              "        background-color: #434B5C;\n",
+              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "        fill: #FFFFFF;\n",
+              "      }\n",
+              "    </style>\n",
+              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('chemotherapy_subset')\"\n",
+              "            title=\"Generate code using this dataframe.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "       width=\"24px\">\n",
+              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "    <script>\n",
+              "      (() => {\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#id_2844e6e8-4263-47eb-951f-4393f532e115 button.colab-df-generate');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      buttonEl.onclick = () => {\n",
+              "        google.colab.notebook.generateWithVariable('chemotherapy_subset');\n",
+              "      }\n",
+              "      })();\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "variable_name": "chemotherapy_subset",
+              "summary": "{\n  \"name\": \"chemotherapy_subset\",\n  \"rows\": 1,\n  \"fields\": [\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"hcc_tace_seg\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"table_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"bigquery-public-data.idc_v20_clinical.hcc_tace_seg_clinical\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"chemotherapy\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"column_label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"chemotherapy used for TACE procedure \"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"values\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 33
+        }
+      ],
+      "source": [
+        "chemotherapy_subset = c.clinical_index[c.clinical_index[\"column_label\"].str.contains(\"[Cc]hemotherapy\", na=False)][[ \"collection_id\", \"table_name\", \"column\", \"column_label\",\"values\"]]\n",
+        "\n",
+        "chemotherapy_subset = chemotherapy_subset[chemotherapy_subset[\"collection_id\"] == \"hcc_tace_seg\"]\n",
+        "\n",
+        "chemotherapy_subset"
       ]
-     },
-     "metadata": {},
-     "execution_count": 35
-    }
-   ],
-   "source": [
-    "hcc_tace_seg_clinical_df = c.get_clinical_table(\"hcc_tace_seg_clinical\")\n",
-    "\n",
-    "hcc_tace_seg_clinical_df[\"chemotherapy\"].unique()\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "QQRPfWs7GsEg"
-   },
-   "source": [
-    "Next we can examine the data to see what therapy individual patients had."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "source": [
-    "# prompt: select distinct combinations of values in dicom_patient_id and chemotherapy columns from the hcc_tace_seg_clinical_df pandas dataframe where chemotherapy column is defined\n",
-    "\n",
-    "distinct_combinations = hcc_tace_seg_clinical_df[hcc_tace_seg_clinical_df['chemotherapy'].notna()][['dicom_patient_id', 'chemotherapy']].drop_duplicates()\n",
-    "distinct_combinations"
-   ],
-   "metadata": {
-    "id": "mx7mxP5xUEYX",
-    "outputId": "9f166baf-ea82-42c0-86dc-5c81b0a6c285",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 479
-    }
-   },
-   "execution_count": 36,
-   "outputs": [
+    },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "NGu6dVggelsQ"
+      },
+      "source": [
+        "Before we select subjects that meet the criteria defined above, let's confirm  the values encountered in table `hcc_tace_seg_clinical` column `chemotherapy` match the value set in the `column_metadata` table. Here are the values of the dictionary we observe from the `column_metadata` inventory."
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "    dicom_patient_id                         chemotherapy\n",
-       "0            HCC_024  Cisplatin, doxorubicin, Mitomycin-C\n",
-       "1            HCC_045  Cisplatin, doxorubicin, Mitomycin-C\n",
-       "2            HCC_050  Cisplatin, doxorubicin, Mitomycin-C\n",
-       "3            HCC_065  Cisplatin, doxorubicin, Mitomycin-C\n",
-       "4            HCC_073                 doxorubicin LC beads\n",
-       "..               ...                                  ...\n",
-       "100          HCC_056  Cisplatin, doxorubicin, Mitomycin-C\n",
-       "101          HCC_070                 doxorubicin LC beads\n",
-       "102          HCC_061                                   NA\n",
-       "103          HCC_075                 doxorubicin LC beads\n",
-       "104          HCC_099                 doxorubicin LC beads\n",
-       "\n",
-       "[105 rows x 2 columns]"
+      "cell_type": "code",
+      "execution_count": 34,
+      "metadata": {
+        "id": "pcJeHJrifjIG",
+        "outputId": "e4916bb9-80f1-4d56-9bc2-c273e393752a",
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "[array([{'option_code': 'Cisplastin', 'option_description': None},\n",
+              "        {'option_code': 'Cisplatin, Mitomycin-C', 'option_description': None},\n",
+              "        {'option_code': 'Cisplatin, doxorubicin, Mitomycin-C', 'option_description': None},\n",
+              "        {'option_code': 'NA', 'option_description': None},\n",
+              "        {'option_code': 'doxorubicin LC beads', 'option_description': None}],\n",
+              "       dtype=object)]"
+            ]
+          },
+          "metadata": {},
+          "execution_count": 34
+        }
       ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-69c6b2b0-8564-4cb0-8eb8-ab5fc43cee9f\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>dicom_patient_id</th>\n",
-       "      <th>chemotherapy</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>0</th>\n",
-       "      <td>HCC_024</td>\n",
-       "      <td>Cisplatin, doxorubicin, Mitomycin-C</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1</th>\n",
-       "      <td>HCC_045</td>\n",
-       "      <td>Cisplatin, doxorubicin, Mitomycin-C</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2</th>\n",
-       "      <td>HCC_050</td>\n",
-       "      <td>Cisplatin, doxorubicin, Mitomycin-C</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3</th>\n",
-       "      <td>HCC_065</td>\n",
-       "      <td>Cisplatin, doxorubicin, Mitomycin-C</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>4</th>\n",
-       "      <td>HCC_073</td>\n",
-       "      <td>doxorubicin LC beads</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>...</th>\n",
-       "      <td>...</td>\n",
-       "      <td>...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>100</th>\n",
-       "      <td>HCC_056</td>\n",
-       "      <td>Cisplatin, doxorubicin, Mitomycin-C</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>101</th>\n",
-       "      <td>HCC_070</td>\n",
-       "      <td>doxorubicin LC beads</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>102</th>\n",
-       "      <td>HCC_061</td>\n",
-       "      <td>NA</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>103</th>\n",
-       "      <td>HCC_075</td>\n",
-       "      <td>doxorubicin LC beads</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>104</th>\n",
-       "      <td>HCC_099</td>\n",
-       "      <td>doxorubicin LC beads</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "<p>105 rows × 2 columns</p>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-69c6b2b0-8564-4cb0-8eb8-ab5fc43cee9f')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-69c6b2b0-8564-4cb0-8eb8-ab5fc43cee9f button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-69c6b2b0-8564-4cb0-8eb8-ab5fc43cee9f');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "<div id=\"df-38204da8-62fc-448b-b524-e96a614da87b\">\n",
-       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-38204da8-62fc-448b-b524-e96a614da87b')\"\n",
-       "            title=\"Suggest charts\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "     width=\"24px\">\n",
-       "    <g>\n",
-       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
-       "    </g>\n",
-       "</svg>\n",
-       "  </button>\n",
-       "\n",
-       "<style>\n",
-       "  .colab-df-quickchart {\n",
-       "      --bg-color: #E8F0FE;\n",
-       "      --fill-color: #1967D2;\n",
-       "      --hover-bg-color: #E2EBFA;\n",
-       "      --hover-fill-color: #174EA6;\n",
-       "      --disabled-fill-color: #AAA;\n",
-       "      --disabled-bg-color: #DDD;\n",
-       "  }\n",
-       "\n",
-       "  [theme=dark] .colab-df-quickchart {\n",
-       "      --bg-color: #3B4455;\n",
-       "      --fill-color: #D2E3FC;\n",
-       "      --hover-bg-color: #434B5C;\n",
-       "      --hover-fill-color: #FFFFFF;\n",
-       "      --disabled-bg-color: #3B4455;\n",
-       "      --disabled-fill-color: #666;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart {\n",
-       "    background-color: var(--bg-color);\n",
-       "    border: none;\n",
-       "    border-radius: 50%;\n",
-       "    cursor: pointer;\n",
-       "    display: none;\n",
-       "    fill: var(--fill-color);\n",
-       "    height: 32px;\n",
-       "    padding: 0;\n",
-       "    width: 32px;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart:hover {\n",
-       "    background-color: var(--hover-bg-color);\n",
-       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "    fill: var(--button-hover-fill-color);\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart-complete:disabled,\n",
-       "  .colab-df-quickchart-complete:disabled:hover {\n",
-       "    background-color: var(--disabled-bg-color);\n",
-       "    fill: var(--disabled-fill-color);\n",
-       "    box-shadow: none;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-spinner {\n",
-       "    border: 2px solid var(--fill-color);\n",
-       "    border-color: transparent;\n",
-       "    border-bottom-color: var(--fill-color);\n",
-       "    animation:\n",
-       "      spin 1s steps(1) infinite;\n",
-       "  }\n",
-       "\n",
-       "  @keyframes spin {\n",
-       "    0% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "      border-left-color: var(--fill-color);\n",
-       "    }\n",
-       "    20% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    30% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    40% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    60% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    80% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "    90% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "  }\n",
-       "</style>\n",
-       "\n",
-       "  <script>\n",
-       "    async function quickchart(key) {\n",
-       "      const quickchartButtonEl =\n",
-       "        document.querySelector('#' + key + ' button');\n",
-       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
-       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
-       "      try {\n",
-       "        const charts = await google.colab.kernel.invokeFunction(\n",
-       "            'suggestCharts', [key], {});\n",
-       "      } catch (error) {\n",
-       "        console.error('Error during call to suggestCharts:', error);\n",
-       "      }\n",
-       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
-       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
-       "    }\n",
-       "    (() => {\n",
-       "      let quickchartButtonEl =\n",
-       "        document.querySelector('#df-38204da8-62fc-448b-b524-e96a614da87b button');\n",
-       "      quickchartButtonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "    })();\n",
-       "  </script>\n",
-       "</div>\n",
-       "\n",
-       "  <div id=\"id_3cbb1c7c-cdaa-4912-82d9-e4277172522a\">\n",
-       "    <style>\n",
-       "      .colab-df-generate {\n",
-       "        background-color: #E8F0FE;\n",
-       "        border: none;\n",
-       "        border-radius: 50%;\n",
-       "        cursor: pointer;\n",
-       "        display: none;\n",
-       "        fill: #1967D2;\n",
-       "        height: 32px;\n",
-       "        padding: 0 0 0 0;\n",
-       "        width: 32px;\n",
-       "      }\n",
-       "\n",
-       "      .colab-df-generate:hover {\n",
-       "        background-color: #E2EBFA;\n",
-       "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "        fill: #174EA6;\n",
-       "      }\n",
-       "\n",
-       "      [theme=dark] .colab-df-generate {\n",
-       "        background-color: #3B4455;\n",
-       "        fill: #D2E3FC;\n",
-       "      }\n",
-       "\n",
-       "      [theme=dark] .colab-df-generate:hover {\n",
-       "        background-color: #434B5C;\n",
-       "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "        fill: #FFFFFF;\n",
-       "      }\n",
-       "    </style>\n",
-       "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('distinct_combinations')\"\n",
-       "            title=\"Generate code using this dataframe.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "       width=\"24px\">\n",
-       "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "    <script>\n",
-       "      (() => {\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#id_3cbb1c7c-cdaa-4912-82d9-e4277172522a button.colab-df-generate');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      buttonEl.onclick = () => {\n",
-       "        google.colab.notebook.generateWithVariable('distinct_combinations');\n",
-       "      }\n",
-       "      })();\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
+      "source": [
+        "chemotherapy_subset[\"values\"].tolist()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "GEKd7J4Dfa46"
+      },
+      "source": [
+        "The query below will select the distinct values encountered in the `chemotherapy` column of the `hcc_tace_seg_clinical` table, which we confirm matches those in the dictionary."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 35,
+      "metadata": {
+        "id": "sUdb9uOoJg7p",
+        "outputId": "b6e54a83-9803-40b0-922b-24b79e0247d4",
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        }
+      },
+      "outputs": [
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "array(['Cisplatin, doxorubicin, Mitomycin-C', 'doxorubicin LC beads',\n",
+              "       'NA', 'Cisplastin', 'Cisplatin, Mitomycin-C'], dtype=object)"
+            ]
+          },
+          "metadata": {},
+          "execution_count": 35
+        }
       ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "variable_name": "distinct_combinations",
-       "summary": "{\n  \"name\": \"distinct_combinations\",\n  \"rows\": 105,\n  \"fields\": [\n    {\n      \"column\": \"dicom_patient_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 105,\n        \"samples\": [\n          \"HCC_008\",\n          \"HCC_059\",\n          \"HCC_058\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"chemotherapy\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"doxorubicin LC beads\",\n          \"Cisplatin, Mitomycin-C\",\n          \"NA\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
-      }
-     },
-     "metadata": {},
-     "execution_count": 36
-    }
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "NjdEJe1rHvQC"
-   },
-   "source": [
-    "`dicom_patient_id` is the key to connect clinical data and imaging data. Let's see what imaging studies we have for patient `HCC_101` that was treated with \tdoxorubicin LC beads. Along the way we can also generate viewer URLs to conveniently examine the images."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "source": [
-    "query = \"\"\"\n",
-    "SELECT\n",
-    "  StudyInstanceUID,\n",
-    "  STRING_AGG(DISTINCT(Modality)) AS modalities,\n",
-    "  STRING_AGG(DISTINCT(collection_id)) AS collection_id,\n",
-    "  COUNT(DISTINCT(SeriesInstanceUID)) AS num_series,\n",
-    "  ANY_VALUE(CONCAT('https://viewer.imaging.datacommons.cancer.gov/viewer/', StudyInstanceUID)) as viewer_url\n",
-    "FROM index\n",
-    "WHERE PatientID = 'HCC_103'\n",
-    "GROUP BY StudyInstanceUID\n",
-    "\"\"\"\n",
-    "\n",
-    "c.sql_query(query)"
-   ],
-   "metadata": {
-    "id": "RqjVoU_7VP65",
-    "outputId": "d0bf6313-26fc-4db4-ed1d-39cbaf6962e5",
-    "colab": {
-     "base_uri": "https://localhost:8080/",
-     "height": 204
-    }
-   },
-   "execution_count": 40,
-   "outputs": [
+      "source": [
+        "hcc_tace_seg_clinical_df = c.get_clinical_table(\"hcc_tace_seg_clinical\")\n",
+        "\n",
+        "hcc_tace_seg_clinical_df[\"chemotherapy\"].unique()\n"
+      ]
+    },
     {
-     "output_type": "stream",
-     "name": "stderr",
-     "text": [
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n",
-      "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: RangeIndex.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
-      "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
-     ]
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "QQRPfWs7GsEg"
+      },
+      "source": [
+        "Next we can examine the data to see what therapy individual patients had."
+      ]
     },
     {
-     "output_type": "execute_result",
-     "data": {
-      "text/plain": [
-       "                                    StudyInstanceUID modalities collection_id  \\\n",
-       "0  1.3.6.1.4.1.14519.5.2.1.1706.8374.304819071818...     CT,SEG  hcc_tace_seg   \n",
-       "1  1.3.6.1.4.1.14519.5.2.1.1706.8374.121752675166...         CT  hcc_tace_seg   \n",
-       "\n",
-       "   num_series                                         viewer_url  \n",
-       "0           3  https://viewer.imaging.datacommons.cancer.gov/...  \n",
-       "1           2  https://viewer.imaging.datacommons.cancer.gov/...  "
+      "cell_type": "code",
+      "source": [
+        "# prompt: select distinct combinations of values in dicom_patient_id and chemotherapy columns from the hcc_tace_seg_clinical_df pandas dataframe where chemotherapy column is defined\n",
+        "\n",
+        "distinct_combinations = hcc_tace_seg_clinical_df[hcc_tace_seg_clinical_df['chemotherapy'].notna()][['dicom_patient_id', 'chemotherapy']].drop_duplicates()\n",
+        "distinct_combinations"
       ],
-      "text/html": [
-       "\n",
-       "  <div id=\"df-9b24b369-80cd-483d-adba-89e4a3320bd3\" class=\"colab-df-container\">\n",
-       "    <div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>StudyInstanceUID</th>\n",
-       "      <th>modalities</th>\n",
-       "      <th>collection_id</th>\n",
-       "      <th>num_series</th>\n",
-       "      <th>viewer_url</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>0</th>\n",
-       "      <td>1.3.6.1.4.1.14519.5.2.1.1706.8374.304819071818...</td>\n",
-       "      <td>CT,SEG</td>\n",
-       "      <td>hcc_tace_seg</td>\n",
-       "      <td>3</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1</th>\n",
-       "      <td>1.3.6.1.4.1.14519.5.2.1.1706.8374.121752675166...</td>\n",
-       "      <td>CT</td>\n",
-       "      <td>hcc_tace_seg</td>\n",
-       "      <td>2</td>\n",
-       "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "</div>\n",
-       "    <div class=\"colab-df-buttons\">\n",
-       "\n",
-       "  <div class=\"colab-df-container\">\n",
-       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9b24b369-80cd-483d-adba-89e4a3320bd3')\"\n",
-       "            title=\"Convert this dataframe to an interactive table.\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
-       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
-       "  </svg>\n",
-       "    </button>\n",
-       "\n",
-       "  <style>\n",
-       "    .colab-df-container {\n",
-       "      display:flex;\n",
-       "      gap: 12px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert {\n",
-       "      background-color: #E8F0FE;\n",
-       "      border: none;\n",
-       "      border-radius: 50%;\n",
-       "      cursor: pointer;\n",
-       "      display: none;\n",
-       "      fill: #1967D2;\n",
-       "      height: 32px;\n",
-       "      padding: 0 0 0 0;\n",
-       "      width: 32px;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-convert:hover {\n",
-       "      background-color: #E2EBFA;\n",
-       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "      fill: #174EA6;\n",
-       "    }\n",
-       "\n",
-       "    .colab-df-buttons div {\n",
-       "      margin-bottom: 4px;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert {\n",
-       "      background-color: #3B4455;\n",
-       "      fill: #D2E3FC;\n",
-       "    }\n",
-       "\n",
-       "    [theme=dark] .colab-df-convert:hover {\n",
-       "      background-color: #434B5C;\n",
-       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-       "      fill: #FFFFFF;\n",
-       "    }\n",
-       "  </style>\n",
-       "\n",
-       "    <script>\n",
-       "      const buttonEl =\n",
-       "        document.querySelector('#df-9b24b369-80cd-483d-adba-89e4a3320bd3 button.colab-df-convert');\n",
-       "      buttonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "\n",
-       "      async function convertToInteractive(key) {\n",
-       "        const element = document.querySelector('#df-9b24b369-80cd-483d-adba-89e4a3320bd3');\n",
-       "        const dataTable =\n",
-       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-       "                                                    [key], {});\n",
-       "        if (!dataTable) return;\n",
-       "\n",
-       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
-       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-       "          + ' to learn more about interactive tables.';\n",
-       "        element.innerHTML = '';\n",
-       "        dataTable['output_type'] = 'display_data';\n",
-       "        await google.colab.output.renderOutput(dataTable, element);\n",
-       "        const docLink = document.createElement('div');\n",
-       "        docLink.innerHTML = docLinkHtml;\n",
-       "        element.appendChild(docLink);\n",
-       "      }\n",
-       "    </script>\n",
-       "  </div>\n",
-       "\n",
-       "\n",
-       "<div id=\"df-4a98ca71-ae88-4fd9-afff-7c64e9de394b\">\n",
-       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4a98ca71-ae88-4fd9-afff-7c64e9de394b')\"\n",
-       "            title=\"Suggest charts\"\n",
-       "            style=\"display:none;\">\n",
-       "\n",
-       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-       "     width=\"24px\">\n",
-       "    <g>\n",
-       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
-       "    </g>\n",
-       "</svg>\n",
-       "  </button>\n",
-       "\n",
-       "<style>\n",
-       "  .colab-df-quickchart {\n",
-       "      --bg-color: #E8F0FE;\n",
-       "      --fill-color: #1967D2;\n",
-       "      --hover-bg-color: #E2EBFA;\n",
-       "      --hover-fill-color: #174EA6;\n",
-       "      --disabled-fill-color: #AAA;\n",
-       "      --disabled-bg-color: #DDD;\n",
-       "  }\n",
-       "\n",
-       "  [theme=dark] .colab-df-quickchart {\n",
-       "      --bg-color: #3B4455;\n",
-       "      --fill-color: #D2E3FC;\n",
-       "      --hover-bg-color: #434B5C;\n",
-       "      --hover-fill-color: #FFFFFF;\n",
-       "      --disabled-bg-color: #3B4455;\n",
-       "      --disabled-fill-color: #666;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart {\n",
-       "    background-color: var(--bg-color);\n",
-       "    border: none;\n",
-       "    border-radius: 50%;\n",
-       "    cursor: pointer;\n",
-       "    display: none;\n",
-       "    fill: var(--fill-color);\n",
-       "    height: 32px;\n",
-       "    padding: 0;\n",
-       "    width: 32px;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart:hover {\n",
-       "    background-color: var(--hover-bg-color);\n",
-       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-       "    fill: var(--button-hover-fill-color);\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-quickchart-complete:disabled,\n",
-       "  .colab-df-quickchart-complete:disabled:hover {\n",
-       "    background-color: var(--disabled-bg-color);\n",
-       "    fill: var(--disabled-fill-color);\n",
-       "    box-shadow: none;\n",
-       "  }\n",
-       "\n",
-       "  .colab-df-spinner {\n",
-       "    border: 2px solid var(--fill-color);\n",
-       "    border-color: transparent;\n",
-       "    border-bottom-color: var(--fill-color);\n",
-       "    animation:\n",
-       "      spin 1s steps(1) infinite;\n",
-       "  }\n",
-       "\n",
-       "  @keyframes spin {\n",
-       "    0% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "      border-left-color: var(--fill-color);\n",
-       "    }\n",
-       "    20% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    30% {\n",
-       "      border-color: transparent;\n",
-       "      border-left-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    40% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-top-color: var(--fill-color);\n",
-       "    }\n",
-       "    60% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "    }\n",
-       "    80% {\n",
-       "      border-color: transparent;\n",
-       "      border-right-color: var(--fill-color);\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "    90% {\n",
-       "      border-color: transparent;\n",
-       "      border-bottom-color: var(--fill-color);\n",
-       "    }\n",
-       "  }\n",
-       "</style>\n",
-       "\n",
-       "  <script>\n",
-       "    async function quickchart(key) {\n",
-       "      const quickchartButtonEl =\n",
-       "        document.querySelector('#' + key + ' button');\n",
-       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
-       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
-       "      try {\n",
-       "        const charts = await google.colab.kernel.invokeFunction(\n",
-       "            'suggestCharts', [key], {});\n",
-       "      } catch (error) {\n",
-       "        console.error('Error during call to suggestCharts:', error);\n",
-       "      }\n",
-       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
-       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
-       "    }\n",
-       "    (() => {\n",
-       "      let quickchartButtonEl =\n",
-       "        document.querySelector('#df-4a98ca71-ae88-4fd9-afff-7c64e9de394b button');\n",
-       "      quickchartButtonEl.style.display =\n",
-       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-       "    })();\n",
-       "  </script>\n",
-       "</div>\n",
-       "\n",
-       "    </div>\n",
-       "  </div>\n"
+      "metadata": {
+        "id": "mx7mxP5xUEYX",
+        "outputId": "9f166baf-ea82-42c0-86dc-5c81b0a6c285",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 479
+        }
+      },
+      "execution_count": 36,
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "    dicom_patient_id                         chemotherapy\n",
+              "0            HCC_024  Cisplatin, doxorubicin, Mitomycin-C\n",
+              "1            HCC_045  Cisplatin, doxorubicin, Mitomycin-C\n",
+              "2            HCC_050  Cisplatin, doxorubicin, Mitomycin-C\n",
+              "3            HCC_065  Cisplatin, doxorubicin, Mitomycin-C\n",
+              "4            HCC_073                 doxorubicin LC beads\n",
+              "..               ...                                  ...\n",
+              "100          HCC_056  Cisplatin, doxorubicin, Mitomycin-C\n",
+              "101          HCC_070                 doxorubicin LC beads\n",
+              "102          HCC_061                                   NA\n",
+              "103          HCC_075                 doxorubicin LC beads\n",
+              "104          HCC_099                 doxorubicin LC beads\n",
+              "\n",
+              "[105 rows x 2 columns]"
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-69c6b2b0-8564-4cb0-8eb8-ab5fc43cee9f\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>dicom_patient_id</th>\n",
+              "      <th>chemotherapy</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>0</th>\n",
+              "      <td>HCC_024</td>\n",
+              "      <td>Cisplatin, doxorubicin, Mitomycin-C</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1</th>\n",
+              "      <td>HCC_045</td>\n",
+              "      <td>Cisplatin, doxorubicin, Mitomycin-C</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>2</th>\n",
+              "      <td>HCC_050</td>\n",
+              "      <td>Cisplatin, doxorubicin, Mitomycin-C</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>3</th>\n",
+              "      <td>HCC_065</td>\n",
+              "      <td>Cisplatin, doxorubicin, Mitomycin-C</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>4</th>\n",
+              "      <td>HCC_073</td>\n",
+              "      <td>doxorubicin LC beads</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>...</th>\n",
+              "      <td>...</td>\n",
+              "      <td>...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>100</th>\n",
+              "      <td>HCC_056</td>\n",
+              "      <td>Cisplatin, doxorubicin, Mitomycin-C</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>101</th>\n",
+              "      <td>HCC_070</td>\n",
+              "      <td>doxorubicin LC beads</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>102</th>\n",
+              "      <td>HCC_061</td>\n",
+              "      <td>NA</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>103</th>\n",
+              "      <td>HCC_075</td>\n",
+              "      <td>doxorubicin LC beads</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>104</th>\n",
+              "      <td>HCC_099</td>\n",
+              "      <td>doxorubicin LC beads</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "<p>105 rows × 2 columns</p>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-69c6b2b0-8564-4cb0-8eb8-ab5fc43cee9f')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-69c6b2b0-8564-4cb0-8eb8-ab5fc43cee9f button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-69c6b2b0-8564-4cb0-8eb8-ab5fc43cee9f');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "<div id=\"df-38204da8-62fc-448b-b524-e96a614da87b\">\n",
+              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-38204da8-62fc-448b-b524-e96a614da87b')\"\n",
+              "            title=\"Suggest charts\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "     width=\"24px\">\n",
+              "    <g>\n",
+              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+              "    </g>\n",
+              "</svg>\n",
+              "  </button>\n",
+              "\n",
+              "<style>\n",
+              "  .colab-df-quickchart {\n",
+              "      --bg-color: #E8F0FE;\n",
+              "      --fill-color: #1967D2;\n",
+              "      --hover-bg-color: #E2EBFA;\n",
+              "      --hover-fill-color: #174EA6;\n",
+              "      --disabled-fill-color: #AAA;\n",
+              "      --disabled-bg-color: #DDD;\n",
+              "  }\n",
+              "\n",
+              "  [theme=dark] .colab-df-quickchart {\n",
+              "      --bg-color: #3B4455;\n",
+              "      --fill-color: #D2E3FC;\n",
+              "      --hover-bg-color: #434B5C;\n",
+              "      --hover-fill-color: #FFFFFF;\n",
+              "      --disabled-bg-color: #3B4455;\n",
+              "      --disabled-fill-color: #666;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart {\n",
+              "    background-color: var(--bg-color);\n",
+              "    border: none;\n",
+              "    border-radius: 50%;\n",
+              "    cursor: pointer;\n",
+              "    display: none;\n",
+              "    fill: var(--fill-color);\n",
+              "    height: 32px;\n",
+              "    padding: 0;\n",
+              "    width: 32px;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart:hover {\n",
+              "    background-color: var(--hover-bg-color);\n",
+              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "    fill: var(--button-hover-fill-color);\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart-complete:disabled,\n",
+              "  .colab-df-quickchart-complete:disabled:hover {\n",
+              "    background-color: var(--disabled-bg-color);\n",
+              "    fill: var(--disabled-fill-color);\n",
+              "    box-shadow: none;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-spinner {\n",
+              "    border: 2px solid var(--fill-color);\n",
+              "    border-color: transparent;\n",
+              "    border-bottom-color: var(--fill-color);\n",
+              "    animation:\n",
+              "      spin 1s steps(1) infinite;\n",
+              "  }\n",
+              "\n",
+              "  @keyframes spin {\n",
+              "    0% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "      border-left-color: var(--fill-color);\n",
+              "    }\n",
+              "    20% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    30% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    40% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    60% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    80% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "    90% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "  }\n",
+              "</style>\n",
+              "\n",
+              "  <script>\n",
+              "    async function quickchart(key) {\n",
+              "      const quickchartButtonEl =\n",
+              "        document.querySelector('#' + key + ' button');\n",
+              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+              "      try {\n",
+              "        const charts = await google.colab.kernel.invokeFunction(\n",
+              "            'suggestCharts', [key], {});\n",
+              "      } catch (error) {\n",
+              "        console.error('Error during call to suggestCharts:', error);\n",
+              "      }\n",
+              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+              "    }\n",
+              "    (() => {\n",
+              "      let quickchartButtonEl =\n",
+              "        document.querySelector('#df-38204da8-62fc-448b-b524-e96a614da87b button');\n",
+              "      quickchartButtonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "    })();\n",
+              "  </script>\n",
+              "</div>\n",
+              "\n",
+              "  <div id=\"id_3cbb1c7c-cdaa-4912-82d9-e4277172522a\">\n",
+              "    <style>\n",
+              "      .colab-df-generate {\n",
+              "        background-color: #E8F0FE;\n",
+              "        border: none;\n",
+              "        border-radius: 50%;\n",
+              "        cursor: pointer;\n",
+              "        display: none;\n",
+              "        fill: #1967D2;\n",
+              "        height: 32px;\n",
+              "        padding: 0 0 0 0;\n",
+              "        width: 32px;\n",
+              "      }\n",
+              "\n",
+              "      .colab-df-generate:hover {\n",
+              "        background-color: #E2EBFA;\n",
+              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "        fill: #174EA6;\n",
+              "      }\n",
+              "\n",
+              "      [theme=dark] .colab-df-generate {\n",
+              "        background-color: #3B4455;\n",
+              "        fill: #D2E3FC;\n",
+              "      }\n",
+              "\n",
+              "      [theme=dark] .colab-df-generate:hover {\n",
+              "        background-color: #434B5C;\n",
+              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "        fill: #FFFFFF;\n",
+              "      }\n",
+              "    </style>\n",
+              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('distinct_combinations')\"\n",
+              "            title=\"Generate code using this dataframe.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "       width=\"24px\">\n",
+              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "    <script>\n",
+              "      (() => {\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#id_3cbb1c7c-cdaa-4912-82d9-e4277172522a button.colab-df-generate');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      buttonEl.onclick = () => {\n",
+              "        google.colab.notebook.generateWithVariable('distinct_combinations');\n",
+              "      }\n",
+              "      })();\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "variable_name": "distinct_combinations",
+              "summary": "{\n  \"name\": \"distinct_combinations\",\n  \"rows\": 105,\n  \"fields\": [\n    {\n      \"column\": \"dicom_patient_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 105,\n        \"samples\": [\n          \"HCC_008\",\n          \"HCC_059\",\n          \"HCC_058\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"chemotherapy\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"doxorubicin LC beads\",\n          \"Cisplatin, Mitomycin-C\",\n          \"NA\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 36
+        }
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "NjdEJe1rHvQC"
+      },
+      "source": [
+        "`dicom_patient_id` is the key to connect clinical data and imaging data. Let's see what imaging studies we have for patient `HCC_101` that was treated with \tdoxorubicin LC beads. Along the way we can also generate viewer URLs to conveniently examine the images."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "query = \"\"\"\n",
+        "SELECT\n",
+        "  StudyInstanceUID,\n",
+        "  STRING_AGG(DISTINCT(Modality)) AS modalities,\n",
+        "  STRING_AGG(DISTINCT(collection_id)) AS collection_id,\n",
+        "  COUNT(DISTINCT(SeriesInstanceUID)) AS num_series,\n",
+        "  ANY_VALUE(CONCAT('https://viewer.imaging.datacommons.cancer.gov/viewer/', StudyInstanceUID)) as viewer_url\n",
+        "FROM index\n",
+        "WHERE PatientID = 'HCC_103'\n",
+        "GROUP BY StudyInstanceUID\n",
+        "\"\"\"\n",
+        "\n",
+        "c.sql_query(query)"
       ],
-      "application/vnd.google.colaboratory.intrinsic+json": {
-       "type": "dataframe",
-       "summary": "{\n  \"name\": \"c\",\n  \"rows\": 2,\n  \"fields\": [\n    {\n      \"column\": \"StudyInstanceUID\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"1.3.6.1.4.1.14519.5.2.1.1706.8374.121752675166759110829287848146\",\n          \"1.3.6.1.4.1.14519.5.2.1.1706.8374.304819071818674684593765963305\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"modalities\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"CT\",\n          \"CT,SEG\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"hcc_tace_seg\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"num_series\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 2,\n        \"max\": 3,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"viewer_url\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.1706.8374.121752675166759110829287848146\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
-      }
-     },
-     "metadata": {},
-     "execution_count": 40
+      "metadata": {
+        "id": "RqjVoU_7VP65",
+        "outputId": "d0bf6313-26fc-4db4-ed1d-39cbaf6962e5",
+        "colab": {
+          "base_uri": "https://localhost:8080/",
+          "height": 204
+        }
+      },
+      "execution_count": 40,
+      "outputs": [
+        {
+          "output_type": "stream",
+          "name": "stderr",
+          "text": [
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: Index.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n",
+            "/usr/local/lib/python3.10/dist-packages/google/colab/_interactive_table_hint_button.py:178: FutureWarning: RangeIndex.format is deprecated and will be removed in a future version. Convert using index.astype(str) or index.map(formatter) instead.\n",
+            "  df_html=dataframe._repr_html_(),  # pylint: disable=protected-access\n"
+          ]
+        },
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "                                    StudyInstanceUID modalities collection_id  \\\n",
+              "0  1.3.6.1.4.1.14519.5.2.1.1706.8374.304819071818...     CT,SEG  hcc_tace_seg   \n",
+              "1  1.3.6.1.4.1.14519.5.2.1.1706.8374.121752675166...         CT  hcc_tace_seg   \n",
+              "\n",
+              "   num_series                                         viewer_url  \n",
+              "0           3  https://viewer.imaging.datacommons.cancer.gov/...  \n",
+              "1           2  https://viewer.imaging.datacommons.cancer.gov/...  "
+            ],
+            "text/html": [
+              "\n",
+              "  <div id=\"df-9b24b369-80cd-483d-adba-89e4a3320bd3\" class=\"colab-df-container\">\n",
+              "    <div>\n",
+              "<style scoped>\n",
+              "    .dataframe tbody tr th:only-of-type {\n",
+              "        vertical-align: middle;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe tbody tr th {\n",
+              "        vertical-align: top;\n",
+              "    }\n",
+              "\n",
+              "    .dataframe thead th {\n",
+              "        text-align: right;\n",
+              "    }\n",
+              "</style>\n",
+              "<table border=\"1\" class=\"dataframe\">\n",
+              "  <thead>\n",
+              "    <tr style=\"text-align: right;\">\n",
+              "      <th></th>\n",
+              "      <th>StudyInstanceUID</th>\n",
+              "      <th>modalities</th>\n",
+              "      <th>collection_id</th>\n",
+              "      <th>num_series</th>\n",
+              "      <th>viewer_url</th>\n",
+              "    </tr>\n",
+              "  </thead>\n",
+              "  <tbody>\n",
+              "    <tr>\n",
+              "      <th>0</th>\n",
+              "      <td>1.3.6.1.4.1.14519.5.2.1.1706.8374.304819071818...</td>\n",
+              "      <td>CT,SEG</td>\n",
+              "      <td>hcc_tace_seg</td>\n",
+              "      <td>3</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "    <tr>\n",
+              "      <th>1</th>\n",
+              "      <td>1.3.6.1.4.1.14519.5.2.1.1706.8374.121752675166...</td>\n",
+              "      <td>CT</td>\n",
+              "      <td>hcc_tace_seg</td>\n",
+              "      <td>2</td>\n",
+              "      <td>https://viewer.imaging.datacommons.cancer.gov/...</td>\n",
+              "    </tr>\n",
+              "  </tbody>\n",
+              "</table>\n",
+              "</div>\n",
+              "    <div class=\"colab-df-buttons\">\n",
+              "\n",
+              "  <div class=\"colab-df-container\">\n",
+              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9b24b369-80cd-483d-adba-89e4a3320bd3')\"\n",
+              "            title=\"Convert this dataframe to an interactive table.\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
+              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
+              "  </svg>\n",
+              "    </button>\n",
+              "\n",
+              "  <style>\n",
+              "    .colab-df-container {\n",
+              "      display:flex;\n",
+              "      gap: 12px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert {\n",
+              "      background-color: #E8F0FE;\n",
+              "      border: none;\n",
+              "      border-radius: 50%;\n",
+              "      cursor: pointer;\n",
+              "      display: none;\n",
+              "      fill: #1967D2;\n",
+              "      height: 32px;\n",
+              "      padding: 0 0 0 0;\n",
+              "      width: 32px;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-convert:hover {\n",
+              "      background-color: #E2EBFA;\n",
+              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "      fill: #174EA6;\n",
+              "    }\n",
+              "\n",
+              "    .colab-df-buttons div {\n",
+              "      margin-bottom: 4px;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert {\n",
+              "      background-color: #3B4455;\n",
+              "      fill: #D2E3FC;\n",
+              "    }\n",
+              "\n",
+              "    [theme=dark] .colab-df-convert:hover {\n",
+              "      background-color: #434B5C;\n",
+              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
+              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
+              "      fill: #FFFFFF;\n",
+              "    }\n",
+              "  </style>\n",
+              "\n",
+              "    <script>\n",
+              "      const buttonEl =\n",
+              "        document.querySelector('#df-9b24b369-80cd-483d-adba-89e4a3320bd3 button.colab-df-convert');\n",
+              "      buttonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "\n",
+              "      async function convertToInteractive(key) {\n",
+              "        const element = document.querySelector('#df-9b24b369-80cd-483d-adba-89e4a3320bd3');\n",
+              "        const dataTable =\n",
+              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
+              "                                                    [key], {});\n",
+              "        if (!dataTable) return;\n",
+              "\n",
+              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
+              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
+              "          + ' to learn more about interactive tables.';\n",
+              "        element.innerHTML = '';\n",
+              "        dataTable['output_type'] = 'display_data';\n",
+              "        await google.colab.output.renderOutput(dataTable, element);\n",
+              "        const docLink = document.createElement('div');\n",
+              "        docLink.innerHTML = docLinkHtml;\n",
+              "        element.appendChild(docLink);\n",
+              "      }\n",
+              "    </script>\n",
+              "  </div>\n",
+              "\n",
+              "\n",
+              "<div id=\"df-4a98ca71-ae88-4fd9-afff-7c64e9de394b\">\n",
+              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4a98ca71-ae88-4fd9-afff-7c64e9de394b')\"\n",
+              "            title=\"Suggest charts\"\n",
+              "            style=\"display:none;\">\n",
+              "\n",
+              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
+              "     width=\"24px\">\n",
+              "    <g>\n",
+              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
+              "    </g>\n",
+              "</svg>\n",
+              "  </button>\n",
+              "\n",
+              "<style>\n",
+              "  .colab-df-quickchart {\n",
+              "      --bg-color: #E8F0FE;\n",
+              "      --fill-color: #1967D2;\n",
+              "      --hover-bg-color: #E2EBFA;\n",
+              "      --hover-fill-color: #174EA6;\n",
+              "      --disabled-fill-color: #AAA;\n",
+              "      --disabled-bg-color: #DDD;\n",
+              "  }\n",
+              "\n",
+              "  [theme=dark] .colab-df-quickchart {\n",
+              "      --bg-color: #3B4455;\n",
+              "      --fill-color: #D2E3FC;\n",
+              "      --hover-bg-color: #434B5C;\n",
+              "      --hover-fill-color: #FFFFFF;\n",
+              "      --disabled-bg-color: #3B4455;\n",
+              "      --disabled-fill-color: #666;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart {\n",
+              "    background-color: var(--bg-color);\n",
+              "    border: none;\n",
+              "    border-radius: 50%;\n",
+              "    cursor: pointer;\n",
+              "    display: none;\n",
+              "    fill: var(--fill-color);\n",
+              "    height: 32px;\n",
+              "    padding: 0;\n",
+              "    width: 32px;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart:hover {\n",
+              "    background-color: var(--hover-bg-color);\n",
+              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
+              "    fill: var(--button-hover-fill-color);\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-quickchart-complete:disabled,\n",
+              "  .colab-df-quickchart-complete:disabled:hover {\n",
+              "    background-color: var(--disabled-bg-color);\n",
+              "    fill: var(--disabled-fill-color);\n",
+              "    box-shadow: none;\n",
+              "  }\n",
+              "\n",
+              "  .colab-df-spinner {\n",
+              "    border: 2px solid var(--fill-color);\n",
+              "    border-color: transparent;\n",
+              "    border-bottom-color: var(--fill-color);\n",
+              "    animation:\n",
+              "      spin 1s steps(1) infinite;\n",
+              "  }\n",
+              "\n",
+              "  @keyframes spin {\n",
+              "    0% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "      border-left-color: var(--fill-color);\n",
+              "    }\n",
+              "    20% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    30% {\n",
+              "      border-color: transparent;\n",
+              "      border-left-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    40% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-top-color: var(--fill-color);\n",
+              "    }\n",
+              "    60% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "    }\n",
+              "    80% {\n",
+              "      border-color: transparent;\n",
+              "      border-right-color: var(--fill-color);\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "    90% {\n",
+              "      border-color: transparent;\n",
+              "      border-bottom-color: var(--fill-color);\n",
+              "    }\n",
+              "  }\n",
+              "</style>\n",
+              "\n",
+              "  <script>\n",
+              "    async function quickchart(key) {\n",
+              "      const quickchartButtonEl =\n",
+              "        document.querySelector('#' + key + ' button');\n",
+              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
+              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
+              "      try {\n",
+              "        const charts = await google.colab.kernel.invokeFunction(\n",
+              "            'suggestCharts', [key], {});\n",
+              "      } catch (error) {\n",
+              "        console.error('Error during call to suggestCharts:', error);\n",
+              "      }\n",
+              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
+              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
+              "    }\n",
+              "    (() => {\n",
+              "      let quickchartButtonEl =\n",
+              "        document.querySelector('#df-4a98ca71-ae88-4fd9-afff-7c64e9de394b button');\n",
+              "      quickchartButtonEl.style.display =\n",
+              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
+              "    })();\n",
+              "  </script>\n",
+              "</div>\n",
+              "\n",
+              "    </div>\n",
+              "  </div>\n"
+            ],
+            "application/vnd.google.colaboratory.intrinsic+json": {
+              "type": "dataframe",
+              "summary": "{\n  \"name\": \"c\",\n  \"rows\": 2,\n  \"fields\": [\n    {\n      \"column\": \"StudyInstanceUID\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"1.3.6.1.4.1.14519.5.2.1.1706.8374.121752675166759110829287848146\",\n          \"1.3.6.1.4.1.14519.5.2.1.1706.8374.304819071818674684593765963305\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"modalities\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"CT\",\n          \"CT,SEG\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"collection_id\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"hcc_tace_seg\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"num_series\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 2,\n        \"max\": 3,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"viewer_url\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.1706.8374.121752675166759110829287848146\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
+            }
+          },
+          "metadata": {},
+          "execution_count": 40
+        }
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "rixPlivvj9de"
+      },
+      "source": [
+        "# Want to learn more?\n",
+        "\n",
+        "* check out other notebooks: https://github.com/ImagingDataCommons/IDC-Tutorials\n",
+        "* join our community forum to ask any questions about IDC: https://discourse.canceridc.dev/\n",
+        "* ask your questions during live discussions with IDC developers at the IDC weekly office hours - join us on Google Meet at https://meet.google.com/xyt-vody-tvb every Tuesday 16:30 – 17:30 (New York) and Wednesday 10:30-11:30 (New York)\n",
+        "* browse IDC portal: https://imaging.datacommons.cancer.gov/explore/\n",
+        "* read IDC paper: https://doi.org/10.1148/rg.230180\n",
+        "* watch a recent presentation about IDC: https://youtu.be/P9ateg9ZUEs"
+      ]
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
     }
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "id": "rixPlivvj9de"
-   },
-   "source": [
-    "# Want to learn more?\n",
-    "\n",
-    "* check out other notebooks: https://github.com/ImagingDataCommons/IDC-Tutorials\n",
-    "* join our community forum to ask any questions about IDC: https://discourse.canceridc.dev/\n",
-    "* ask your questions during live discussions with IDC developers at the IDC weekly office hours - join us on Google Meet at https://meet.google.com/xyt-vody-tvb every Tuesday 16:30 – 17:30 (New York) and Wednesday 10:30-11:30 (New York)\n",
-    "* browse IDC portal: https://imaging.datacommons.cancer.gov/explore/\n",
-    "* read IDC paper: https://doi.org/10.1148/rg.230180\n",
-    "* watch a recent presentation about IDC: https://youtu.be/P9ateg9ZUEs"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Support\n",
-    "\n",
-    "If you have any questions about this notebook, please post your question on the [IDC User Forum](https://discourse.canceridc.dev) or [open an issue](https://github.com/ImagingDataCommons/IDC-Tutorials/issues/new) in the [IDC Tutorials repository](https://github.com/ImagingDataCommons/IDC-Tutorials)."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Acknowledgments\n",
-    "\n",
-    "Imaging Data Commons has been funded in whole or in part with Federal funds from the National Cancer Institute, National Institutes of Health, under Task Order No. HHSN26110071 under Contract No. HHSN261201500003I.\n",
-    "\n",
-    "If you use IDC in your research, please cite the following publication:\n",
-    "\n",
-    "> Fedorov, A., Longabaugh, W. J. R., Pot, D., Clunie, D. A., Pieper, S. D., Gibbs, D. L., Bridge, C., Herrmann, M. D., Homeyer, A., Lewis, R., Aerts, H. J. W., Krishnaswamy, D., Thiriveedhi, V. K., Ciausu, C., Schacherer, D. P., Bontempi, D., Pihl, T., Wagner, U., Farahani, K., Kim, E. & Kikinis, R. _National Cancer Institute Imaging Data Commons: Toward Transparency, Reproducibility, and Scalability in Imaging Artificial Intelligence_. RadioGraphics (2023). [https://doi.org/10.1148/rg.230180](https://doi.org/10.1148/rg.230180)"
-   ]
-  }
- ],
- "metadata": {
-  "colab": {
-   "provenance": [],
-   "include_colab_link": true
-  },
-  "kernelspec": {
-   "display_name": "Python 3",
-   "name": "python3"
   },
-  "language_info": {
-   "name": "python"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
+  "nbformat": 4,
+  "nbformat_minor": 0
+}
\ No newline at end of file
diff --git a/notebooks/advanced_topics/related_external_data/cddp_eagle_gdc_pathology_reports.ipynb b/notebooks/advanced_topics/related_external_data/cddp_eagle_gdc_pathology_reports.ipynb
new file mode 100644
index 0000000..30f1c7d
--- /dev/null
+++ b/notebooks/advanced_topics/related_external_data/cddp_eagle_gdc_pathology_reports.ipynb
@@ -0,0 +1,451 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# GDC Pathology Reports for CDDP_EAGLE-1 Cases in IDC\n",
+    "\n",
+    "[CDDP_EAGLE](https://doi.org/10.7937/TCIA.2019.f4mcnm36) is a lung adenocarcinoma cohort (49 patients) with CT and whole-slide images available in IDC (`cddp_eagle_1`).\n",
+    "The GDC hosts a matched pathology report PDF for 48 of the 49 IDC patients (98%).\n",
+    "\n",
+    "**Key difference from TCGA reports:** these PDFs are **pure scans** — no embedded text layer.\n",
+    "OCR is required to extract text.\n",
+    "The scans are high quality (600 DPI, ~2 MB each) and OCR with tesseract takes ~1.5 s/page.\n",
+    "\n",
+    "**No authentication required** — GDC pathology reports are open-access files.\n",
+    "\n",
+    "## Requirements\n",
+    "\n",
+    "```bash\n",
+    "# System\n",
+    "brew install tesseract          # macOS\n",
+    "sudo apt install tesseract-ocr  # Ubuntu/Debian\n",
+    "\n",
+    "# Python\n",
+    "pip install idc-index requests pymupdf pytesseract\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import csv\n",
+    "import io\n",
+    "import json\n",
+    "import os\n",
+    "import re\n",
+    "import tempfile\n",
+    "import time\n",
+    "from collections import Counter, defaultdict\n",
+    "from pathlib import Path\n",
+    "\n",
+    "import fitz          # pymupdf\n",
+    "import pytesseract\n",
+    "import requests\n",
+    "from idc_index import IDCClient\n",
+    "\n",
+    "GDC_FILES_URL = \"https://api.gdc.cancer.gov/files\"\n",
+    "GDC_DATA_URL  = \"https://api.gdc.cancer.gov/data\"\n",
+    "\n",
+    "client = IDCClient()\n",
+    "print(\"IDC version :\", client.get_idc_version())\n",
+    "print(\"tesseract   :\", pytesseract.get_tesseract_version())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Get CDDP Patient IDs from IDC\n",
+    "\n",
+    "`PatientID` in IDC equals the GDC `submitter_id` (format: `CDDP-XXXX`)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "df = client.sql_query(\"\"\"\n",
+    "    SELECT DISTINCT PatientID\n",
+    "    FROM index\n",
+    "    WHERE collection_id = 'cddp_eagle_1'\n",
+    "    ORDER BY PatientID\n",
+    "\"\"\")\n",
+    "\n",
+    "patient_ids = list(df[\"PatientID\"])\n",
+    "print(f\"{len(patient_ids)} patients in cddp_eagle_1\")\n",
+    "print(\"Sample:\", patient_ids[:5])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Query GDC for Pathology Report Metadata"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "payload = {\n",
+    "    \"filters\": json.dumps({\n",
+    "        \"op\": \"and\",\n",
+    "        \"content\": [\n",
+    "            {\"op\": \"in\", \"content\": {\"field\": \"cases.submitter_id\", \"value\": patient_ids}},\n",
+    "            {\"op\": \"=\",  \"content\": {\"field\": \"data_type\", \"value\": \"Pathology Report\"}},\n",
+    "        ],\n",
+    "    }),\n",
+    "    \"fields\": \"file_id,file_name,cases.submitter_id,file_size,md5sum\",\n",
+    "    \"size\": str(len(patient_ids) + 10),\n",
+    "    \"format\": \"JSON\",\n",
+    "}\n",
+    "r = requests.get(GDC_FILES_URL, params=payload, timeout=30)\n",
+    "r.raise_for_status()\n",
+    "reports = r.json()[\"data\"][\"hits\"]\n",
+    "\n",
+    "case_to_report = {h[\"cases\"][0][\"submitter_id\"]: h for h in reports}\n",
+    "\n",
+    "matched   = set(patient_ids) & case_to_report.keys()\n",
+    "unmatched = set(patient_ids) - case_to_report.keys()\n",
+    "print(f\"Reports found    : {len(reports)}\")\n",
+    "print(f\"Patients matched : {len(matched)} / {len(patient_ids)} ({len(matched)/len(patient_ids)*100:.1f}%)\")\n",
+    "print(f\"Patients missing : {sorted(unmatched)}\")\n",
+    "\n",
+    "sizes = [h[\"file_size\"] for h in reports]\n",
+    "print(f\"\\nFile size  min={min(sizes)/1e6:.1f} MB  \"\n",
+    "      f\"median={sorted(sizes)[len(sizes)//2]/1e6:.1f} MB  \"\n",
+    "      f\"max={max(sizes)/1e6:.1f} MB\")\n",
+    "\n",
+    "sample = reports[0]\n",
+    "print(f\"\\nExample — {sample['cases'][0]['submitter_id']}:\")\n",
+    "print(f\"  file_name: {sample['file_name']}\")\n",
+    "print(f\"  file_id  : {sample['file_id']}\")\n",
+    "print(f\"  download : {GDC_DATA_URL}/{sample['file_id']}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Download Reports\n",
+    "\n",
+    "Each PDF is ~2 MB. The full collection (48 files) is ~100 MB."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def download_gdc_file(file_id: str, dest_path: Path) -> Path:\n",
+    "    \"\"\"Stream a GDC file to disk.\"\"\"\n",
+    "    dest_path.parent.mkdir(parents=True, exist_ok=True)\n",
+    "    with requests.get(f\"{GDC_DATA_URL}/{file_id}\", stream=True, timeout=120) as r:\n",
+    "        r.raise_for_status()\n",
+    "        with open(dest_path, \"wb\") as f:\n",
+    "            for chunk in r.iter_content(1 << 20):\n",
+    "                f.write(chunk)\n",
+    "    return dest_path\n",
+    "\n",
+    "\n",
+    "def download_reports(\n",
+    "    case_to_report: dict,\n",
+    "    output_dir: str | Path,\n",
+    "    limit: int | None = None,\n",
+    "    delay_s: float = 0.3,\n",
+    ") -> list[Path]:\n",
+    "    \"\"\"\n",
+    "    Download CDDP_EAGLE pathology report PDFs.\n",
+    "\n",
+    "    Parameters\n",
+    "    ----------\n",
+    "    case_to_report : dict mapping submitter_id → GDC file-metadata dict\n",
+    "    output_dir     : destination; PDFs saved as {submitter_id}/{file_name}\n",
+    "    limit          : max cases to download (None = all)\n",
+    "    delay_s        : polite pause between requests\n",
+    "    \"\"\"\n",
+    "    output_dir = Path(output_dir)\n",
+    "    downloaded = []\n",
+    "    items = list(case_to_report.items())\n",
+    "    if limit:\n",
+    "        items = items[:limit]\n",
+    "\n",
+    "    for case_id, meta in items:\n",
+    "        dest = output_dir / case_id / meta[\"file_name\"]\n",
+    "        if dest.exists():\n",
+    "            print(f\"  skip (exists): {dest.name}\")\n",
+    "            downloaded.append(dest)\n",
+    "            continue\n",
+    "        print(f\"  downloading {case_id} — {meta['file_size']/1e6:.1f} MB\")\n",
+    "        download_gdc_file(meta[\"file_id\"], dest)\n",
+    "        downloaded.append(dest)\n",
+    "        time.sleep(delay_s)\n",
+    "\n",
+    "    return downloaded\n",
+    "\n",
+    "\n",
+    "# Download a sample (3 cases)\n",
+    "paths = download_reports(\n",
+    "    case_to_report,\n",
+    "    output_dir=\"/tmp/gdc_pathology/cddp_eagle_1\",\n",
+    "    limit=3,\n",
+    ")\n",
+    "for p in paths:\n",
+    "    print(f\"  {p}  ({p.stat().st_size/1e6:.1f} MB)\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. OCR Text Extraction\n",
+    "\n",
+    "CDDP_EAGLE reports are **pure scans** — no embedded text layer.\n",
+    "Every PDF is a single full-page image at 600 DPI (5,100 × 6,600 px).\n",
+    "Tesseract extracts clean text in ~1.5 s/page.\n",
+    "\n",
+    "### PDF structure\n",
+    "\n",
+    "| Property | Value |\n",
+    "|----------|-------|\n",
+    "| Pages | 1–2 |\n",
+    "| Image resolution | 5,100 × 6,600 px (600 DPI letter) |\n",
+    "| File size | 1.5 – 4.4 MB |\n",
+    "| Text layer | None — OCR required |\n",
+    "| OCR speed (tesseract 5, Apple M-series) | ~1.5 s/page |\n",
+    "\n",
+    "### Report sections\n",
+    "\n",
+    "Each report contains (in order):\n",
+    "1. **Material description** — tissue specimens labeled A, B, C …\n",
+    "2. **Macro description** — gross dimensions, appearance\n",
+    "3. **Micro description** — histology, WHO classification, grade, margin status, lymph node status\n",
+    "4. **Staging** — `pTNM: pTx Nx` and ICD-10 code\n",
+    "5. **TCGA-style QA footer** — Diagnosis Discrepancy checkboxes (noisy, ignore)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def ocr_pdf(pdf_path: Path, dpi: int = 300) -> str:\n",
+    "    \"\"\"\n",
+    "    OCR a scan-only PDF using PyMuPDF + tesseract.\n",
+    "\n",
+    "    dpi=300 is a good balance of speed and accuracy for these 600 DPI originals.\n",
+    "    Use dpi=200 for batch speed; dpi=400 for maximum quality.\n",
+    "    \"\"\"\n",
+    "    doc = fitz.open(pdf_path)\n",
+    "    pages_text = []\n",
+    "    for page in doc:\n",
+    "        mat = fitz.Matrix(dpi / 72, dpi / 72)\n",
+    "        pix = page.get_pixmap(matrix=mat, colorspace=fitz.csGRAY)\n",
+    "        with tempfile.NamedTemporaryFile(suffix=\".png\", delete=False) as f:\n",
+    "            tmp = f.name\n",
+    "        pix.save(tmp)\n",
+    "        text = pytesseract.image_to_string(tmp, lang=\"eng\", config=\"--psm 6\")\n",
+    "        os.unlink(tmp)\n",
+    "        pages_text.append(text)\n",
+    "    doc.close()\n",
+    "    return \"\\n\\n\".join(pages_text)\n",
+    "\n",
+    "\n",
+    "# Demo on downloaded samples\n",
+    "for p in sorted(Path(\"/tmp/gdc_pathology/cddp_eagle_1\").rglob(\"*.pdf\")):\n",
+    "    t0 = time.time()\n",
+    "    text = ocr_pdf(p)\n",
+    "    elapsed = time.time() - t0\n",
+    "    doc = fitz.open(p)\n",
+    "    pages = len(doc); doc.close()\n",
+    "    print(f\"{p.parent.name}  pages={pages}  {elapsed:.1f}s  {len(text):,} chars\")\n",
+    "    print(text[:500].strip())\n",
+    "    print()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5. Structured Field Extraction\n",
+    "\n",
+    "The OCR'd text is clean enough for regex extraction of key staging fields.\n",
+    "CDDP_EAGLE reports are from a single Italian institution (EAGLE trial) and\n",
+    "follow a consistent template, which makes pattern matching reliable."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def extract_staging_fields(text: str) -> dict:\n",
+    "    \"\"\"Best-effort regex extraction from OCR'd CDDP_EAGLE pathology reports.\"\"\"\n",
+    "    fields = {}\n",
+    "\n",
+    "    # TNM — pT*, pN*, pM* (including combined pT2,N0 style)\n",
+    "    tnm = re.findall(r'p[TtNnMm]\\d[\\w,]*', text)\n",
+    "    if tnm:\n",
+    "        fields[\"tnm\"] = list(dict.fromkeys(tnm))\n",
+    "\n",
+    "    # Histologic grade — G1–G4 or 'grade N'\n",
+    "    grade = re.findall(r'\\bG\\s*[1-4]\\b|[Gg]rade\\s+[1-4]', text)\n",
+    "    if grade:\n",
+    "        fields[\"grade\"] = list(dict.fromkeys(grade))\n",
+    "\n",
+    "    # WHO histology classification line\n",
+    "    who = re.search(\n",
+    "        r'(?:Pulmonary|Lung)[^\\n]{0,60}(?:adenocarcinoma|carcinoma|squamous)[^\\n]{0,80}',\n",
+    "        text, re.IGNORECASE\n",
+    "    )\n",
+    "    if who:\n",
+    "        fields[\"histology\"] = who.group(0).strip()\n",
+    "\n",
+    "    # Tumor 3D size from gross description — e.g. '3.2 x 2.5 x 2 cm'\n",
+    "    sizes_3d = re.findall(\n",
+    "        r'(\\d+\\.?\\d*)\\s*[xX×]\\s*(\\d+\\.?\\d*)\\s*[xX×]\\s*(\\d+\\.?\\d*)\\s*(cm|mm)',\n",
+    "        text, re.IGNORECASE\n",
+    "    )\n",
+    "    if sizes_3d:\n",
+    "        fields[\"tumor_size_3d\"] = [f\"{a}×{b}×{c} {u}\" for a, b, c, u in sizes_3d[:2]]\n",
+    "\n",
+    "    # Lymph node status — 'N lymph nodes free of metastases'\n",
+    "    ln_free = re.findall(r'(\\d+)\\s*(?:peribronchial\\s*)?lymph\\s*nodes?\\s*free', text, re.IGNORECASE)\n",
+    "    if ln_free:\n",
+    "        fields[\"lymph_nodes_free\"] = [int(n) for n in ln_free]\n",
+    "\n",
+    "    # ICD-10 site code — e.g. C34.1, C34.3\n",
+    "    icd = re.findall(r'\\bC3[0-9]\\.[0-9]\\b', text)\n",
+    "    if icd:\n",
+    "        fields[\"icd10_site\"] = list(dict.fromkeys(icd))\n",
+    "\n",
+    "    # Margin status\n",
+    "    if re.search(r'margin[s]?\\s*(?:of\\s*\\w+\\s*)?(?:free|negative|clear)', text, re.IGNORECASE):\n",
+    "        fields[\"margins\"] = \"negative\"\n",
+    "\n",
+    "    return fields\n",
+    "\n",
+    "\n",
+    "for p in sorted(Path(\"/tmp/gdc_pathology/cddp_eagle_1\").rglob(\"*.pdf\")):\n",
+    "    text = ocr_pdf(p)\n",
+    "    fields = extract_staging_fields(text)\n",
+    "    print(f\"{p.parent.name}\")\n",
+    "    for k, v in fields.items():\n",
+    "        print(f\"  {k:20s}: {v}\")\n",
+    "    print()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Batch OCR and Save to Text Files\n",
+    "\n",
+    "For larger workflows, OCR all downloaded reports once and save `.txt` sidecars.\n",
+    "Subsequent runs skip already-processed files."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def ocr_collection(\n",
+    "    pdf_root: str | Path,\n",
+    "    dpi: int = 300,\n",
+    "    force: bool = False,\n",
+    ") -> dict[str, str]:\n",
+    "    \"\"\"\n",
+    "    OCR all PDFs under pdf_root, saving .txt sidecars next to each PDF.\n",
+    "\n",
+    "    Returns dict mapping case_id → extracted text.\n",
+    "    \"\"\"\n",
+    "    pdf_root = Path(pdf_root)\n",
+    "    results = {}\n",
+    "    pdfs = sorted(pdf_root.rglob(\"*.pdf\"))\n",
+    "    print(f\"Found {len(pdfs)} PDFs under {pdf_root}\")\n",
+    "\n",
+    "    for pdf in pdfs:\n",
+    "        case_id = pdf.parent.name\n",
+    "        txt_path = pdf.with_suffix(\".txt\")\n",
+    "\n",
+    "        if txt_path.exists() and not force:\n",
+    "            text = txt_path.read_text()\n",
+    "            print(f\"  {case_id}  (cached, {len(text):,} chars)\")\n",
+    "        else:\n",
+    "            t0 = time.time()\n",
+    "            text = ocr_pdf(pdf, dpi=dpi)\n",
+    "            txt_path.write_text(text)\n",
+    "            print(f\"  {case_id}  OCR {time.time()-t0:.1f}s → {len(text):,} chars\")\n",
+    "\n",
+    "        results[case_id] = text\n",
+    "\n",
+    "    return results\n",
+    "\n",
+    "\n",
+    "texts = ocr_collection(\"/tmp/gdc_pathology/cddp_eagle_1\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 7. Generate a GDC Download Manifest\n",
+    "\n",
+    "For downloading all 48 reports (~100 MB) the [GDC Data Transfer Tool](https://gdc.cancer.gov/access-data/gdc-data-transfer-tool) is more efficient than streaming individually."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def make_gdc_manifest(reports: list[dict]) -> str:\n",
+    "    \"\"\"Return manifest TSV for: gdc-client download -m manifest.txt -d ./output\"\"\"\n",
+    "    buf = io.StringIO()\n",
+    "    w = csv.writer(buf, delimiter=\"\\t\")\n",
+    "    w.writerow([\"id\", \"filename\", \"md5\", \"size\", \"state\"])\n",
+    "    for r in reports:\n",
+    "        w.writerow([r[\"file_id\"], r[\"file_name\"], r.get(\"md5sum\", \"\"), r.get(\"file_size\", \"\"), \"released\"])\n",
+    "    return buf.getvalue()\n",
+    "\n",
+    "\n",
+    "manifest = make_gdc_manifest(reports)\n",
+    "manifest_path = Path(\"/tmp/gdc_pathology/cddp_eagle_1_manifest.txt\")\n",
+    "manifest_path.parent.mkdir(parents=True, exist_ok=True)\n",
+    "manifest_path.write_text(manifest)\n",
+    "\n",
+    "print(f\"Manifest: {manifest_path} ({len(reports)} files)\")\n",
+    "print(\"\\n\".join(manifest.splitlines()[:4]))\n",
+    "print(f\"\\n# gdc-client download -m {manifest_path} -d /path/to/output\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.11.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/notebooks/advanced_topics/related_external_data/gtex_pathology_annotations.ipynb b/notebooks/advanced_topics/related_external_data/gtex_pathology_annotations.ipynb
new file mode 100644
index 0000000..88a9fba
--- /dev/null
+++ b/notebooks/advanced_topics/related_external_data/gtex_pathology_annotations.ipynb
@@ -0,0 +1,330 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "# GTEx Pathology Annotations for Tissue Slides in IDC\n\nThe [GTEx](https://gtexportal.org) collection in IDC (`gtex`) contains 25,503 H&E whole-slide images from 971 healthy postmortem donors across 50+ tissue types.\nEach slide was reviewed and annotated by a board-certified pathologist.\n\n**Source:** GTEx portal API (`https://gtexportal.org/api/v2/biobank/sample`) — not GDC.\nNo PDF reports exist; annotations are returned as structured JSON.\n\n**Coverage:** 22,909 of 25,503 IDC tissue samples (89.8%) matched; 100% of those have pathology notes.\n\n## Annotation types\n\n| Field | Type | Content |\n|-------|------|---------|\n| `pathologyNotes` | Free text | Specimen quality notes — piece count, autolysis, congestion, special findings |\n| `pathologyNotesCategories` | Dict of 57 booleans | Structured pathological findings (atherosclerosis, fibrosis, congestion, …) |\n\n## Join key\n\n| IDC field | GTEx API field | How to map |\n|-----------|----------------|-----------|\n| `sm_index.ContainerIdentifier` | `biobank/sample.tissueSampleId` | See note below |\n\n**ContainerIdentifier format note:**\n\n99% of IDC ContainerIdentifiers use the format `GTEX-DONOR-XYZS5` (4-digit suffix ending in `5`).\nThese map to the GTEx API `tissueSampleId` `GTEX-DONOR-XYZS6` — same prefix, last digit `5→6`.\n\nExample: `GTEX-1117F-1025` (IDC) ↔ `GTEX-1117F-1026` (API)\n\nThe remaining ~1% (263 slides) use a different schema (`GTEX-DONOR-5XXX`) that does not appear\nin the current GTEx biobank API and are therefore unmatched.\n\n## Requirements\n\n```\npip install idc-index requests pandas\n```"
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "import time\n",
+    "from pathlib import Path\n",
+    "\n",
+    "import pandas as pd\n",
+    "import requests\n",
+    "from idc_index import IDCClient\n",
+    "\n",
+    "GTEX_SAMPLE_API = \"https://gtexportal.org/api/v2/biobank/sample\"\n",
+    "\n",
+    "client = IDCClient()\n",
+    "client.fetch_index(\"sm_index\")\n",
+    "print(\"IDC version:\", client.get_idc_version())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Get Tissue Sample IDs from IDC\n",
+    "\n",
+    "`sm_index.ContainerIdentifier` holds the GTEx tissue sample ID as recorded on the slide (e.g., `GTEX-1117F-1025`).\n",
+    "`PatientID` (e.g., `GTEX-1117F`) is the donor/subject ID.\n",
+    "Both come from the sm_index joined to the primary index."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "idc_samples = client.sql_query(\"\"\"\n",
+    "    SELECT DISTINCT\n",
+    "        i.PatientID                                    AS donor_id,\n",
+    "        s.ContainerIdentifier                          AS container_id,\n",
+    "        s.primaryAnatomicStructure_CodeMeaning         AS tissue,\n",
+    "        s.SeriesInstanceUID\n",
+    "    FROM sm_index s\n",
+    "    JOIN index i USING (SeriesInstanceUID)\n",
+    "    WHERE i.collection_id = 'gtex'\n",
+    "    ORDER BY i.PatientID, s.ContainerIdentifier\n",
+    "\"\"\")\n",
+    "\n",
+    "print(f\"IDC GTEx tissue samples : {len(idc_samples):,}\")\n",
+    "print(f\"Unique donors           : {idc_samples['donor_id'].nunique()}\")\n",
+    "print(f\"Unique tissues          : {idc_samples['tissue'].nunique()}\")\n",
+    "print()\n",
+    "print(idc_samples.head(6).to_string(index=False))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Fetch Pathology Annotations from GTEx API\n",
+    "\n",
+    "The GTEx portal API returns all biobank sample records including pathology notes.\n",
+    "Pagination is required (250 records/page, ~303 pages for the full dataset).\n",
+    "\n",
+    "**Runtime:** ~2 minutes to download all 75,568 sample records."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def fetch_all_gtex_samples(page_size: int = 250, cache_path: Path | None = None) -> dict[str, dict]:\n",
+    "    \"\"\"\n",
+    "    Download all GTEx biobank sample records and return a dict keyed by tissueSampleId.\n",
+    "\n",
+    "    If cache_path is given, saves/loads a JSON cache to avoid re-downloading.\n",
+    "    \"\"\"\n",
+    "    if cache_path and Path(cache_path).exists():\n",
+    "        print(f\"Loading cache: {cache_path}\")\n",
+    "        return json.loads(Path(cache_path).read_text())\n",
+    "\n",
+    "    samples_by_id = {}\n",
+    "    page = 0\n",
+    "    while True:\n",
+    "        r = requests.get(GTEX_SAMPLE_API,\n",
+    "            params={\"pageSize\": page_size, \"page\": page}, timeout=30)\n",
+    "        r.raise_for_status()\n",
+    "        data = r.json()\n",
+    "        for s in data[\"sample\"]:\n",
+    "            samples_by_id[s[\"tissueSampleId\"]] = s\n",
+    "        if page >= data[\"numPages\"] - 1:\n",
+    "            break\n",
+    "        page += 1\n",
+    "        if page % 50 == 0:\n",
+    "            print(f\"  page {page}/{data['numPages']}  ({len(samples_by_id):,} records)\")\n",
+    "\n",
+    "    print(f\"Total records fetched: {len(samples_by_id):,}\")\n",
+    "    if cache_path:\n",
+    "        Path(cache_path).write_text(json.dumps(samples_by_id))\n",
+    "        print(f\"Cached to: {cache_path}\")\n",
+    "    return samples_by_id\n",
+    "\n",
+    "\n",
+    "# ~2 min first run; instant on subsequent runs with cache\n",
+    "api_samples = fetch_all_gtex_samples(\n",
+    "    cache_path=\"/tmp/gtex_api_samples.json\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "## 3. Join IDC Slides to Pathology Annotations\n\n**99% of IDC ContainerIdentifiers** use the format `GTEX-DONOR-XYZS5`. These map to the GTEx API\n`tissueSampleId` `GTEX-DONOR-XYZS6` (same prefix, last digit `5→6`).\n\nThe remaining ~1% use a different schema (`GTEX-DONOR-5XXX`) not present in the current API.\nThey are included in the output with null annotation fields."
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def gtex_api_id(container_id: str) -> str:\n",
+    "    \"\"\"Convert IDC ContainerIdentifier to GTEx API tissueSampleId.\"\"\"\n",
+    "    return container_id[:-1] + \"6\"\n",
+    "\n",
+    "\n",
+    "rows = []\n",
+    "for _, row in idc_samples.iterrows():\n",
+    "    api_id = gtex_api_id(row[\"container_id\"])\n",
+    "    api = api_samples.get(api_id)\n",
+    "    rows.append({\n",
+    "        \"donor_id\"          : row[\"donor_id\"],\n",
+    "        \"container_id\"      : row[\"container_id\"],\n",
+    "        \"api_sample_id\"     : api_id,\n",
+    "        \"tissue_idc\"        : row[\"tissue\"],\n",
+    "        \"tissue_api\"        : api[\"tissueSiteDetail\"] if api else None,\n",
+    "        \"sex\"               : api[\"sex\"]              if api else None,\n",
+    "        \"age_bracket\"       : api[\"ageBracket\"]       if api else None,\n",
+    "        \"hardy_scale\"       : api[\"hardyScale\"]       if api else None,\n",
+    "        \"autolysis_score\"   : api[\"autolysisScore\"]   if api else None,\n",
+    "        \"pathology_notes\"   : api[\"pathologyNotes\"]   if api else None,\n",
+    "        \"pathology_cats\"    : api[\"pathologyNotesCategories\"] if api else None,\n",
+    "        \"SeriesInstanceUID\" : row[\"SeriesInstanceUID\"],\n",
+    "    })\n",
+    "\n",
+    "df = pd.DataFrame(rows)\n",
+    "matched   = df[\"tissue_api\"].notna()\n",
+    "has_notes = df[\"pathology_notes\"].fillna('').str.len() > 0\n",
+    "\n",
+    "print(f\"Joined records        : {len(df):,}\")\n",
+    "print(f\"Matched in API        : {matched.sum():,} ({matched.mean()*100:.1f}%)\")\n",
+    "print(f\"Has pathology notes   : {has_notes.sum():,} ({has_notes.mean()*100:.1f}%)\")\n",
+    "print()\n",
+    "print(df[matched].head(4)[[\"donor_id\",\"tissue_api\",\"age_bracket\",\"hardy_scale\",\"pathology_notes\"]].to_string(index=False))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Structured Pathology Categories\n",
+    "\n",
+    "The `pathologyNotesCategories` dict contains up to 56 boolean flags for specific pathological findings.\n",
+    "Categories are tissue-type specific — e.g., `spermatogenesis` only applies to testis,\n",
+    "`glomerulosclerosis` to kidney, `emphysema` to lung.\n",
+    "\n",
+    "~22% of matched samples have at least one True category; the rest have all-False (typical for clean specimens)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from collections import Counter\n",
+    "\n",
+    "# Expand pathology_cats column into boolean columns\n",
+    "cat_dicts = df.loc[matched & df[\"pathology_cats\"].notna(), \"pathology_cats\"]\n",
+    "\n",
+    "# Get full list of category keys\n",
+    "all_cats = sorted({k for d in cat_dicts for k in d.keys()})\n",
+    "print(f\"Pathology category keys ({len(all_cats)}):\")\n",
+    "print(all_cats)\n",
+    "\n",
+    "# Expand to boolean DataFrame\n",
+    "cats_df = pd.DataFrame(list(cat_dicts), index=cat_dicts.index).fillna(False).astype(bool)\n",
+    "\n",
+    "# Prevalence across all matched samples with categories\n",
+    "prevalence = cats_df.sum().sort_values(ascending=False)\n",
+    "print(f\"\\nTop 20 most common True categories (out of {len(cats_df):,} samples with categories):\")\n",
+    "print(prevalence.head(20).to_string())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "from collections import Counter\n\n# Expand pathology_cats column into boolean columns\ncat_dicts = df.loc[matched & df[\"pathology_cats\"].notna(), \"pathology_cats\"]\n\n# Get full list of category keys\nall_cats = sorted({k for d in cat_dicts for k in d.keys()})\nprint(f\"Pathology category keys ({len(all_cats)}):\")\nprint(all_cats)\n\n# Expand to boolean DataFrame\ncats_df = (\n    pd.DataFrame(list(cat_dicts), index=cat_dicts.index)\n    .infer_objects(copy=False)\n    .fillna(False)\n    .astype(bool)\n)\n\n# Prevalence across all matched samples with categories\nprevalence = cats_df.sum().sort_values(ascending=False)\nprint(f\"\\nTop 20 most common True categories (out of {len(cats_df):,} samples with categories):\")\nprint(prevalence.head(20).to_string())"
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5. Donor Metadata\n",
+    "\n",
+    "The API also provides basic donor metadata per tissue sample:\n",
+    "- `sex`: male / female\n",
+    "- `ageBracket`: age range (20-29, 30-39, …, 70-79)\n",
+    "- `hardyScale`: death circumstance — Ventilator case / Fast death – violent / Fast death – natural causes / Intermediate / Slow death\n",
+    "- `autolysisScore`: tissue quality — None / Mild / Moderate / Severe"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "matched_df = df[matched].copy()\n",
+    "\n",
+    "print(\"Sex distribution:\")\n",
+    "print(matched_df.drop_duplicates(\"donor_id\")[\"sex\"].value_counts().to_string())\n",
+    "\n",
+    "print(\"\\nAge bracket distribution:\")\n",
+    "print(matched_df.drop_duplicates(\"donor_id\")[\"age_bracket\"].value_counts().sort_index().to_string())\n",
+    "\n",
+    "print(\"\\nHardy scale (death circumstance):\")\n",
+    "print(matched_df.drop_duplicates(\"donor_id\")[\"hardy_scale\"].value_counts().to_string())\n",
+    "\n",
+    "print(\"\\nAutolysis score (tissue quality):\")\n",
+    "print(matched_df[\"autolysis_score\"].value_counts(dropna=False).to_string())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Per-Donor Lookup\n",
+    "\n",
+    "Fetch annotations for a single donor, merge with IDC viewer URLs."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "DONOR = \"GTEX-1117F\"\n",
+    "\n",
+    "donor_df = df[df[\"donor_id\"] == DONOR].copy()\n",
+    "print(f\"{DONOR} — {len(donor_df)} tissue slides\")\n",
+    "print()\n",
+    "\n",
+    "for _, row in donor_df.iterrows():\n",
+    "    url = client.get_viewer_URL(seriesInstanceUID=row[\"SeriesInstanceUID\"])\n",
+    "    notes = row.get(\"pathology_notes\") or \"—\"\n",
+    "    # Flag any non-clean categories\n",
+    "    cats = row.get(\"pathology_cats\") or {}\n",
+    "    findings = [k for k, v in cats.items() if v and k != \"clean_specimens\"]\n",
+    "    print(f\"  {row['tissue_api'] or row['tissue_idc']:40s}  {notes[:60]}\")\n",
+    "    if findings:\n",
+    "        print(f\"    findings: {findings}\")\n",
+    "    # Uncomment to open in browser:\n",
+    "    # import webbrowser; webbrowser.open(url)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 7. Save Joined Annotations as CSV\n",
+    "\n",
+    "Export the full join as a flat CSV for downstream analysis."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "out_path = Path(\"/tmp/gtex_pathology_annotations.csv\")\n",
+    "\n",
+    "# Expand pathology_cats dict into flat boolean columns\n",
+    "cats_expanded = pd.DataFrame(\n",
+    "    df[\"pathology_cats\"].apply(lambda x: x if isinstance(x, dict) else {}).tolist(),\n",
+    "    index=df.index\n",
+    ").fillna(False).astype(bool)\n",
+    "\n",
+    "out_df = pd.concat([\n",
+    "    df[[\"donor_id\", \"container_id\", \"api_sample_id\", \"tissue_api\",\n",
+    "        \"sex\", \"age_bracket\", \"hardy_scale\", \"autolysis_score\",\n",
+    "        \"pathology_notes\", \"SeriesInstanceUID\"]],\n",
+    "    cats_expanded\n",
+    "], axis=1)\n",
+    "\n",
+    "out_df.to_csv(out_path, index=False)\n",
+    "print(f\"Saved: {out_path}\")\n",
+    "print(f\"Shape: {out_df.shape}  ({out_df.shape[1]} columns = 10 metadata + {cats_expanded.shape[1]} category flags)\")\n",
+    "print(out_df.head(3).to_string(index=False))"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "python3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
\ No newline at end of file
diff --git a/notebooks/advanced_topics/related_external_data/tcga_gdc_pathology_reports.ipynb b/notebooks/advanced_topics/related_external_data/tcga_gdc_pathology_reports.ipynb
new file mode 100644
index 0000000..0825859
--- /dev/null
+++ b/notebooks/advanced_topics/related_external_data/tcga_gdc_pathology_reports.ipynb
@@ -0,0 +1,470 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Retrieving GDC Pathology Reports for TCGA Cases in IDC\n",
+    "\n",
+    "TCGA whole-slide images in IDC are linked to pathology report PDFs hosted on the NCI Genomic Data Commons (GDC).  \n",
+    "The join key is the **TCGA case barcode** (`PatientID` in IDC, `submitter_id` in GDC).\n",
+    "\n",
+    "**Coverage (TCGA-BRCA baseline):** 1,094 of 1,098 IDC patients (99.6%) have a pathology report on GDC.  \n",
+    "A small number of patients have two reports; the rest have exactly one.\n",
+    "\n",
+    "**No authentication required** — GDC pathology reports are open-access files.\n",
+    "\n",
+    "## Requirements\n",
+    "\n",
+    "```\n",
+    "pip install idc-index requests pymupdf\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import csv\n",
+    "import io\n",
+    "import json\n",
+    "import re\n",
+    "import time\n",
+    "from collections import Counter, defaultdict\n",
+    "from pathlib import Path\n",
+    "\n",
+    "import fitz          # pymupdf\n",
+    "import pandas as pd\n",
+    "import requests\n",
+    "from idc_index import IDCClient\n",
+    "\n",
+    "GDC_FILES_URL = \"https://api.gdc.cancer.gov/files\"\n",
+    "GDC_DATA_URL  = \"https://api.gdc.cancer.gov/data\"\n",
+    "\n",
+    "client = IDCClient()\n",
+    "print(\"IDC version:\", client.get_idc_version())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Get TCGA Patient IDs from IDC\n",
+    "\n",
+    "`PatientID` in IDC equals the GDC `submitter_id` for TCGA collections."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "COLLECTION = \"tcga_brca\"   # change to any tcga_* collection_id\n",
+    "\n",
+    "df = client.sql_query(f\"\"\"\n",
+    "    SELECT DISTINCT PatientID\n",
+    "    FROM index\n",
+    "    WHERE collection_id = '{COLLECTION}'\n",
+    "    ORDER BY PatientID\n",
+    "\"\"\")\n",
+    "\n",
+    "patient_ids = list(df[\"PatientID\"])\n",
+    "print(f\"{len(patient_ids)} patients in {COLLECTION}\")\n",
+    "print(\"Sample:\", patient_ids[:5])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Query GDC for Pathology Report File Metadata\n",
+    "\n",
+    "GDC stores pathology reports as open-access PDF files.\n",
+    "We filter by `data_type = \"Pathology Report\"` and match on `cases.submitter_id`.\n",
+    "\n",
+    "Batching is required — GDC filters with `in` operator accept up to ~500 values per request."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def query_gdc_pathology_reports(submitter_ids, batch_size=200):\n",
+    "    \"\"\"Return list of GDC file metadata dicts for the given TCGA case submitter_ids.\"\"\"\n",
+    "    all_hits = []\n",
+    "    for i in range(0, len(submitter_ids), batch_size):\n",
+    "        batch = submitter_ids[i : i + batch_size]\n",
+    "        payload = {\n",
+    "            \"filters\": json.dumps({\n",
+    "                \"op\": \"and\",\n",
+    "                \"content\": [\n",
+    "                    {\"op\": \"in\", \"content\": {\"field\": \"cases.submitter_id\", \"value\": batch}},\n",
+    "                    {\"op\": \"=\",  \"content\": {\"field\": \"data_type\", \"value\": \"Pathology Report\"}},\n",
+    "                ],\n",
+    "            }),\n",
+    "            \"fields\": \"file_id,file_name,cases.submitter_id,file_size,md5sum\",\n",
+    "            \"size\": str(batch_size),\n",
+    "            \"format\": \"JSON\",\n",
+    "        }\n",
+    "        r = requests.get(GDC_FILES_URL, params=payload, timeout=30)\n",
+    "        r.raise_for_status()\n",
+    "        all_hits.extend(r.json()[\"data\"][\"hits\"])\n",
+    "    return all_hits\n",
+    "\n",
+    "\n",
+    "reports = query_gdc_pathology_reports(patient_ids)\n",
+    "print(f\"{len(reports)} reports found for {len(patient_ids)} IDC patients\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Build a Lookup Table: Case → Report(s)\n",
+    "\n",
+    "Most patients have exactly one report; a few have two."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "case_to_reports = defaultdict(list)\n",
+    "for hit in reports:\n",
+    "    submitter_id = hit[\"cases\"][0][\"submitter_id\"]\n",
+    "    case_to_reports[submitter_id].append(hit)\n",
+    "\n",
+    "idc_patients_set = set(patient_ids)\n",
+    "matched   = idc_patients_set & case_to_reports.keys()\n",
+    "unmatched = idc_patients_set - case_to_reports.keys()\n",
+    "\n",
+    "print(f\"IDC patients with ≥1 GDC report : {len(matched)} ({len(matched)/len(patient_ids)*100:.1f}%)\")\n",
+    "print(f\"IDC patients with no GDC report  : {len(unmatched)}\")\n",
+    "print(\"Report-count distribution:\", dict(Counter(len(v) for v in case_to_reports.values())))\n",
+    "\n",
+    "sample_case   = next(iter(matched))\n",
+    "sample_report = case_to_reports[sample_case][0]\n",
+    "print(f\"\\nExample — {sample_case}:\")\n",
+    "print(f\"  file_id  : {sample_report['file_id']}\")\n",
+    "print(f\"  file_name: {sample_report['file_name']}\")\n",
+    "print(f\"  file_size: {sample_report['file_size']:,} bytes\")\n",
+    "print(f\"  download : {GDC_DATA_URL}/{sample_report['file_id']}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Download Reports\n",
+    "\n",
+    "Files are downloaded one at a time with `GET /data/{file_id}`.  \n",
+    "GDC does not require authentication for open-access files.\n",
+    "\n",
+    "**Bulk download** of many files is faster with the [GDC Data Transfer Tool](https://gdc.cancer.gov/access-data/gdc-data-transfer-tool) (`gdc-client`), which supports parallel downloads from a manifest. The manifest format is shown in section 5."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def download_gdc_file(file_id: str, dest_path: Path, chunk_size: int = 1 << 20) -> Path:\n",
+    "    \"\"\"Stream a GDC file to disk. Returns the destination path.\"\"\"\n",
+    "    dest_path.parent.mkdir(parents=True, exist_ok=True)\n",
+    "    with requests.get(f\"{GDC_DATA_URL}/{file_id}\", stream=True, timeout=60) as r:\n",
+    "        r.raise_for_status()\n",
+    "        with open(dest_path, \"wb\") as f:\n",
+    "            for chunk in r.iter_content(chunk_size):\n",
+    "                f.write(chunk)\n",
+    "    return dest_path\n",
+    "\n",
+    "\n",
+    "def download_reports_for_collection(\n",
+    "    case_to_reports: dict,\n",
+    "    output_dir: str | Path,\n",
+    "    limit: int | None = None,\n",
+    "    delay_s: float = 0.2,\n",
+    ") -> list[Path]:\n",
+    "    \"\"\"\n",
+    "    Download pathology report PDFs.\n",
+    "\n",
+    "    Parameters\n",
+    "    ----------\n",
+    "    case_to_reports : dict mapping submitter_id → list of GDC file-metadata dicts\n",
+    "    output_dir      : destination directory; PDFs are saved as {submitter_id}/{file_name}\n",
+    "    limit           : max number of cases to download (None = all)\n",
+    "    delay_s         : polite pause between requests\n",
+    "    \"\"\"\n",
+    "    output_dir = Path(output_dir)\n",
+    "    downloaded = []\n",
+    "    cases = list(case_to_reports.items())\n",
+    "    if limit:\n",
+    "        cases = cases[:limit]\n",
+    "\n",
+    "    for case_id, file_list in cases:\n",
+    "        for meta in file_list:\n",
+    "            dest = output_dir / case_id / meta[\"file_name\"]\n",
+    "            if dest.exists():\n",
+    "                print(f\"  skip (exists): {dest.name}\")\n",
+    "                downloaded.append(dest)\n",
+    "                continue\n",
+    "            print(f\"  downloading {case_id} — {meta['file_name']} ({meta['file_size']:,} B)\")\n",
+    "            download_gdc_file(meta[\"file_id\"], dest)\n",
+    "            downloaded.append(dest)\n",
+    "            time.sleep(delay_s)\n",
+    "\n",
+    "    return downloaded\n",
+    "\n",
+    "\n",
+    "# Download a small sample (3 cases)\n",
+    "paths = download_reports_for_collection(\n",
+    "    case_to_reports,\n",
+    "    output_dir=f\"/tmp/gdc_pathology/{COLLECTION}\",\n",
+    "    limit=3,\n",
+    ")\n",
+    "for p in paths:\n",
+    "    print(p, f\"({p.stat().st_size:,} bytes)\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5. Generate a GDC Download Manifest\n",
+    "\n",
+    "For large downloads the [GDC Data Transfer Tool](https://gdc.cancer.gov/access-data/gdc-data-transfer-tool) is more efficient.  \n",
+    "A manifest is a TSV with columns `id`, `filename`, `md5`, `size`, `state`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def make_gdc_manifest(reports: list[dict]) -> str:\n",
+    "    \"\"\"Return manifest TSV string suitable for: gdc-client download -m manifest.txt\"\"\"\n",
+    "    buf = io.StringIO()\n",
+    "    writer = csv.writer(buf, delimiter=\"\\t\")\n",
+    "    writer.writerow([\"id\", \"filename\", \"md5\", \"size\", \"state\"])\n",
+    "    for r in reports:\n",
+    "        writer.writerow([\n",
+    "            r[\"file_id\"],\n",
+    "            r[\"file_name\"],\n",
+    "            r.get(\"md5sum\", \"\"),\n",
+    "            r.get(\"file_size\", \"\"),\n",
+    "            \"released\",\n",
+    "        ])\n",
+    "    return buf.getvalue()\n",
+    "\n",
+    "\n",
+    "manifest = make_gdc_manifest(reports)\n",
+    "manifest_path = Path(f\"/tmp/gdc_pathology/{COLLECTION}_manifest.txt\")\n",
+    "manifest_path.parent.mkdir(parents=True, exist_ok=True)\n",
+    "manifest_path.write_text(manifest)\n",
+    "\n",
+    "print(f\"Manifest written: {manifest_path} ({len(reports)} files)\")\n",
+    "print(\"\\n\".join(manifest.splitlines()[:4]))\n",
+    "print()\n",
+    "print(f\"# gdc-client download -m {manifest_path} -d /path/to/output\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Text Extraction\n",
+    "\n",
+    "All GDC pathology report PDFs contain a **text layer** — no OCR is required. `fitz` (PyMuPDF) extracts it reliably.\n",
+    "\n",
+    "### Document types\n",
+    "\n",
+    "Two distinct types exist in the corpus with very different extraction quality:\n",
+    "\n",
+    "| Type | How to detect | Typical size | Usable content |\n",
+    "|------|--------------|-------------|----------------|\n",
+    "| **Pathology report** | Contains diagnosis narrative | 400 – 10,000+ chars | High — structured prose with TNM, grade, margin, LVI |\n",
+    "| **Missing Report Form** | `'Missing Pathology Report Form'` in text | ~1,400 chars | Low — form template is clean; handwritten field values are absent or garbled |\n",
+    "\n",
+    "### PDF structure notes\n",
+    "\n",
+    "- Many reports embed a full-page scanned image (3,400 × 4,400 px ≈ 400 DPI letter). The text layer is either the original digital source or OCR placed over the scan.\n",
+    "- A **TCGA QA checklist footer** appears on almost every report (Diagnosis Discrepancy, HIPAA Discrepancy, Prior Malignancy History, Dual/Synchronous checkboxes). Checkbox values in that section are unreliable — rendered as barcode-like strings.\n",
+    "- Core diagnostic narrative text is clean in the vast majority of reports.\n",
+    "\n",
+    "### Library comparison\n",
+    "\n",
+    "All four libraries (`fitz`, `pypdf`, `pdfminer.six`, `pdfplumber`) extract equivalent content from these files. `fitz` is ~3× faster; `pdfminer.six` adds slightly cleaner inter-line whitespace. `fitz` is recommended."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def extract_text(pdf_path: Path) -> str:\n",
+    "    \"\"\"Extract full text from a PDF using PyMuPDF.\"\"\"\n",
+    "    doc = fitz.open(pdf_path)\n",
+    "    text = \"\\n\".join(page.get_text() for page in doc)\n",
+    "    doc.close()\n",
+    "    return text\n",
+    "\n",
+    "\n",
+    "def classify_report(text: str) -> str:\n",
+    "    \"\"\"Return 'missing_form' or 'pathology_report'.\"\"\"\n",
+    "    return \"missing_form\" if \"Missing Pathology Report Form\" in text else \"pathology_report\"\n",
+    "\n",
+    "\n",
+    "# Demo on the 3 downloaded reports\n",
+    "for p in sorted(Path(f\"/tmp/gdc_pathology/{COLLECTION}\").rglob(\"*.PDF\")):\n",
+    "    text = extract_text(p)\n",
+    "    doc = fitz.open(p)\n",
+    "    pages  = len(doc)\n",
+    "    images = sum(len(pg.get_images()) for pg in doc)\n",
+    "    doc.close()\n",
+    "    print(f\"{p.parent.name}  [{classify_report(text)}]  \"\n",
+    "          f\"pages={pages}  images={images}  chars={len(text):,}\")\n",
+    "    print(text[:300].strip())\n",
+    "    print()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Structured field extraction\n",
+    "\n",
+    "For pathology reports the text is clean enough for regex-based extraction of key staging fields.\n",
+    "For production use, consider [scispaCy](https://allenai.github.io/scispacy/) or an LLM for more robust entity recognition."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def extract_staging_fields(text: str) -> dict:\n",
+    "    \"\"\"Best-effort regex extraction of common pathology staging fields.\"\"\"\n",
+    "    fields = {}\n",
+    "\n",
+    "    # TNM — pT*, pN*, pM* (including combined forms like pT2pN0)\n",
+    "    tnm = re.findall(r'p[TtNnMm]\\d[\\w]*', text)\n",
+    "    if tnm:\n",
+    "        fields[\"tnm\"] = list(dict.fromkeys(tnm))  # deduplicated, order preserved\n",
+    "\n",
+    "    # Histologic grade — G1–G4 or 'grade N' or 'SBR grade N'\n",
+    "    grade = re.findall(r'\\bG\\s*[1-4]\\b|[Gg]rade\\s+[1-4]|SBR\\s+grade\\s+\\d', text)\n",
+    "    if grade:\n",
+    "        fields[\"grade\"] = list(dict.fromkeys(grade))\n",
+    "\n",
+    "    # Tumor size — e.g. '2.1 cm', '35 mm'\n",
+    "    sizes = re.findall(r'\\b(\\d+\\.?\\d*)\\s*(cm|mm)\\b', text, re.IGNORECASE)\n",
+    "    if sizes:\n",
+    "        fields[\"size_mentions\"] = [f\"{v} {u}\" for v, u in sizes[:5]]\n",
+    "\n",
+    "    # Lymph node summary — 'X of Y lymph nodes'\n",
+    "    ln = re.findall(\n",
+    "        r'(\\d+)\\s*(?:of|/)\\s*(\\d+)\\s*(?:lymph\\s*nodes?|axillary)', text, re.IGNORECASE\n",
+    "    )\n",
+    "    if ln:\n",
+    "        fields[\"lymph_nodes_pos_total\"] = [f\"{pos}/{total}\" for pos, total in ln]\n",
+    "\n",
+    "    # Margin status\n",
+    "    if re.search(r'margin[s]?\\s*(?:are\\s*)?(?:clear|negative|free|tumor.free)', text, re.IGNORECASE):\n",
+    "        fields[\"margins\"] = \"negative\"\n",
+    "    elif re.search(r'margin[s]?\\s*(?:are\\s*)?positive|tumor\\s+at\\s+margin', text, re.IGNORECASE):\n",
+    "        fields[\"margins\"] = \"positive\"\n",
+    "\n",
+    "    # Vascular / lymphatic invasion\n",
+    "    vi = re.search(\n",
+    "        r'(?:vascular|lymphatic|lymphovascular)\\s+invasion[:\\s]+(present|not\\s+present|absent)',\n",
+    "        text, re.IGNORECASE\n",
+    "    )\n",
+    "    if vi:\n",
+    "        fields[\"vascular_invasion\"] = vi.group(1).lower()\n",
+    "\n",
+    "    return fields\n",
+    "\n",
+    "\n",
+    "for p in sorted(Path(f\"/tmp/gdc_pathology/{COLLECTION}\").rglob(\"*.PDF\")):\n",
+    "    text = extract_text(p)\n",
+    "    if classify_report(text) == \"missing_form\":\n",
+    "        print(f\"{p.parent.name}  [missing_form — skipped]\")\n",
+    "        continue\n",
+    "    fields = extract_staging_fields(text)\n",
+    "    print(f\"{p.parent.name}\")\n",
+    "    for k, v in fields.items():\n",
+    "        print(f\"  {k:25s}: {v}\")\n",
+    "    print()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 7. Multi-Collection Survey\n",
+    "\n",
+    "Check GDC report availability across all TCGA collections present in IDC."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tcga_collections = client.sql_query(\"\"\"\n",
+    "    SELECT collection_id, COUNT(DISTINCT PatientID) AS idc_patients\n",
+    "    FROM index\n",
+    "    WHERE collection_id LIKE 'tcga_%'\n",
+    "    GROUP BY collection_id\n",
+    "    ORDER BY idc_patients DESC\n",
+    "\"\"\")\n",
+    "\n",
+    "rows = []\n",
+    "for _, row in tcga_collections.iterrows():\n",
+    "    coll_id    = row[\"collection_id\"]\n",
+    "    gdc_project = coll_id.replace(\"tcga_\", \"TCGA-\").upper()\n",
+    "    n_idc      = int(row[\"idc_patients\"])\n",
+    "    payload = {\n",
+    "        \"filters\": json.dumps({\"op\": \"and\", \"content\": [\n",
+    "            {\"op\": \"=\", \"content\": {\"field\": \"cases.project.project_id\", \"value\": gdc_project}},\n",
+    "            {\"op\": \"=\", \"content\": {\"field\": \"data_type\", \"value\": \"Pathology Report\"}},\n",
+    "        ]}),\n",
+    "        \"fields\": \"file_id\", \"size\": \"1\", \"format\": \"JSON\",\n",
+    "    }\n",
+    "    r = requests.get(GDC_FILES_URL, params=payload, timeout=30)\n",
+    "    total_gdc = r.json()[\"data\"][\"pagination\"][\"total\"]\n",
+    "    rows.append({\"collection_id\": coll_id, \"idc_patients\": n_idc, \"gdc_reports\": total_gdc})\n",
+    "    time.sleep(0.1)\n",
+    "\n",
+    "survey = pd.DataFrame(rows)\n",
+    "survey[\"coverage_pct\"] = (survey[\"gdc_reports\"] / survey[\"idc_patients\"] * 100).round(1)\n",
+    "print(survey.to_string(index=False))"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.11.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}