From bf25bda7100de4e1c774204c36fe49d21f9a2a16 Mon Sep 17 00:00:00 2001 From: Manushi Majumdar Date: Wed, 1 Jul 2026 11:31:50 -0400 Subject: [PATCH 1/2] Guide for AI utility services added in 2.4.3 --- .../introduction_to_ai_utility_services.ipynb | 1167 +++++++++++++++++ 1 file changed, 1167 insertions(+) create mode 100644 guide/18-working-with-AI-capabilities/introduction_to_ai_utility_services.ipynb diff --git a/guide/18-working-with-AI-capabilities/introduction_to_ai_utility_services.ipynb b/guide/18-working-with-AI-capabilities/introduction_to_ai_utility_services.ipynb new file mode 100644 index 0000000000..7767af1684 --- /dev/null +++ b/guide/18-working-with-AI-capabilities/introduction_to_ai_utility_services.ipynb @@ -0,0 +1,1167 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "b1fdca13-eb69-4584-82f7-016dd9e622a2", + "metadata": {}, + "source": [ + "## Introduction to AI Utility Services\n", + "\n", + "ArcGIS Online comes equipped with AI Utility services, supported in the ArcGIS API for Python via the [`arcgis.ai`](https://developers.arcgis.com/python/latest/api-reference/arcgis.ai.html) module. It is designed to extract valuable information, and as an extension, help automate tasks that involve unstructured content — text, images, and documents — rather than traditional GIS layers.\n", + "\n", + "The API supports three services or methods:\n", + "1. [`analyze_image()`](https://developers.arcgis.com/python/latest/api-reference/arcgis.ai.html#analyze-image) - Analyzes images based on the user provided instructions.\n", + "2. [`analyze_text()`](https://developers.arcgis.com/python/latest/api-reference/arcgis.ai.html#analyze-text) - Analyzes provided text against the specified context.\n", + "3. [`translate()`](https://developers.arcgis.com/python/latest/api-reference/arcgis.ai.html#translate) - Translates language strings.\n", + "\n", + "We will explore these methods through a generic example, as well as examples involving ArcGIS items to see potential uses of these AI services applicable to our ArcGIS data.\n", + "\n", + "> Note: Using these services consumes credits:\n", + "> 1. `analyze_image()` - __5 images per credit__\n", + "> 2. `analyze_text()` - __5,000 characters per credit__\n", + "> 3. `translate()` - __1,000 characters per credit__" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "17d8b94b-d89b-477d-baa9-133ae71c6943", + "metadata": {}, + "outputs": [], + "source": [ + "from arcgis.gis import GIS\n", + "from arcgis.ai import analyze_image, analyze_text, translate, AIUtilsResponse" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "36fe7886-0c9d-47dc-897d-3f1b0a344463", + "metadata": {}, + "outputs": [], + "source": [ + "gis = GIS(profile='my_online_admin_profile')" + ] + }, + { + "cell_type": "markdown", + "id": "3258c4f5-f5fe-4d3f-89bd-23459d1e64f7", + "metadata": {}, + "source": [ + "### __Analyze Image__\n", + "\n", + "This tool helps analyze images using AI-powered models for object detection, text extraction, image description, and segmentation. Currently supported image formats are: JPEG, PNG and WEBP.\n", + "\n", + "> Note: The method takes image url as input and Only images hosted on *.arcgis.com domains are supported via url input. \n", + "\n", + "\n", + "Let us start with this image hosted in ArcGIS Online." + ] + }, + { + "cell_type": "markdown", + "id": "d76c1db3-2237-45f9-bf6b-ed72a2782dba", + "metadata": {}, + "source": [ + "![image](https://github.com/user-attachments/assets/e94eaad2-d6e6-4b82-b726-93b312167d95)" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "id": "9ae25c09-2957-44e3-b9bf-f4c801377b0f", + "metadata": {}, + "outputs": [], + "source": [ + "image_url = \"https://geosaurus.maps.arcgis.com/sharing/rest/content/items/6e9ff93df28a4231b4bd6e729ce42898/data\"" + ] + }, + { + "cell_type": "markdown", + "id": "b7dc71ee-6c1f-4a44-a1ee-768721f180f2", + "metadata": {}, + "source": [ + "Prompts to the `analyze_image()` method may be provided as string, list or dictionary inputs. We will see examples for all 3 below. " + ] + }, + { + "cell_type": "markdown", + "id": "f3a09af8-2751-419d-8717-ca21f9d6890e", + "metadata": {}, + "source": [ + "#### 1. Prompt as `str` input" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "16dbdf69-e1cd-4b3c-acd8-43342de3bca5", + "metadata": {}, + "outputs": [], + "source": [ + "prompt_str = \"List all the colors present in the image.\"" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "d26f9264-5ab2-4ecd-8913-32a953fa92cd", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIUtilsResponse(success=True, meta=AIUtilsResponseMetaData(usage={'inputTokens': 1831, 'outputTokens': 62, 'totalTokens': 1893}, model='gpt-5.4'), results=[AnalyzeResult(success=True, value=['green', 'dark green', 'light green', 'brown', 'dark brown', 'orange', 'yellow', 'red', 'pink', 'purple', 'blue', 'black', 'gray', 'white'], key='unique_string_1')])" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "colors = analyze_image(image = image_url, prompt = prompt_str)\n", + "colors" + ] + }, + { + "cell_type": "markdown", + "id": "c913fa32-9d71-4b1f-8a63-9aa189d60204", + "metadata": {}, + "source": [ + "As we will continue to see through this guide, the results generated from running these tools in the ArcGIS API for Python are of [`AIUtilsResponse`](https://developers.arcgis.com/python/latest/api-reference/arcgis.ai.html#arcgis.ai.AIUtilsResponse) type. \n", + "\n", + "You can verify if the request was successful using the `success` field of the response as shown below. " + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "a022d63d-f1bd-47e3-b8f7-17fd8627d6df", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "colors.success" + ] + }, + { + "cell_type": "markdown", + "id": "bc82764b-5614-452e-bf6e-020a5184e242", + "metadata": {}, + "source": [ + "You can access the specific results of the response as shown below." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "7bf338af-8e3a-457f-8539-f95a50bb38bc", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['green',\n", + " 'dark green',\n", + " 'light green',\n", + " 'brown',\n", + " 'dark brown',\n", + " 'orange',\n", + " 'yellow',\n", + " 'red',\n", + " 'pink',\n", + " 'purple',\n", + " 'blue',\n", + " 'black',\n", + " 'gray',\n", + " 'white']" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "colors.results[0].value" + ] + }, + { + "cell_type": "markdown", + "id": "f7c09fc8-84cb-4f48-853e-21ef26a10204", + "metadata": {}, + "source": [ + "The service has correctly identified the colors in this image.\n", + "\n", + "We will now proceed to see examples involving the other input types through an ArcGIS item. This is a [map image](https://www.arcgis.com/home/item.html?id=36c34ac9e2c5402ab94762f97f3349af#overview) representing the hurricane history for Virginia. " + ] + }, + { + "cell_type": "markdown", + "id": "e22cf3ba-c198-4a86-8a48-343f9a9dfa76", + "metadata": {}, + "source": [ + "![virginia_map](https://geosaurus.maps.arcgis.com/sharing/rest/content/items/36c34ac9e2c5402ab94762f97f3349af/data)" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "id": "1dcdeb90-54db-4080-ab89-a6910740c5a8", + "metadata": {}, + "outputs": [], + "source": [ + "virginia_map_url = \"https://geosaurus.maps.arcgis.com/sharing/rest/content/items/36c34ac9e2c5402ab94762f97f3349af/data\"" + ] + }, + { + "cell_type": "markdown", + "id": "339ed176-0dbe-44f4-96f6-7623a9a0cef2", + "metadata": {}, + "source": [ + "#### 2. Prompt as `list` input" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "id": "4e647c79-771b-49b6-ad2f-c9035805baec", + "metadata": {}, + "outputs": [], + "source": [ + "prompt_list = [\n", + " \"Describe the image.\",\n", + " \"Identify any hurricane related text in the image.\"\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "id": "43ec5bdf-c5d8-45b7-977b-94ac67484347", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIUtilsResponse(success=True, meta=AIUtilsResponseMetaData(usage={'inputTokens': 1822, 'outputTokens': 448, 'totalTokens': 2270}, model='gpt-5.4'), results=[AnalyzeResult(success=True, value='A grayscale map of Virginia showing hurricane, tropical storm, and tropical depression tracks across the state and nearby Atlantic coast. The title reads \"Virginia Hurricanes, Tropical Storms, and Tropical Depressions by Date 1852-2016, 2017-2021.\" Many storm paths are drawn in light blue for 1852-2016 and in orange/red shades for 2017-2021. County boundaries, neighboring states, a mileage scale, north arrow, legend, and source information are also shown.', key='unique_string_0'), AnalyzeResult(success=True, value=['Virginia Hurricanes, Tropical Storms, and Tropical Depressions by Date 1852-2016, 2017-2021', 'Hurricane Category 2, 2017-2021', 'Tropical Storm, 2017-2021', 'Tropical Depression 2017-2021', 'Hurricane Category 4, 1852-2016', 'Hurricane Category 3, 1852-2016', 'Hurricane Category 2, 1852-2016', 'Hurricane Category 1, 1852-2016', 'Tropical Storm, 1852-2016', 'Tropical Depression, 1852-2016', 'FLORENCE, 2018', 'IDA, 2021', 'ZETA, 2020', 'BERTHA, 2020', 'MICHAEL, 2018', 'ELSA, 2021', 'ISAIAS, 2020', 'FAY, 2020', 'CLAUDETTE, 2021', 'DORIAN, 2019', 'FRED, 2021'], key='unique_string_1')])" + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result_list = analyze_image(image = virginia_map_url, prompt = prompt_list)\n", + "result_list" + ] + }, + { + "cell_type": "markdown", + "id": "345e81cb-8b3f-48a9-beae-691a8c7d487c", + "metadata": {}, + "source": [ + "We can access results for the individual prompts via the prompt index provided to the `results` field as demonstrated below." + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "id": "e3779890-7eb3-405b-8c4e-626289fe9615", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'A grayscale map of Virginia showing hurricane, tropical storm, and tropical depression tracks across the state and nearby Atlantic coast. The title reads \"Virginia Hurricanes, Tropical Storms, and Tropical Depressions by Date 1852-2016, 2017-2021.\" Many storm paths are drawn in light blue for 1852-2016 and in orange/red shades for 2017-2021. County boundaries, neighboring states, a mileage scale, north arrow, legend, and source information are also shown.'" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result_list.results[0].value" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "id": "126503e6-217a-481b-87d6-b2a541d3e609", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['Virginia Hurricanes, Tropical Storms, and Tropical Depressions by Date 1852-2016, 2017-2021',\n", + " 'Hurricane Category 2, 2017-2021',\n", + " 'Tropical Storm, 2017-2021',\n", + " 'Tropical Depression 2017-2021',\n", + " 'Hurricane Category 4, 1852-2016',\n", + " 'Hurricane Category 3, 1852-2016',\n", + " 'Hurricane Category 2, 1852-2016',\n", + " 'Hurricane Category 1, 1852-2016',\n", + " 'Tropical Storm, 1852-2016',\n", + " 'Tropical Depression, 1852-2016',\n", + " 'FLORENCE, 2018',\n", + " 'IDA, 2021',\n", + " 'ZETA, 2020',\n", + " 'BERTHA, 2020',\n", + " 'MICHAEL, 2018',\n", + " 'ELSA, 2021',\n", + " 'ISAIAS, 2020',\n", + " 'FAY, 2020',\n", + " 'CLAUDETTE, 2021',\n", + " 'DORIAN, 2019',\n", + " 'FRED, 2021']" + ] + }, + "execution_count": 47, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result_list.results[1].value" + ] + }, + { + "cell_type": "markdown", + "id": "8f70d85b-c2e6-46df-9e1d-e3b6c869ac09", + "metadata": {}, + "source": [ + "The response provides a succinct summary of this map and extracts key information regarding hurricanes referenced. " + ] + }, + { + "cell_type": "markdown", + "id": "5dc8a7e9-33a3-4684-a2f4-e463ff340556", + "metadata": {}, + "source": [ + "#### 3. Prompt as `dict` input " + ] + }, + { + "cell_type": "markdown", + "id": "d0067f71-e2e1-415f-a686-01e31effd616", + "metadata": {}, + "source": [ + "In order to provide prompts as a dictionary, it is essential to provide the values for the `key` and `context` keys of the dictionary for each prompt that you wish to provide. " + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "id": "bc6d62b5-2780-46e7-8228-c200ffd708cf", + "metadata": {}, + "outputs": [], + "source": [ + "prompt_dicts = [\n", + " {\"key\": \"images_prompt\", \"context\": \"How many sub-boundaries of Virginia are included in the image?\"},\n", + " {\"key\": \"text_prompt\", \"context\": \"Identify the states named in the map.\"}\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "id": "f94354b5-5251-4edb-bd10-a8927bba9873", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIUtilsResponse(success=True, meta=AIUtilsResponseMetaData(usage={'inputTokens': 1826, 'outputTokens': 117, 'totalTokens': 1943}, model='gpt-5.4'), results=[AnalyzeResult(success=True, value=95, key='images_prompt'), AnalyzeResult(success=True, value=['Ohio', 'Pennsylvania', 'Maryland', 'Delaware', 'Kentucky', 'Tennessee', 'North Carolina', 'Virginia'], key='text_prompt')])" + ] + }, + "execution_count": 41, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result_dicts = analyze_image(image = virginia_map_url, prompt = prompt_dicts)\n", + "result_dicts" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "id": "bda129d9-807a-4d34-9a73-57f677a77ed1", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "95" + ] + }, + "execution_count": 42, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result_dicts.results[0].value" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "id": "95980b7f-7a17-437c-8a0d-b0c00d751fc8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['Ohio',\n", + " 'Pennsylvania',\n", + " 'Maryland',\n", + " 'Delaware',\n", + " 'Kentucky',\n", + " 'Tennessee',\n", + " 'North Carolina',\n", + " 'Virginia']" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "dicts_result.results[1].value" + ] + }, + { + "cell_type": "markdown", + "id": "c1395c40-3097-4a9a-9d20-2635ee53a9c1", + "metadata": {}, + "source": [ + "The tool here currently identifies 95 sub-boundaries (counties) for Virginia, and detects all the states mentioned in the map image. " + ] + }, + { + "cell_type": "markdown", + "id": "efc61dca-97f4-4e72-bcbe-63a6e618f48a", + "metadata": {}, + "source": [ + "### __Analyze Text__\n", + "\n", + "This tool analyzes the provided text against the given prompt(s) using AI capabilities. Users are allowed to submit text for analysis, which can include tasks such as sentiment analysis, entity recognition, and other natural language processing operations.\n", + "\n", + "Similar to the `analyze_image()` method, prompts to the `analyze_text()` method may be provided as string, list or dictionary inputs. We will see examples for all 3 input types below. " + ] + }, + { + "cell_type": "markdown", + "id": "a568c69f-fdc2-4b3f-959b-528bbd8d16d4", + "metadata": {}, + "source": [ + "#### 1. Prompt as `str` input" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "id": "0235aef9-9bb7-416f-ba64-88516e53e7ff", + "metadata": {}, + "outputs": [], + "source": [ + "sample_text = \"During our recent travels, we explored bustling city centers like London and Prague, \\\n", + " wandered through the ancient, narrow streets of Rome, and relaxed on the sunny beaches of \\\n", + " Alanya near the historic Alanya Castle. We visited the majestic Taj Mahal in Agra, toured the \\\n", + " expansive Amber Palace in Jaipur, and took a relaxing break by the calm waters of the Dim River.\"\n", + "\n", + "prompt_str = \"Detect all locations in this text\"" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "id": "6f14b6d6-3ec0-4f1e-b99b-7865addc5eb3", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIUtilsResponse(success=True, meta=AIUtilsResponseMetaData(usage={'inputTokens': 750, 'outputTokens': 55, 'totalTokens': 805}, model='gpt-5.4'), results=[AnalyzeResult(success=True, value=['London', 'Prague', 'Rome', 'Alanya', 'Alanya Castle', 'Taj Mahal', 'Agra', 'Amber Palace', 'Jaipur', 'Dim River'], key='key:1')])" + ] + }, + "execution_count": 50, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "locations = analyze_text(text = sample_text, prompt = prompt_str)\n", + "locations" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "id": "c69102c3-84a0-4587-b406-41ff70fd7050", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['London',\n", + " 'Prague',\n", + " 'Rome',\n", + " 'Alanya',\n", + " 'Alanya Castle',\n", + " 'Taj Mahal',\n", + " 'Agra',\n", + " 'Amber Palace',\n", + " 'Jaipur',\n", + " 'Dim River']" + ] + }, + "execution_count": 51, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "locations.results[0].value" + ] + }, + { + "cell_type": "markdown", + "id": "41532a46-9e05-43c6-802b-e3ac73720b80", + "metadata": {}, + "source": [ + "#### 2. Prompt as `list` input" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "id": "f754e903-f258-4dff-8a07-ebc3be38296a", + "metadata": {}, + "outputs": [], + "source": [ + "prompt_list = [\n", + " \"Detect the sentiment of the text.\",\n", + " \"Identify countries in the text.\"\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "id": "53a34b53-6881-41fc-91ab-f98f69f5bf35", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIUtilsResponse(success=True, meta=AIUtilsResponseMetaData(usage={'inputTokens': 768, 'outputTokens': 54, 'totalTokens': 822}, model='gpt-5.4'), results=[AnalyzeResult(success=True, value='positive', key='unique_string_0'), AnalyzeResult(success=True, value=['United Kingdom', 'Czech Republic', 'Italy', 'Turkey', 'India'], key='unique_string_1')])" + ] + }, + "execution_count": 61, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result_list = analyze_text(text = sample_text, prompt = prompt_list)\n", + "result_list" + ] + }, + { + "cell_type": "markdown", + "id": "c930517e-7a48-4b0e-8a4d-05894ab0bf59", + "metadata": {}, + "source": [ + "We can access results for the individual prompts via the prompt index provided to the `results` field as demonstrated below." + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "id": "fea6b707-b448-4e18-b725-6bf782f82743", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'positive'" + ] + }, + "execution_count": 62, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result_list.results[0].value" + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "id": "0275cf94-021d-4d95-913c-74bd9accc9d5", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['United Kingdom', 'Czech Republic', 'Italy', 'Turkey', 'India']" + ] + }, + "execution_count": 63, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result_list.results[1].value" + ] + }, + { + "cell_type": "markdown", + "id": "bbed4555-ab32-413e-8f6b-0d7c9cf2faec", + "metadata": {}, + "source": [ + "#### 3. Prompt as `dict` input\n", + "\n", + "We will now look at an example that uses a dictionary of prompt inputs using an ArcGIS item. This is a [document](https://www.arcgis.com/home/item.html?id=c7b4e9240149481ea69af628513962fe#overview) about Esri's Water Resources. Let us learn more about this item. " + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "id": "50fedaa5-909b-43c6-aec6-8ab7276341b8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "
\n", + " \n", + " \n", + " \n", + "
\n", + "\n", + "
\n", + " Esri - Water Resources\n", + " \n", + "
Esri's Water Resources GIS Platform provides tools and solutions for managing and analyzing water data to improve decision-making across sectors like watershed management, stormwater planning, and natural resource conservation. The platform supports data modeling, monitoring, and collaboration, offering resources like Arc Hydro, Living Atlas water datasets, and a vibrant Hydro Community. By leveraging geospatial technology, users can harness big data, simulate environmental impacts, and facilitate stakeholder engagement, enabling sustainable water resource management.
Document Link by NSGIC_Dev1\n", + "
Last Modified: January 06, 2025\n", + "
0 comments, 506 views\n", + "
\n", + "
\n", + " " + ], + "text/plain": [ + "" + ] + }, + "execution_count": 66, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "document = gis.content.get('c7b4e9240149481ea69af628513962fe')\n", + "document" + ] + }, + { + "cell_type": "markdown", + "id": "22d7dab1-11e3-4172-a0b8-b563d87bea28", + "metadata": {}, + "source": [ + "We extract the description of this item using the `description` property of the `Item` and pass it as the text input to our method. " + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "id": "b5369419-361c-4d94-a8d6-4f95ff882489", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\"
Esri's Water Resources GIS Platform offers a comprehensive suite of tools and resources designed to modernize water resource management. It emphasizes geospatial solutions for monitoring, analyzing, and modeling water systems, helping decision-makers tackle challenges like drought resilience, flood mitigation, and environmental protection. By leveraging the capabilities of ArcGIS, users can transform raw water data into actionable insights, ensuring more efficient and effective water resource management.

A central feature of the platform is Arc Hydro, a specialized data model and toolkit developed for GIS-based water resource analysis. This toolset allows users to integrate, analyze, and visualize water datasets for applications ranging from live stream gauge monitoring to pollution control. Additionally, the platform connects users to the ArcGIS Living Atlas of the World, which offers extensive water-related datasets such as rivers, wetlands, and soils, supporting in-depth analyses of hydrologic conditions. The Hydro Community further enhances collaboration, enabling stakeholders to share expertise, discuss challenges, and build innovative solutions together.

Esri’s platform also provides training opportunities and professional services to empower users with technical knowledge and skills. Through instructor-led courses, documentation, and best practices, users gain expertise in using ArcGIS and Arc Hydro for their specific water management needs. The combination of tools, datasets, and community engagement makes Esri's water resources platform a powerful asset for advancing sustainable water management initiatives across public and private sectors.

\"" + ] + }, + "execution_count": 67, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "item_description = document.description\n", + "item_description" + ] + }, + { + "cell_type": "markdown", + "id": "d552feb7-3e1e-4211-ac19-e7d2647db490", + "metadata": {}, + "source": [ + "Similar to the `analyze_image()` method, in order to provide prompts as a dictionary it is essential to provide the values for the key and context keys of the dictionary for each prompt that you wish to provide." + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "id": "69fb9c2d-f440-48dd-967d-f65bd202addd", + "metadata": {}, + "outputs": [], + "source": [ + "prompt_dicts = [\n", + " {\"key\": \"summary_prompt\", \"context\": \"Summarize this text\"},\n", + " {\"key\": \"tools_prompt\", \"context\": \"Identify the tools mentioned in the text.\"}\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "id": "76042778-fe79-4c18-a3e2-878e77ffa995", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIUtilsResponse(success=True, meta=AIUtilsResponseMetaData(usage={'inputTokens': 997, 'outputTokens': 140, 'totalTokens': 1137}, model='gpt-5.4'), results=[AnalyzeResult(success=True, value=\"Esri's Water Resources GIS Platform provides geospatial tools, datasets, training, and community resources to improve water resource management. It uses ArcGIS capabilities to monitor, analyze, and model water systems for issues such as drought resilience, flood mitigation, and environmental protection. Key components include Arc Hydro for GIS-based water analysis, the ArcGIS Living Atlas of the World for water-related datasets, and the Hydro Community for collaboration and knowledge sharing.\", key='summary_prompt'), AnalyzeResult(success=True, value=['ArcGIS', 'Arc Hydro', 'ArcGIS Living Atlas of the World', 'Hydro Community'], key='tools_prompt')])" + ] + }, + "execution_count": 69, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result_dicts = analyze_text(text = item_description, prompt = prompt_dicts)\n", + "result_dicts" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "id": "451bc7c5-705d-47fc-9b17-2ab14a8f6fd6", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\"Esri's Water Resources GIS Platform provides geospatial tools, datasets, training, and community resources to improve water resource management. It uses ArcGIS capabilities to monitor, analyze, and model water systems for issues such as drought resilience, flood mitigation, and environmental protection. Key components include Arc Hydro for GIS-based water analysis, the ArcGIS Living Atlas of the World for water-related datasets, and the Hydro Community for collaboration and knowledge sharing.\"" + ] + }, + "execution_count": 70, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result_dicts.results[0].value" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "id": "4b8d061d-4180-4442-aa2d-049805f7d1aa", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['ArcGIS', 'Arc Hydro', 'ArcGIS Living Atlas of the World', 'Hydro Community']" + ] + }, + "execution_count": 71, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "result_dicts.results[1].value" + ] + }, + { + "cell_type": "markdown", + "id": "557525ee-d848-4c0c-9d59-7e00390616df", + "metadata": {}, + "source": [ + "The tool summarizes the description of this item well and also identifies the ArcGIS tools mentioned. " + ] + }, + { + "cell_type": "markdown", + "id": "459ee9ed-d2ee-4558-8566-a829eb2246eb", + "metadata": {}, + "source": [ + "### __Translate Text__\n", + "\n", + "The `translate()` method translates language strings passed to the desired array of output cultures. To identify the languages supported in your ArcGIS Online Organization and the `culture` of those languages, you may invoke the `languages` property of the `GIS` object as shown below." + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "id": "1e628904-6cc1-4118-bcef-1eccfb7e234b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'language': 'Arabic', 'culture': 'ar', 'localizedName': 'عربي'},\n", + " {'language': 'Bosnian', 'culture': 'bs', 'localizedName': 'Bosanski'},\n", + " {'language': 'Bulgarian', 'culture': 'bg', 'localizedName': 'Български'},\n", + " {'language': 'Simplified Chinese',\n", + " 'culture': 'zh-CN',\n", + " 'localizedName': '简体中文'},\n", + " {'language': 'Catalan', 'culture': 'ca', 'localizedName': 'Català'},\n", + " {'language': 'Croatian', 'culture': 'hr', 'localizedName': 'Hrvatski'},\n", + " {'language': 'Czech', 'culture': 'cs', 'localizedName': 'čeština'},\n", + " {'language': 'Danish', 'culture': 'da', 'localizedName': 'Dansk'},\n", + " {'language': 'Dutch', 'culture': 'nl', 'localizedName': 'Nederlands'},\n", + " {'language': 'English',\n", + " 'culture': 'en',\n", + " 'localizedName': 'English',\n", + " 'cultureFormats': [{'name': 'United States',\n", + " 'localizedName': 'United States',\n", + " 'format': 'us'},\n", + " {'name': 'Great Britain', 'localizedName': 'Great Britain', 'format': 'gb'},\n", + " {'name': 'Australia', 'localizedName': 'Australia', 'format': 'au'},\n", + " {'name': 'Canada', 'localizedName': 'Canada', 'format': 'ca'}]},\n", + " {'language': 'Estonian', 'culture': 'et', 'localizedName': 'Eesti'},\n", + " {'language': 'Finnish', 'culture': 'fi', 'localizedName': 'Suomi'},\n", + " {'language': 'French',\n", + " 'culture': 'fr',\n", + " 'localizedName': 'Français',\n", + " 'cultureFormats': [{'name': 'France',\n", + " 'localizedName': 'France',\n", + " 'format': 'fr'},\n", + " {'name': 'Switzerland', 'localizedName': 'Suisse', 'format': 'ch'},\n", + " {'name': 'Canada', 'localizedName': 'Canada', 'format': 'ca'}]},\n", + " {'language': 'Greek', 'culture': 'el', 'localizedName': 'Ελλάδα'},\n", + " {'language': 'German',\n", + " 'culture': 'de',\n", + " 'localizedName': 'Deutsch',\n", + " 'cultureFormats': [{'name': 'Germany',\n", + " 'localizedName': 'Deutschland',\n", + " 'format': 'de'},\n", + " {'name': 'Switzerland', 'localizedName': 'Schweiz', 'format': 'ch'},\n", + " {'name': 'Austria', 'localizedName': 'Österreich', 'format': 'at'}]},\n", + " {'language': 'Hebrew', 'culture': 'he', 'localizedName': 'עברית'},\n", + " {'language': 'Hungarian', 'culture': 'hu', 'localizedName': 'Magyar'},\n", + " {'language': 'Indonesian',\n", + " 'culture': 'id',\n", + " 'localizedName': 'Bahasa Indonesia'},\n", + " {'language': 'Italian',\n", + " 'culture': 'it',\n", + " 'localizedName': 'Italiano',\n", + " 'cultureFormats': [{'name': 'Italy',\n", + " 'localizedName': 'Italia',\n", + " 'format': 'it'},\n", + " {'name': 'Switzerland', 'localizedName': 'Svizzera', 'format': 'ch'}]},\n", + " {'language': 'Japanese', 'culture': 'ja', 'localizedName': '日本語'},\n", + " {'language': 'Korean', 'culture': 'ko', 'localizedName': '한국어'},\n", + " {'language': 'Latvian', 'culture': 'lv', 'localizedName': 'Latviešu'},\n", + " {'language': 'Lithuanian', 'culture': 'lt', 'localizedName': 'Lietuvių'},\n", + " {'language': 'Norwegian', 'culture': 'nb', 'localizedName': 'Norsk'},\n", + " {'language': 'Polish', 'culture': 'pl', 'localizedName': 'Polski'},\n", + " {'language': 'Portuguese (Brazil)',\n", + " 'culture': 'pt-BR',\n", + " 'localizedName': 'Português (Brasil)'},\n", + " {'language': 'Portuguese (Portugal)',\n", + " 'culture': 'pt-PT',\n", + " 'localizedName': 'Português (Portugal)'},\n", + " {'language': 'Romanian', 'culture': 'ro', 'localizedName': 'Română'},\n", + " {'language': 'Russian', 'culture': 'ru', 'localizedName': 'Русский'},\n", + " {'language': 'Serbian', 'culture': 'sr', 'localizedName': 'Srpski'},\n", + " {'language': 'Slovenian', 'culture': 'sl', 'localizedName': 'Slovenščina'},\n", + " {'language': 'Slovak', 'culture': 'sk', 'localizedName': 'Slovenčina'},\n", + " {'language': 'Spanish',\n", + " 'culture': 'es',\n", + " 'localizedName': 'Español',\n", + " 'cultureFormats': [{'name': 'Spain',\n", + " 'localizedName': 'España',\n", + " 'format': 'es'},\n", + " {'name': 'Mexico', 'localizedName': 'México', 'format': 'mx'}]},\n", + " {'language': 'Swedish', 'culture': 'sv', 'localizedName': 'Svenska'},\n", + " {'language': 'Thai', 'culture': 'th', 'localizedName': 'ไทย'},\n", + " {'language': 'Traditional Chinese (Hong Kong)',\n", + " 'culture': 'zh-HK',\n", + " 'localizedName': '繁體中文（香港）'},\n", + " {'language': 'Traditional Chinese (Taiwan)',\n", + " 'culture': 'zh-TW',\n", + " 'localizedName': '繁體中文（台灣）'},\n", + " {'language': 'Ukrainian', 'culture': 'uk', 'localizedName': 'Україна'},\n", + " {'language': 'Turkish', 'culture': 'tr', 'localizedName': 'Türkiye'},\n", + " {'language': 'Vietnamese', 'culture': 'vi', 'localizedName': 'Tiếng Việt'}]" + ] + }, + "execution_count": 73, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "gis.languages" + ] + }, + { + "cell_type": "markdown", + "id": "162fb0d9-e643-4ad8-b3b8-ad06e7aaaade", + "metadata": {}, + "source": [ + "We start with an example to translate a few lines in Spanish to English." + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "id": "ef427c9e-6f9d-44ec-95d3-38d2fa994071", + "metadata": {}, + "outputs": [], + "source": [ + "text = \"El bosque está lleno de vida. Mientras el sol brilla a través de las altas ramas, el viento sopla suavemente. Los \\\n", + " pájaros cantan en los árboles y las mariposas vuelan sobre las flores. Estar en la naturaleza nos da paz y tranquilidad.\"" + ] + }, + { + "cell_type": "markdown", + "id": "8da55e1f-2055-44e6-9fc0-0f7de417f271", + "metadata": {}, + "source": [ + "Note that the input to the `to_language` parameter in the `translate()` method accepts input as a `list`, to support multiple output languages at once. However, the `from_language` parameter accepts a `str` input. " + ] + }, + { + "cell_type": "code", + "execution_count": 75, + "id": "de312ee7-2a86-4d7e-af89-46325f6c6c48", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIUtilsResponse(success=True, meta=AIUtilsResponseMetaData(usage={'inputCharacters': 242}, model='azure-ai-translator'), results=[TranslateResult(translations=[{'text': 'The forest is full of life. As the sun shines through the tall branches, the wind blows gently. Birds sing in the trees and butterflies fly over the flowers. Being in nature gives us peace and tranquility.', 'to': 'en'}], key='key:1')])" + ] + }, + "execution_count": 75, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "spanish_to_english = translate(text = text, to_language = ['en'], from_language = 'es')\n", + "spanish_to_english" + ] + }, + { + "cell_type": "markdown", + "id": "cfea8971-3162-4acd-a782-7b2c247de120", + "metadata": {}, + "source": [ + "Unlike the `analyze_image()` and `analyze_text()` methods, the process to access the response (translated text) is slightly different, as shown below." + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "id": "26fab339-0b29-4afc-a9d7-2537228f016f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'The forest is full of life. As the sun shines through the tall branches, the wind blows gently. Birds sing in the trees and butterflies fly over the flowers. Being in nature gives us peace and tranquility.'" + ] + }, + "execution_count": 80, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "spanish_to_english.results[0].translations[0]['text']" + ] + }, + { + "cell_type": "markdown", + "id": "6e287c1f-1226-45e5-a217-b87fd4be06f9", + "metadata": {}, + "source": [ + "Let us conclude with an example using an ArcGIS item from Ontario, Canada in French. We fetch the item and access its `snippet` which we provide as input to the `translate()` method. " + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "id": "1d72fc50-f34b-4cce-8d2e-50b9aa960784", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "
\n", + " \n", + " \n", + " \n", + "
\n", + "\n", + "
\n", + " Données du rapport annuel historique sur les forêts de l’Ontario\n", + " \n", + "
Les données du rapport annuel historique sur les forêts de l’Ontario constituent un ensemble de données consolidées à l’échelle de la province, provenant des rapports annuels sur les opérations forestières depuis 2002. Format des données : ESRI Geodatabase
Document Link by LandInformationOntario\n", + "
Last Modified: March 20, 2026\n", + "
0 comments, 35 views\n", + "
\n", + "
\n", + " " + ], + "text/plain": [ + "" + ] + }, + "execution_count": 81, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "ontario_canada_item = gis.content.get('9c2e4d1a671f41eb913411f048aaf661')\n", + "ontario_canada_item" + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "id": "02be7f7c-152b-4561-a329-b13e68bbc334", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'Les données du rapport annuel historique sur les forêts de l’Ontario constituent un ensemble de données consolidées à l’échelle de la province, provenant des rapports annuels sur les opérations forestières depuis 2002. Format des données : ESRI Geodatabase'" + ] + }, + "execution_count": 82, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "item_snippet = ontario_canada_item.snippet\n", + "item_snippet" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "id": "a5784b86-27d2-4dd1-9b17-7e1191e8e503", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIUtilsResponse(success=True, meta=AIUtilsResponseMetaData(usage={'inputCharacters': 257}, model='azure-ai-translator'), results=[TranslateResult(translations=[{'text': 'The Ontario Historical Annual Forest Report is a province-wide consolidated dataset from annual reports on forest operations since 2002. Data Format: ESRI Geodatabase', 'to': 'en'}], key='key:1')])" + ] + }, + "execution_count": 83, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "french_to_english = translate(text = item_snippet, to_language = ['en'], from_language = 'fr')\n", + "french_to_english" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "id": "c95db6d6-ebb5-4d76-ae23-8877eafb0039", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'The Ontario Historical Annual Forest Report is a province-wide consolidated dataset from annual reports on forest operations since 2002. Data Format: ESRI Geodatabase'" + ] + }, + "execution_count": 85, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "french_to_english.results[0].translations[0]['text']" + ] + }, + { + "cell_type": "markdown", + "id": "2954c8c2-5a87-4eb3-accb-e7fe086ace5b", + "metadata": {}, + "source": [ + "We see that the tool has successfully translated the snippet from French to English." + ] + }, + { + "cell_type": "markdown", + "id": "477a9c36-4019-4691-b785-0100403f3dd5", + "metadata": {}, + "source": [ + "### Conclusion\n", + "\n", + "The `arcgis.ai` module is extremely useful when your workflows involve unstructured data in the form of field notes, inspection reports, public comments, customer requests, attached photos or scanned documents. Deriving necessary information from these data sources using these services supported within the ArcGIS API for Python can significantly improve productivity. \n", + "\n", + "To delve deeper into the `arcgis.ai` module with an example that involves real-world data, we encourage you to take a look at this [sample](https://developers.arcgis.com/python/latest/samples/ai-utility-services-enhancing-service-requests/). " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1b8a82a0-fb84-4d7e-95c2-e7c2428d61da", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.11" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From 1d1ca612ef93d7ea4f39080d96b9d47ee099ec67 Mon Sep 17 00:00:00 2001 From: Manushi Majumdar Date: Thu, 2 Jul 2026 11:16:36 -0400 Subject: [PATCH 2/2] addressed requested changes --- .../introduction_to_ai_utility_services.ipynb | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/guide/18-working-with-AI-capabilities/introduction_to_ai_utility_services.ipynb b/guide/18-working-with-AI-capabilities/introduction_to_ai_utility_services.ipynb index 7767af1684..87c8c7455c 100644 --- a/guide/18-working-with-AI-capabilities/introduction_to_ai_utility_services.ipynb +++ b/guide/18-working-with-AI-capabilities/introduction_to_ai_utility_services.ipynb @@ -5,9 +5,9 @@ "id": "b1fdca13-eb69-4584-82f7-016dd9e622a2", "metadata": {}, "source": [ - "## Introduction to AI Utility Services\n", + "## Introduction to AI-Powered services in ArcGIS Online\n", "\n", - "ArcGIS Online comes equipped with AI Utility services, supported in the ArcGIS API for Python via the [`arcgis.ai`](https://developers.arcgis.com/python/latest/api-reference/arcgis.ai.html) module. It is designed to extract valuable information, and as an extension, help automate tasks that involve unstructured content — text, images, and documents — rather than traditional GIS layers.\n", + "ArcGIS Online comes equipped with AI-powered services, supported in the ArcGIS API for Python via the [`arcgis.ai`](https://developers.arcgis.com/python/latest/api-reference/arcgis.ai.html) module. It is designed to extract valuable information and help automate tasks involving unstructured content, such as text, images, and documents.\n", "\n", "The API supports three services or methods:\n", "1. [`analyze_image()`](https://developers.arcgis.com/python/latest/api-reference/arcgis.ai.html#analyze-image) - Analyzes images based on the user provided instructions.\n", @@ -40,7 +40,7 @@ "metadata": {}, "outputs": [], "source": [ - "gis = GIS(profile='my_online_admin_profile')" + "gis = GIS(profile='your_online_profile')" ] }, { @@ -52,7 +52,7 @@ "\n", "This tool helps analyze images using AI-powered models for object detection, text extraction, image description, and segmentation. Currently supported image formats are: JPEG, PNG and WEBP.\n", "\n", - "> Note: The method takes image url as input and Only images hosted on *.arcgis.com domains are supported via url input. \n", + "> Note: Input image could be provided as an URL or Base64. Only images hosted on *.arcgis.com domains are supported via url input. \n", "\n", "\n", "Let us start with this image hosted in ArcGIS Online." @@ -160,7 +160,7 @@ "id": "bc82764b-5614-452e-bf6e-020a5184e242", "metadata": {}, "source": [ - "You can access the specific results of the response as shown below." + "Since `AIUtilsResponse` is a pydantic model, you can access the specific results of the response as shown below." ] }, { @@ -1129,7 +1129,7 @@ "source": [ "### Conclusion\n", "\n", - "The `arcgis.ai` module is extremely useful when your workflows involve unstructured data in the form of field notes, inspection reports, public comments, customer requests, attached photos or scanned documents. Deriving necessary information from these data sources using these services supported within the ArcGIS API for Python can significantly improve productivity. \n", + "The `arcgis.ai` module is useful when your workflows involve unstructured data in the form of field notes, inspection reports, public comments, customer requests, attached photos or scanned documents. Deriving necessary information from these data sources using these services supported within the ArcGIS API for Python can improve productivity. \n", "\n", "To delve deeper into the `arcgis.ai` module with an example that involves real-world data, we encourage you to take a look at this [sample](https://developers.arcgis.com/python/latest/samples/ai-utility-services-enhancing-service-requests/). " ]