Add Indic Multimodal Document Intelligence & Security Guardrails recipe

examples/Indic_Multimodal_Document_Intelligence.ipynb

@@ -0,0 +1,271 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 🇮🇳 Indic Multimodal Document Intelligence & Security Guardrails with Gemini 2.0 Flash\n",
+    "\n",
+    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/cookbook/blob/main/examples/Indic_Multimodal_Document_Intelligence.ipynb\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" height=30/></a>\n",
+    "\n",
+    "**Author:** Nandhakumar Murugan ([@nandhakumar-murugan](https://github.com/nandhakumar-murugan))  \n",
+    "**Role:** Google Student Ambassador & AI Engineer, KGiSL Institute of Technology  \n",
+    "**SDK:** Official Google GenAI SDK (`google-genai`)\n",
+    "\n",
+    "---\n",
+    "\n",
+    "## 📌 Overview\n",
+    "In multilingual regions across India and Southeast Asia, engineering teams and students frequently interact with complex technical architecture diagrams, system schematics, and cloud workflows. However, conceptual breakdowns are rarely accessible in native Indic vernacular languages.\n",
+    "\n",
+    "This recipe demonstrates how to use **Gemini 2.0 Flash** with the modern **Interactions API** (`client.interactions.create`) to:\n",
+    "1. 👁️ **Analyze Visual Documents**: Parse complex cloud architectures and system diagrams.\n",
+    "2. 🌐 **Generate Trilingual Knowledge Graphs**: Extract core technical concepts with high-fidelity explanations in **English**, **Tamil (தமிழ்)**, and **Hindi (हिन्दी)**.\n",
+    "3. 🛡️ **Enforce Security Guardrails**: Detect exposed credentials, architectural vulnerabilities, and compliance risks using Pydantic **Structured Outputs** (`response_schema`)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🛠️ Step 1: Install Dependencies\n",
+    "Use `%pip` with the required minimum SDK version as per the official style guide."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Install the official Google GenAI SDK and supporting libraries\n",
+    "%pip install -q -U 'google-genai>=2.9.0' pydantic pillow tabulate"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🔑 Step 2: Initialize the Gemini Client\n",
+    "Configure your `GEMINI_API_KEY` from [Google AI Studio](https://aistudio.google.com) and select the model using the Colab parameter selector."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "from google import genai\n",
+    "from google.genai import types\n",
+    "\n",
+    "# Retrieve API key from Colab Secrets or environment variable\n",
+    "try:\n",
+    "    from google.colab import userdata\n",
+    "    api_key = userdata.get('GEMINI_API_KEY')\n",
+    "except Exception:\n",
+    "    api_key = os.environ.get('GEMINI_API_KEY')\n",
+    "\n",
+    "if not api_key:\n",
+    "    api_key = input(\"Enter your Google Gemini API Key from aistudio.google.com: \").strip()\n",
+    "\n",
+    "client = genai.Client(api_key=api_key)\n",
+    "MODEL_ID = \"gemini-2.0-flash\"  # @param [\"gemini-2.0-flash\", \"gemini-2.5-flash\"] {\"allow-input\":true, \"isTemplate\": true}\n",
+    "print(f\"✅ Gemini Client initialized successfully with {MODEL_ID}!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 📐 Step 3: Define Pydantic Structured Output Schema\n",
+    "By passing a Pydantic model to `response_schema`, Gemini guarantees 100% deterministic JSON output conforming to our data types."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import List\n",
+    "from pydantic import BaseModel, Field\n",
+    "\n",
+    "class ConceptBreakdown(BaseModel):\n",
+    "    concept_name: str = Field(description=\"Name of the technical concept or system component\")\n",
+    "    english_explanation: str = Field(description=\"Concise technical explanation in English\")\n",
+    "    tamil_explanation: str = Field(description=\"Precise Tamil (தமிழ்) translation and conceptual breakdown\")\n",
+    "    hindi_explanation: str = Field(description=\"Precise Hindi (हिन्दी) translation and conceptual breakdown\")\n",
+    "\n",
+    "class SecurityAssessment(BaseModel):\n",
+    "    contains_sensitive_data: bool = Field(description=\"Whether the diagram reveals credentials, secrets, or internal IPs\")\n",
+    "    security_risk_level: str = Field(description=\"Risk Level: Low, Medium, High, or Critical\")\n",
+    "    security_recommendations: List[str] = Field(description=\"Actionable security recommendations\")\n",
+    "\n",
+    "class IndicDocumentAnalysis(BaseModel):\n",
+    "    document_title: str = Field(description=\"Extracted title or summary of the diagram/document\")\n",
+    "    executive_summary: str = Field(description=\"Brief 2-sentence executive summary\")\n",
+    "    key_concepts: List[ConceptBreakdown] = Field(description=\"List of core technical concepts broken down into Indic languages\")\n",
+    "    security_evaluation: SecurityAssessment = Field(description=\"Security posture and risk findings\")\n",
+    "\n",
+    "print(\"✅ Pydantic Structured Output Schemas defined.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🖼️ Step 4: Load or Generate a Sample Cloud Architecture Diagram"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# @title Create an illustrative architecture diagram\n",
+    "from PIL import Image, ImageDraw, ImageFont\n",
+    "import io\n",
+    "\n",
+    "# Create an illustrative architecture diagram\n",
+    "img = Image.new('RGB', (900, 480), color=(248, 249, 250))\n",
+    "draw = ImageDraw.Draw(img)\n",
+    "\n",
+    "# Title\n",
+    "draw.rectangle([(0, 0), (900, 50)], fill=(26, 115, 232))\n",
+    "draw.text((20, 15), \"Cloud Microservices Architecture: Zero-Trust Telemetry Pipeline\", fill=(255, 255, 255))\n",
+    "\n",
+    "# Boxes\n",
+    "draw.rectangle([(40, 100), (220, 200)], fill=(232, 240, 254), outline=(26, 115, 232), width=2)\n",
+    "draw.text((55, 130), \"Client Apps / IoT\\n[Mobile, Web, Android]\", fill=(32, 33, 36))\n",
+    "\n",
+    "draw.rectangle([(280, 100), (460, 200)], fill=(254, 247, 224), outline=(249, 171, 0), width=2)\n",
+    "draw.text((295, 130), \"Google Cloud API\\nGateway + OAuth 2.0\", fill=(32, 33, 36))\n",
+    "\n",
+    "draw.rectangle([(520, 100), (700, 200)], fill=(230, 244, 234), outline=(52, 168, 83), width=2)\n",
+    "draw.text((535, 130), \"Gemini 2.0 Flash\\nAI Inference Cluster\", fill=(32, 33, 36))\n",
+    "\n",
+    "draw.rectangle([(740, 100), (870, 200)], fill=(252, 232, 230), outline=(234, 67, 53), width=2)\n",
+    "draw.text((750, 130), \"BigQuery +\\nCloud Spanner\", fill=(32, 33, 36))\n",
+    "\n",
+    "# Notes\n",
+    "draw.rectangle([(40, 280), (870, 440)], fill=(255, 255, 255), outline=(218, 220, 224), width=1)\n",
+    "draw.text((60, 300), \"Security Configuration:\\n- TLS 1.3 encryption across all internal microservice hops\\n- Strict Rate Limiting: 1,000 req/min per API key\\n- Data Residency: asia-south1 (Mumbai) Region\\n- Zero-Trust mTLS identity authentication\", fill=(60, 64, 67))\n",
+    "\n",
+    "img.save(\"cloud_architecture_sample.png\")\n",
+    "print(\"✅ Sample architecture diagram generated: cloud_architecture_sample.png\")\n",
+    "display(img)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🚀 Step 5: Run Multimodal Intelligence with Interactions API\n",
+    "We use the modern **Interactions API** (`client.interactions.create`) as recommended by the Google Gemini Cookbook style guide."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prompt = \"\"\"\n",
+    "You are a Principal Cloud Architect and AI Researcher specialized in Indic linguistic localization.\n",
+    "Analyze the provided cloud architecture diagram in detail:\n",
+    "1. Extract the main system title and executive summary.\n",
+    "2. For each key technical component in the diagram, provide a clear technical breakdown in:\n",
+    "   - English\n",
+    "   - Tamil (தமிழ்) - using high-standard, natural technical vocabulary\n",
+    "   - Hindi (हिन्दी) - using high-standard, natural technical vocabulary\n",
+    "3. Perform a rigorous security assessment evaluating encryption, identity, data residency, and vulnerabilities.\n",
+    "\"\"\"\n",
+    "\n",
+    "print(f\"Analyzing visual architecture with {MODEL_ID} using Interactions API...\")\n",
+    "\n",
+    "interaction = client.interactions.create(\n",
+    "    model=MODEL_ID,\n",
+    "    input=[img, prompt],\n",
+    "    config=types.GenerateContentConfig(\n",
+    "        response_mime_type=\"application/json\",\n",
+    "        response_schema=IndicDocumentAnalysis,\n",
+    "        temperature=0.2,\n",
+    "    ),\n",
+    ")\n",
+    "\n",
+    "# Parse JSON output from the InteractionStep content\n",
+    "import json\n",
+    "result_data = json.loads(interaction.steps[-1].content[0].text)\n",
+    "analysis = IndicDocumentAnalysis(**result_data)\n",
+    "print(\"✅ Analysis completed successfully!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 📊 Step 6: Visualizing the Trilingual Knowledge Breakdown & Security Report"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "print(f\"\\n{'='*70}\")\n",
+    "print(f\"📌 DOCUMENT: {analysis.document_title}\")\n",
+    "print(f\"📝 SUMMARY:  {analysis.executive_summary}\")\n",
+    "print(f\"{'='*70}\\n\")\n",
+    "\n",
+    "print(\"🌐 TRILINGUAL CONCEPT BREAKDOWN:\\n\")\n",
+    "for idx, c in enumerate(analysis.key_concepts, 1):\n",
+    "    print(f\"[{idx}] Concept: {c.concept_name}\")\n",
+    "    print(f\"    🇬🇧 English: {c.english_explanation}\")\n",
+    "    print(f\"    🇮🇳 தமிழ்:    {c.tamil_explanation}\")\n",
+    "    print(f\"    🇮🇳 हिन्दी:   {c.hindi_explanation}\")\n",
+    "    print(\"-\" * 70)\n",
+    "\n",
+    "sec = analysis.security_evaluation\n",
+    "print(f\"\\n🛡️ SECURITY ASSESSMENT:\")\n",
+    "print(f\"- Risk Level: {sec.security_risk_level}\")\n",
+    "print(f\"- Contains Exposed Secrets: {sec.contains_sensitive_data}\")\n",
+    "print(\"- Security Recommendations:\")\n",
+    "for r in sec.security_recommendations:\n",
+    "    print(f\"  • {r}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🎯 Conclusion & Next Steps\n",
+    "- **Gemini 2.0 Flash** allows native multimodal understanding with structured output at ultra-low latency.\n",
+    "- Vernacular localization (Tamil, Hindi) bridges educational and enterprise barriers for developers across India.\n",
+    "\n",
+    "### 🔗 Official Resources\n",
+    "- [Google Gemini API Documentation](https://ai.google.dev/gemini-api/docs)\n",
+    "- [Google AI Studio](https://aistudio.google.com/)\n",
+    "- [Google GenAI SDK GitHub](https://github.com/googleapis/python-genai)\n",
+    "- [Google Student Ambassadors Community](https://developers.google.com/community/gdsc)"
+   ]
+  }
+ ],
+ "metadata": {
+  "accelerator": "GPU",
+  "colab": {
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "Python 3",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}