b30a4f0d32
The diagram path now follows a documented standard procedure (lookup the closest Oracle Architecture Center reference → confirm components → author absolute_layout → spec validator → render → visually verify) and ships persistent guardrails so layout regressions can't recur. Persistent procedure changes (apply to all users, all sessions): - tools/diagram_spec_validator.py — geometry checks (CONTAINER_TOO_THIN, CONTAINER_PADDING_VIOLATION, LABEL_OVERFLOW_PARENT) run BEFORE either renderer (drawio + PPTX). Catches the subnet-collapse / label-overflow bugs that the post-render drawio validator missed. - tools/oci_diagram_gen.py + tools/oci_pptx_diagram_gen.py — call the spec validator before emitting any output. Adds mysql / mysql_heatwave type aliases. - tools/archcenter_pattern_lookup.py — scores against cached page descriptions (not just the 1-line summary), supports --queries for multi-fragment composition, and applies synonym expansion via kb/architecture-center/synonyms.yaml so "LB HA cross AD" matches "load balancer high availability availability domain". - kb/architecture-center/synonyms.yaml — canonical synonym table (load balancer, autonomous database, data guard, …) used by the lookup scorer. KB enrichment: - tools/archcenter_description_fetcher.py + 121 cached _description.md under kb/diagram/assets/archcenter-refs/<slug>/. Removes the runtime dependency on docs.oracle.com when authoring specs and feeds the pattern-lookup scorer. - 110+ cached .drawio / .svg / .png references for offline reuse, plus the OCI Toolkit v24.2 import (kb/diagram/assets/oci-toolkit-drawio). Documentation: - docs/skill/output-formats.md — new "Standard diagram-generation procedure (MANDATORY)" + geometry rules + the new validator entry. - SKILL.md option 2 — references the mandatory procedure. - README.md — describes the spec validator, archcenter_pattern_lookup and description fetcher, and updates the KB-health table. Tooling that backs the procedure (cumulative across recent sessions): tools/archcenter_case_runner.py, archcenter_batch_driver.py, archcenter_zip_downloader.py, drawio_visual_validator.py, drawio_fidelity_eval.py, harvest_drawio_icon.py, import_oci_library.py, oci_pptx_diagram_gen.py, oci_pptx_render.py, refresh_pptx_icon_index.py. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4.2 KiB
Stream Kafka topics to Oracle Autonomous Database using Oracle Integration 3
- Source: https://docs.oracle.com/en/solutions/oci-oad-kafka-oi/index.html
- Date: 2025-02
- Type: reference-architecture
- Services: adb-s, oic3, streaming
- Tags: database, integration, autonomous
Summary (catalog)
Kafka to ADB-S streaming ingestion via OIC3. Oracle Integration adapter for Kafka topics with schema mapping to ADB-S tables. Near-real-time data pipeline for analytics workloads.
Architecture (fetched from source)
Architecture
This architecture shows on-premises Kafka Streams, Oracle Integration 3, Oracle GoldenGate Stream Analytics and Oracle Autonomous Database in an OCI region.
Use this architecture to capture Kafka streaming messages from an on-premises Kafka stream or Oracle GoldenGate Stream Analytics into Oracle Autonomous Database using Oracle Integration 3.
The OCI region containing Oracle Integration 3 ingests data from the on-premises Kafka streams or Oracle GoldenGate Stream Analytics.
Oracle Integration 3 stores the data in a relational table in Oracle Autonomous Database .
Description of the illustration oci-oad-kafka-oi.png
oci-oad-kafka-oi-oracle.zip
This architecture supports the following components:
-
Kafka Streams Kafka Streams is a client library for building applications and microservices, where the input and output data are stored in Kafka clusters. It combines the simplicity of writing and deploying standard Java and Scala applications on the client side with the benefits of Kafka's server-side cluster technology.
-
Autonomous Database Oracle Autonomous Database is a fully-managed, preconfigured database environment that you can use for transaction processing and data warehousing workloads. You do not need to configure or manage any hardware, or install any software. Oracle Cloud Infrastructure handles creating, backing up, patching, upgrading, and tuning the database.
-
Oracle Integration 3 Oracle Integration 3 is a fully managed, preconfigured environment that gives you the power to integrate your cloud and on-premises applications, automate business processes, develop visual applications, use an SFTP-compliant file server to store and retrieve files, and exchange business documents with a B2B trading partner.
-
Oracle GoldenGate Stream Analytics GoldenGate Stream Analytics allows for the creation of custom operational dashboards that provide real-time monitoring and analyses of event streams in an Apache Spark-based system. It enables customers to identify events of interest in their Apache Spark-based system, execute queries against those event streams in real time and drive operational dashboards or raise alerts based on that analysis.
-
Oracle SQL Developer Oracle SQL Developer is a free, integrated development environment that simplifies the development and management of Oracle Database in both traditional and Cloud deployments. SQL Developer offers complete end-to-end development of your PL/SQL applications, a worksheet for running queries and scripts, a DBA console for managing the database, a reports interface, a complete data modeling solution, and a migration platform for moving your 3rd party databases to Oracle.
About Required Products and Roles
This solution requires the following products and roles:
-
Oracle Autonomous Database
-
Oracle Integration 3
-
Oracle Cloud Infrastructure
-
Oracle GoldenGate Stream Analytics
These are the roles needed for each service.
Product Name: Role Required to...
Oracle Autonomous Database : admin
Create the credentials.
Oracle Integration 3: admin
Create the credentials.
Oracle Cloud Infrastructure : admin
Create and manage OCI resources.
Oracle GoldenGate Stream Analytics: admin
Configure Kafka producer to ingest data.
See Oracle Products, Solutions, and Services to get what you need.
Title and Copyright Information
Stream Kafka topics to Oracle Autonomous Database using Oracle Integration 3
G24768-01
February 2025
Copyright © 2025,
Oracle and/or its affiliates.