❌ Bypass Ontology to query databases directly

Traditional Microservice Architecture:

Service A ──── Database A (MySQL)
Service B ──── Database B (PostgreSQL)
Service C ──── Database C (MongoDB)
Service D ──── Database D (Redis)

Problem: data silos, no unified semantics, cross-service queries are difficult

Ontology-Driven Architecture:

                    ┌──────────────────────┐
                    │  OntologyRuntimeSvc  │
Service A ──gRPC──> │  (Schema validation) │
Service B ──gRPC──> │  (Permission check)  │──> Doris / Iceberg / ...
Service C ──gRPC──> │  (Audit logging)     │
Service D ──gRPC──> │  (Event publishing)  │
                    └──────────────────────┘

The single entry point for all data access

service OntologyRuntimeService {
  // Instance lifecycle
  rpc CreateObject(CreateObjectRequest) returns (CreateObjectResponse);
  rpc GetObject(GetObjectRequest) returns (GetObjectResponse);
  rpc UpdateObject(UpdateObjectRequest) returns (UpdateObjectResponse);
  rpc DeleteObject(DeleteObjectRequest) returns (DeleteObjectResponse);

  // Batch operations
  rpc BatchGetObjects(BatchGetRequest) returns (BatchGetResponse);
  rpc SearchObjects(SearchObjectsRequest) returns (SearchObjectsResponse);

  // Relation operations
  rpc CreateLink(CreateLinkRequest) returns (CreateLinkResponse);
  rpc GetLinkedObjects(GetLinkedRequest) returns (GetLinkedResponse);

  // Aggregate queries
  rpc AggregateObjects(AggregateRequest) returns (AggregateResponse);

  // Derived properties
  rpc GetDerivedProperty(DerivedPropertyRequest) returns (DerivedPropertyResponse);
}

Request arrives → Fetch ObjectType definition from SchemaRegistry
               → Validate requested fields exist in the Schema
               → Validate field types match
               → Validate constraints (required, range, enum values)
               → Validate no unknown/disallowed fields

Schema validation passes → Extract WorldContext from request context
                        → Determine current Tenant/Org/Space/Project/World
                        → Fetch RBAC + ABAC rules from PolicyEngine
                        → Check read/write permission per property
                        → Inject row-level filter conditions

ObjectType: Employee
  - name: readable by everyone
  - department: readable by everyone
  - salary: readable only by HR role
  - ssn: readable only by Compliance role

Direct DB query → SELECT * FROM entity_common WHERE type='Employee'
               → Exposes salary and ssn fields

Through Ontology → OntologyRuntimeService filters based on caller's role
                → Automatically strips unauthorized properties
                → Response does not include salary or ssn

Permission check passes → Record operation audit log
                       → Who (principal) at what time
                       → On which object in which World
                       → What operation was performed
                       → What was the result
                       → Send to Kafka audit topic

Audit logging complete → Publish data change event to Kafka
                      → Trigger subscriber notifications
                      → Trigger derived property cascade computation
                      → Trigger materialized view incremental updates
                      → Trigger search index incremental sync

Direct Doris query:
  Network latency: ~0.5ms
  Query execution: ~1-5ms (depends on data volume)
  Total latency:   ~1.5-5.5ms

Through OntologyRuntimeService:
  gRPC call overhead:        ~0.3ms
  Schema validation:         ~0.2ms (after caching)
  Permission check:          ~0.5ms (after caching)
  Audit logging (async):     ~0.1ms (non-blocking)
  Event publishing (async):  ~0.1ms (non-blocking)
  Doris query:               ~1-5ms
  gRPC response serialization: ~0.2ms
  Total latency:             ~2.4-6.4ms

First request:
  SchemaRegistry.GetObjectType() → Redis → PostgreSQL
  Cache duration: 5 minutes (Redis), 60 seconds (local memory)
  Schema changes actively invalidate cache via Kafka events

Subsequent requests:
  Local memory cache hit → ~0.01ms
  Hit rate: >99.5%

PolicyEngine pre-compiles RBAC + ABAC rules into a decision tree
First compilation: ~5ms
Subsequent evaluation: ~0.1ms (binary tree traversal)
Incremental recompilation on rule changes

Audit logging → Kafka Producer (async, no ACK wait)
Event publishing → Kafka Producer (async, batched)

Main flow blocking: <0.2ms
At-least-once delivery guaranteed via Kafka acks=1

Requesting 100 objects in a single call:
  Direct DB: 100 * 5ms = 500ms (one by one)
             or 5-10ms (WHERE IN batch)

  Ontology layer: Schema validation once (shared ObjectType)
                  Permission check once (shared WorldContext)
                  Doris query once (WHERE IN)
                  Audit 1 record (batch operation entry)
                  Total latency: ~8-15ms

  In batch scenarios, the Ontology Tax is amortized to near zero

┌─────────────────────────────────────────────────────┐
│              OntologyRuntimeService                  │
│                                                      │
│  ┌──────────┐  ┌──────────┐  ┌──────────────────┐  │
│  │ gRPC     │  │ Schema   │  │ Permission       │  │
│  │ Endpoint │→ │ Resolver │→ │ Evaluator        │  │
│  └──────────┘  └──────────┘  └──────────────────┘  │
│                                      │               │
│                                      ▼               │
│  ┌──────────────────────────────────────────────┐   │
│  │           Query Planner                       │   │
│  │  ┌───────┐ ┌────────┐ ┌──────────────────┐  │   │
│  │  │ Doris │ │Iceberg │ │ DerivedProperty   │  │   │
│  │  │ Query │ │ Query  │ │ Calculator        │  │   │
│  │  └───────┘ └────────┘ └──────────────────┘  │   │
│  └──────────────────────────────────────────────┘   │
│                       │                              │
│                       ▼                              │
│  ┌──────────┐  ┌──────────┐  ┌──────────────────┐  │
│  │ Audit    │  │ Event    │  │ Response          │  │
│  │ Logger   │  │Publisher │  │ Assembler         │  │
│  └──────────┘  └──────────┘  └──────────────────┘  │
│                                                      │
└─────────────────────────────────────────────────────┘

SchemaResolver.resolve("Equipment", worldContext)
  → Check local cache
  → Miss → Check Redis cache
  → Miss → Query SchemaRegistry (gRPC)
  → Get Equipment ObjectType definition
  → Check inheritance: Equipment extends Asset
  → Recursively resolve Asset definition
  → Merge properties: Asset.properties + Equipment.properties
  → Merge constraints: Asset.constraints + Equipment.constraints
  → Cache result
  → Return complete ResolvedSchema

Check flow:
  1. Extract user identity from WorldContext
  2. Query user's roles in current Org/Space/Project/World
  3. Query role permissions for target ObjectType
  4. Permission inheritance: World → Project → Space → Org → Tenant
  5. Result: ALLOW / DENY / PARTIAL (property-level filtering)

Check flow:
  1. Get ABAC policy rules
  2. Evaluate condition expressions:
     - User attributes (department, level, clearance)
     - Resource attributes (classification, owner, region)
     - Environment attributes (time, ip, device)
  3. Generate row-level filter conditions for injection into queries

Scenario 1: Simple property query
  → Generate Doris SQL: SELECT col1, col2 FROM entity_common WHERE ...
  → Inject permission filter conditions
  → Execute

Scenario 2: Includes derived properties
  → Separate regular and derived properties
  → Regular properties → Doris query
  → Derived properties → ComputationCoordinator
  → Merge results

Scenario 3: Includes relation traversal
  → Generate entity_edge table JOIN query
  → Or generate multi-step query plan (breadth-first traversal)
  → Execute permission checks on each traversal step

Scenario 4: Aggregate query
  → Generate Doris aggregate SQL
  → Consider impact of permission filtering on aggregate results
  → If row-level filters exist, filter first then aggregate

REST API (external):
GET /api/v1/ontology/objects/Equipment?
  filter=status eq 'RUNNING'&
  select=name,location,utilization&
  pageSize=20

→ API Gateway converts to gRPC call
→ OntologyRuntimeService.SearchObjects()

Step 1: gRPC request arrives (t=0ms)
  ├── Parse SearchObjectsRequest
  ├── Extract WorldContext (from gRPC metadata)
  └── Record request start time

Step 2: Schema resolution (t=0.1ms)
  ├── SchemaResolver.resolve("Equipment")
  ├── Local cache hit ✓
  ├── Validate: "name" → STRING ✓
  ├── Validate: "location" → STRING ✓
  ├── Validate: "utilization" → DERIVED_DOUBLE ✓ (marked as derived)
  ├── Validate: "status" → ENUM[RUNNING,STOPPED,MAINTENANCE] ✓
  └── Validate: filter expression syntax is valid ✓

Step 3: Permission check (t=0.5ms)
  ├── User role: FactoryOperator
  ├── RBAC check: FactoryOperator has READ on Equipment ✓
  ├── ABAC check: User department=Manufacturing
  │   ├── Policy: "Manufacturing dept can only see their own factory equipment"
  │   └── Generate row filter: AND factory_id IN ('F001', 'F002')
  ├── Property-level check:
  │   ├── name → allowed ✓
  │   ├── location → allowed ✓
  │   └── utilization → allowed ✓
  └── Done

Step 4: Query planning (t=0.8ms)
  ├── Separate regular properties: name, location (Doris query)
  ├── Separate derived properties: utilization (needs computation)
  ├── Generate Doris SQL:
  │   SELECT object_id, attributes->>'name' AS name,
  │          attributes->>'location' AS location
  │   FROM entity_common
  │   WHERE type = 'Equipment'
  │     AND attributes->>'status' = 'RUNNING'
  │     AND factory_id IN ('F001', 'F002')  -- ABAC injection
  │     AND world_id = 'W-2024-001'
  │   ORDER BY object_id
  │   LIMIT 20
  └── Prepare ComputationCoordinator call

Step 5: Execute queries (t=1.0ms → t=4.5ms)
  ├── Doris query execution: returns 20 records
  └── Call ComputationCoordinator for 20 object_ids
      ├── utilization strategy routing: EXPRESSION (simple calculation)
      ├── Formula: running_hours / total_hours * 100
      ├── Requires running_hours and total_hours → query Doris
      └── Computation complete: returns 20 utilization values

Step 6: Result assembly (t=5.0ms)
  ├── Merge Doris results with derived property results
  ├── Property filtering: remove unauthorized properties (all allowed this time)
  ├── Build gRPC Response
  └── Set pagination cursor

Step 7: Audit + Events (async, t=5.1ms)
  ├── Send audit log to Kafka audit.data-access
  │   { principal: "user-123", action: "SEARCH",
  │     objectType: "Equipment", resultCount: 20,
  │     worldId: "W-2024-001", timestamp: "..." }
  └── (Search operations do not trigger data change events)

Step 8: Response returned (t=5.2ms)
  └── Total latency: ~5.2ms (Ontology Tax ≈ 1.7ms)

Developer A "optimizes for performance" by querying Doris directly:

SELECT * FROM entity_common
WHERE type = 'Employee'
AND attributes->>'department' = 'Engineering'

Result: Returns all properties for all engineering employees
        Including salary, ssn, performance_review
        Information that should only be visible to HR

Impact: Data leak → Compliance violation → Legal risk

Developer B directly modifies inventory quantity in Doris:

UPDATE entity_common
SET attributes = jsonb_set(attributes, '{quantity}', '100')
WHERE object_id = 'INV-001'

Result: quantity changed, but derived property total_value (= quantity * unit_price)
        was not recalculated, still showing the old value
        Inventory reports show inconsistent data

Impact: Business decisions based on wrong data → Financial discrepancies → Audit issues

Developer C directly deletes a sensitive record from Doris:

DELETE FROM entity_common
WHERE object_id = 'CASE-007'

Result: Record is gone, but there's no deletion entry in audit logs
        Cannot explain data whereabouts during regulatory review
        Nessie version history also has no corresponding change

Impact: Audit non-compliance → Regulatory penalties → Reputation damage

Developer D directly updates an order status:

UPDATE entity_common
SET attributes = jsonb_set(attributes, '{status}', '"SHIPPED"')
WHERE object_id = 'ORDER-123'

Result: Order status changed, but:
  - Downstream "shipment notification" Action was not triggered
  - Customer did not receive shipping alert
  - Inventory "shipped quantity" derived property was not updated
  - Operations dashboard "pending shipment count" was not decremented
  - Associated logistics World was not synchronized

Impact: Entire business process breaks → Customer complaints → Operational chaos

Developer E writes a cross-tenant statistics query:

SELECT type, COUNT(*) FROM entity_common
GROUP BY type

Result: Returns statistics across all tenants
        Tenant A can infer Tenant B's business scale
        Violates data isolation commitments between tenants

Impact: Tenant trust crisis → Contract breach → Customer churn

message SearchObjectsRequest {
  string object_type = 1;           // What type I want to query
  string filter = 2;                // Filter conditions
  repeated string select_properties = 3; // Which properties I need
  int32 page_size = 4;              // Items per page
  string page_token = 5;            // Pagination cursor
  WorldContext context = 6;         // In which World
}

Solution: Batch Import Pipeline

1. Data first lands in Staging Area (Parquet files on MinIO)
2. SchemaValidator validates the entire file in bulk
3. PermissionChecker checks import permission once
4. BulkLoader writes directly to Doris (bypassing per-record gRPC)
5. AuditLogger records one bulk import audit entry
6. EventPublisher publishes bulk change event

Key: Not "bypassing Ontology" but "Ontology's batch mode"
     Schema validation and permission checks still execute
     Only changed from per-record to bulk

Solution: Ontology-Aware SQL Gateway

1. Analyst writes SQL query
2. SQL Gateway parses the SQL
3. Injects permission filter conditions (based on user's RBAC/ABAC)
4. Forwards to Doris for execution
5. Records audit log
6. Returns results

Key: Allows SQL flexibility
     But permissions and audit are still guaranteed by the Ontology layer

Solution: Embedded Ontology Validator

1. Stream processor preloads relevant ObjectType schemas on startup
2. Executes schema validation in-process (avoids network calls)
3. Permission checks via precomputed rule cache
4. Audit logs sent in async batches
5. Schema changes synced in real time to stream processor via Kafka events

Key: Embeds Ontology logic into the stream processor
     Eliminates network call overhead
     But Ontology's semantic guarantees remain unchanged

# ❌ Prohibited: Using Doris client directly
from doris_client import DorisConnection
conn = DorisConnection("doris:9030")
result = conn.query("SELECT * FROM entity_common WHERE type='Equipment'")

# ✅ Correct: Access through Ontology SDK
from ontology_sdk import OntologyClient
client = OntologyClient(world_context=ctx)
result = client.search("Equipment", filter="status eq 'RUNNING'")

Dependency graph for all services:

Service A ──> ontology-sdk ──> OntologyRuntimeService ──> Doris
Service B ──> ontology-sdk ──> OntologyRuntimeService ──> Doris
Service C ──> ontology-sdk ──> OntologyRuntimeService ──> Doris

No service directly depends on a Doris client library

# Dependency audit in .gitlab-ci.yml
dependency-audit:
  script:
    - |
      # Check if any service directly depends on database clients
      if grep -r "doris_client\|mysql.connector\|psycopg2" src/; then
        echo "ERROR: Direct database access detected!"
        echo "Use ontology-sdk instead."
        exit 1
      fi