派生属性：让数据自己"算"出来

“

系列：S4 本体建模 · 第 7 篇 | 难度：中级 | 阅读时间：18 分钟

#TL;DR

派生属性（Derived Property）是"计算列"的 Ontology 版本——定义一次计算规则，数据变更时自动重算，消费者读取时永远拿到最新值，终结了"在 10 个地方写同一个计算逻辑"的噩梦。
**9 种 Reducer（SUM / AVG / COUNT / MIN / MAX / FIRST / LAST / CONCAT / CUSTOM）**覆盖从简单聚合到自定义函数的全部计算场景，每种 Reducer 都有明确的语义和性能特征。
**三种求值模式（实时 / 延迟 / SQL-Backed）**允许根据数据规模和延迟要求选择最优策略。

#1. 为什么需要派生属性

#1.1 传统方式的痛点

Code

场景：订单的"总金额"

传统方式（到处写计算逻辑）：

前端：
  const total = order.items.reduce((sum, item) =>
    sum + item.price * item.quantity, 0)  // 逻辑 #1

后端 API：
  double total = lineItems.stream()
    .mapToDouble(item -> item.getPrice() * item.getQuantity())
    .sum();  // 逻辑 #2

报表系统：
  SELECT SUM(price * quantity) FROM line_items
  WHERE order_id = ?;  // 逻辑 #3

数据管道：
  df['total'] = df['price'] * df['quantity']
  df.groupby('order_id')['total'].sum()  // 逻辑 #4

问题：
├── 同一个计算在 4 个地方写了 4 遍
├── 其中一个地方改了（比如加了折扣），其他 3 个忘了改
├── 不同消费者看到不同的"总金额" → 数据不一致
├── 新人入职不知道"正确的"计算方式是哪个
└── 没有审计记录（谁定义的这个计算？何时改的？）

#1.2 coomia-dip 的解法

Code

coomia-dip 方式（Derived Property）：

定义一次：
┌──────────────────────────────────────────┐
│  ObjectType: Order                       │
│                                          │
│  properties:                             │
│  ├── orderId: STRING (primary key)       │
│  ├── status: ENUM                        │
│  └── totalAmount: DERIVED                │
│       ├── reducer: SUM                   │
│       ├── source: Order_contains_LineItem │
│       ├── field: lineTotal               │
│       └── where: status != 'CANCELLED'   │
└──────────────────────────────────────────┘

所有消费者：
  前端：order.totalAmount  // 直接读
  后端：order.totalAmount  // 直接读
  报表：order.totalAmount  // 直接读
  管道：order.totalAmount  // 直接读

  → 同一个值，同一个计算逻辑，永远一致 ✅

#2. 派生属性数据模型

#2.1 核心定义

Python

from ontology_sdk import OntologyClient

client = OntologyClient(base_url="http://localhost:8080")

# 创建带派生属性的 ObjectType
client.schema.create_object_type({
    "name": "Order",
    "primaryKey": "orderId",
    "properties": {
        "orderId": {"type": "STRING", "required": True},
        "customerId": {"type": "STRING", "required": True},
        "status": {"type": "ENUM", "enumValues": ["DRAFT", "CONFIRMED", "SHIPPED", "DELIVERED", "CANCELLED"]},

        # 派生属性：订单总金额
        "totalAmount": {
            "type": "DOUBLE",
            "derived": True,
            "derivation": {
                "reducer": "SUM",
                "sourceRelation": "Order_contains_LineItem",
                "sourceProperty": "lineTotal",
                "filter": {"status": {"neq": "CANCELLED"}},
            },
        },

        # 派生属性：行项目数量
        "itemCount": {
            "type": "INTEGER",
            "derived": True,
            "derivation": {
                "reducer": "COUNT",
                "sourceRelation": "Order_contains_LineItem",
                "filter": {"status": {"neq": "CANCELLED"}},
            },
        },

        # 派生属性：平均单价
        "avgItemPrice": {
            "type": "DOUBLE",
            "derived": True,
            "derivation": {
                "reducer": "AVG",
                "sourceRelation": "Order_contains_LineItem",
                "sourceProperty": "unitPrice",
            },
        },

        # 派生属性：最高单价商品
        "maxPriceItem": {
            "type": "STRING",
            "derived": True,
            "derivation": {
                "reducer": "MAX",
                "sourceRelation": "Order_contains_LineItem",
                "sourceProperty": "unitPrice",
                "returnProperty": "productName",  # 返回最高单价对应的商品名
            },
        },
    },
})

#2.2 Protobuf Schema

PROTOBUF

message DerivedPropertyDef {
    ReducerType reducer = 1;
    string source_relation = 2;
    string source_property = 3;
    string return_property = 4;
    FilterExpression filter = 5;
    EvaluationMode evaluation_mode = 6;
    string custom_expression = 7;
    string sql_query = 8;
    CacheConfig cache = 9;
}

enum ReducerType {
    SUM = 0;
    AVG = 1;
    COUNT = 2;
    MIN = 3;
    MAX = 4;
    FIRST = 5;
    LAST = 6;
    CONCAT = 7;
    CUSTOM = 8;
}

enum EvaluationMode {
    REALTIME = 0;    // 每次读取时计算
    DEFERRED = 1;    // 变更后异步重算
    SQL_BACKED = 2;  // 委托给数据库计算
}

#3. 9 种 Reducer 详解

#3.1 SUM — 求和

Python

# 订单总金额 = 所有行项目金额之和
"totalAmount": {
    "type": "DOUBLE",
    "derived": True,
    "derivation": {
        "reducer": "SUM",
        "sourceRelation": "Order_contains_LineItem",
        "sourceProperty": "lineTotal",  # lineTotal = unitPrice * quantity
    },
}

# 示例数据：
# LineItem-1: lineTotal = 100.00
# LineItem-2: lineTotal = 250.00
# LineItem-3: lineTotal = 75.50
# → totalAmount = 425.50

#3.2 AVG — 平均值

Python

# 客户平均订单金额
"avgOrderAmount": {
    "type": "DOUBLE",
    "derived": True,
    "derivation": {
        "reducer": "AVG",
        "sourceRelation": "Customer_hasOrder_Order",
        "sourceProperty": "totalAmount",
        "filter": {"status": {"in": ["DELIVERED", "SHIPPED"]}},
    },
}

#3.3 COUNT — 计数

Python

# 部门员工数
"employeeCount": {
    "type": "INTEGER",
    "derived": True,
    "derivation": {
        "reducer": "COUNT",
        "sourceRelation": "Department_hasEmployee_Employee",
        "filter": {"status": {"eq": "ACTIVE"}},
    },
}

#3.4 MIN — 最小值

Python

# 产品线中最低价格
"lowestPrice": {
    "type": "DOUBLE",
    "derived": True,
    "derivation": {
        "reducer": "MIN",
        "sourceRelation": "ProductLine_contains_Product",
        "sourceProperty": "price",
    },
}

#3.5 MAX — 最大值

Python

# 设备最近一次告警时间
"lastAlertTime": {
    "type": "TIMESTAMP",
    "derived": True,
    "derivation": {
        "reducer": "MAX",
        "sourceRelation": "Equipment_hasAlert_Alert",
        "sourceProperty": "createdAt",
    },
}

#3.6 FIRST — 第一个值

Python

# 客户的第一笔订单日期
"firstOrderDate": {
    "type": "DATE",
    "derived": True,
    "derivation": {
        "reducer": "FIRST",
        "sourceRelation": "Customer_hasOrder_Order",
        "sourceProperty": "orderDate",
        "orderBy": "orderDate",
        "orderDirection": "ASC",
    },
}

#3.7 LAST — 最后一个值

Python

# 客户的最近活动
"lastActivity": {
    "type": "STRING",
    "derived": True,
    "derivation": {
        "reducer": "LAST",
        "sourceRelation": "Customer_hasActivity_Activity",
        "sourceProperty": "description",
        "orderBy": "timestamp",
        "orderDirection": "ASC",
    },
}

#3.8 CONCAT — 拼接

Python

# 产品的所有标签（逗号分隔）
"tagList": {
    "type": "STRING",
    "derived": True,
    "derivation": {
        "reducer": "CONCAT",
        "sourceRelation": "Product_hasTag_Tag",
        "sourceProperty": "name",
        "separator": ", ",
        "orderBy": "name",
        "maxLength": 500,
    },
}
# → "AI, Machine Learning, NLP, Python"

#3.9 CUSTOM — 自定义表达式

Python

# 自定义计算：加权平均评分
"weightedRating": {
    "type": "DOUBLE",
    "derived": True,
    "derivation": {
        "reducer": "CUSTOM",
        "expression": """
            sum(review.rating * review.weight) / sum(review.weight)
        """,
        "sourceRelation": "Product_hasReview_Review",
        "variables": {
            "review.rating": "rating",
            "review.weight": "helpfulVotes",
        },
    },
}

# 自定义计算：健康评分（基于多个传感器读数）
"healthScore": {
    "type": "DOUBLE",
    "derived": True,
    "derivation": {
        "reducer": "CUSTOM",
        "expression": """
            let temp_score = if(temperature > 80, 0, if(temperature > 60, 50, 100));
            let vibration_score = if(vibration > 10, 0, if(vibration > 5, 50, 100));
            let pressure_score = if(pressure < 2 || pressure > 8, 0, 100);
            (temp_score + vibration_score + pressure_score) / 3
        """,
        "sourceRelation": "Equipment_hasReading_SensorReading",
        "variables": {
            "temperature": "readings.temperature",
            "vibration": "readings.vibration",
            "pressure": "readings.pressure",
        },
    },
}

#4. 三种求值模式

#4.1 REALTIME — 实时求值

Code

特点：
├── 每次读取时实时计算
├── 数据永远最新
├── 计算开销在读取时承担
└── 适合关联数据量小（< 1000 条）的场景

┌──────┐    读取请求    ┌──────────────┐    查询关联数据    ┌──────┐
│Client│──────────────►│ Runtime      │──────────────────►│  DB  │
│      │◄──────────────│              │◄──────────────────│      │
│      │    计算结果    │  (实时计算)   │    原始数据       │      │
└──────┘               └──────────────┘                   └──────┘

Python

# 配置实时求值
"employeeCount": {
    "type": "INTEGER",
    "derived": True,
    "derivation": {
        "reducer": "COUNT",
        "sourceRelation": "Department_hasEmployee_Employee",
        "evaluationMode": "REALTIME",
    },
}

#4.2 DEFERRED — 延迟求值（推荐）

Code

特点：
├── 源数据变更后异步重新计算
├── 计算结果缓存，读取时直接返回缓存值
├── 写入时触发，读取时无额外开销
├── 有短暂的数据不一致窗口（通常 < 5 秒）
└── 适合大多数业务场景

写入流程：
┌──────┐   写入LineItem   ┌──────┐   发布变更事件   ┌───────────┐
│Client│─────────────────►│  DB  │────────────────►│ Event Bus │
└──────┘                  └──────┘                 └─────┬─────┘
                                                         │
                                                         ▼
                                                  ┌───────────┐
                                                  │ Recompute │
                                                  │  Worker   │
                                                  │           │
                                                  │ SUM(...)  │
                                                  └─────┬─────┘
                                                        │
                                                        ▼ 更新缓存值
                                                  ┌───────────┐
                                                  │   Cache   │
                                                  └───────────┘

读取流程：
┌──────┐   读取 totalAmount   ┌───────────┐
│Client│────────────────────►│   Cache   │──► 直接返回缓存值
└──────┘                     └───────────┘

Python

# 配置延迟求值（默认模式）
"totalAmount": {
    "type": "DOUBLE",
    "derived": True,
    "derivation": {
        "reducer": "SUM",
        "sourceRelation": "Order_contains_LineItem",
        "sourceProperty": "lineTotal",
        "evaluationMode": "DEFERRED",
        "cache": {
            "ttlSeconds": 300,       # 缓存有效期 5 分钟
            "invalidateOnChange": True,  # 源数据变更时立即失效
        },
    },
}

#4.3 SQL_BACKED — SQL 委托求值

Code

特点：
├── 计算逻辑下推到数据库执行
├── 利用数据库的计算能力（适合复杂聚合）
├── 适合数据量大、计算复杂的场景
├── 依赖数据库性能
└── 支持窗口函数、子查询等高级 SQL 特性

┌──────┐   读取请求   ┌──────────────┐   执行 SQL   ┌──────┐
│Client│────────────►│   Runtime    │────────────►│ Doris│
│      │◄────────────│              │◄────────────│      │
│      │   计算结果   │(SQL Proxy)   │  查询结果   │      │
└──────┘             └──────────────┘             └──────┘

Python

# 配置 SQL 委托求值
"rollingAvg30d": {
    "type": "DOUBLE",
    "derived": True,
    "derivation": {
        "reducer": "CUSTOM",
        "evaluationMode": "SQL_BACKED",
        "sqlQuery": """
            SELECT AVG(daily_amount) as value
            FROM (
                SELECT DATE(order_date) as dt, SUM(total_amount) as daily_amount
                FROM orders
                WHERE customer_id = :objectId
                  AND order_date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY)
                GROUP BY DATE(order_date)
            ) daily_sums
        """,
    },
}

# 使用窗口函数的例子
"salesRank": {
    "type": "INTEGER",
    "derived": True,
    "derivation": {
        "evaluationMode": "SQL_BACKED",
        "sqlQuery": """
            SELECT rank_value as value FROM (
                SELECT product_id,
                       RANK() OVER (
                           PARTITION BY category
                           ORDER BY total_sales DESC
                       ) as rank_value
                FROM product_sales_summary
            ) ranked
            WHERE product_id = :objectId
        """,
    },
}

#5. 求值模式选择指南

Code

┌──────────────┬──────────────┬──────────────┬──────────────┐
│     特征      │   REALTIME   │   DEFERRED   │  SQL_BACKED  │
├──────────────┼──────────────┼──────────────┼──────────────┤
│ 数据新鲜度    │ 100% 实时    │ 延迟 < 5秒   │ 取决于SQL    │
│ 读取性能     │ 慢（需计算）  │ 快（读缓存） │ 取决于SQL    │
│ 写入影响     │ 无           │ 触发重算     │ 无           │
│ 数据量支持   │ < 1K 关联    │ < 100K 关联  │ 无限制       │
│ 计算复杂度   │ 简单聚合     │ 简单聚合     │ 任意复杂     │
│ 适用场景     │ 实时仪表板   │ 大多数业务   │ 复杂分析     │
│ 缓存开销     │ 无           │ 有           │ 可选         │
└──────────────┴──────────────┴──────────────┴──────────────┘

决策树：
  数据量 > 10 万？
  ├── Yes → SQL_BACKED
  └── No → 需要 100% 实时？
            ├── Yes → REALTIME
            └── No → DEFERRED（推荐）

#6. 表达式求值引擎

Python

# 派生属性支持丰富的表达式语法

# 数学运算
"profit": {
    "derived": True,
    "derivation": {
        "reducer": "CUSTOM",
        "expression": "self.revenue - self.cost",  # 直接引用本对象属性
    },
}

# 条件表达式
"riskLevel": {
    "derived": True,
    "derivation": {
        "reducer": "CUSTOM",
        "expression": """
            if(self.creditScore >= 750, 'LOW',
               if(self.creditScore >= 600, 'MEDIUM',
                  'HIGH'))
        """,
    },
}

# 日期计算
"daysSinceLastOrder": {
    "derived": True,
    "derivation": {
        "reducer": "CUSTOM",
        "expression": "dateDiff(now(), self.lastOrderDate, 'DAYS')",
    },
}

# 字符串操作
"fullAddress": {
    "derived": True,
    "derivation": {
        "reducer": "CUSTOM",
        "expression": """
            concat(self.homeAddress.province, ' ',
                   self.homeAddress.city, ' ',
                   self.homeAddress.street)
        """,
    },
}

# 跨关系引用
"managerName": {
    "derived": True,
    "derivation": {
        "reducer": "CUSTOM",
        "expression": "traverse(self, 'Employee_reportsTo_Employee').name",
    },
}

#7. 派生属性的更新机制

#7.1 触发式更新

Code

变更传播路径：

LineItem.quantity 变更
    │
    ▼
LineItem.lineTotal 重算（lineTotal = unitPrice * quantity）
    │
    ▼
Order.totalAmount 重算（SUM of lineTotal）
    │
    ▼
Order.avgItemPrice 重算（AVG of unitPrice）
    │
    ▼
Customer.totalSpent 重算（SUM of Order.totalAmount）
    │
    ▼
Customer.loyaltyTier 重算（基于 totalSpent 的等级）

这就是"级联重算"——下一篇（S4-08）将详细介绍

#7.2 批量重算

Python

# 手动触发批量重算（当数据迁移或修复后使用）
result = client.schema.recompute_derived_property(
    object_type="Order",
    property_name="totalAmount",
    filter={"status": {"in": ["CONFIRMED", "SHIPPED"]}},
    batch_size=1000,
    concurrency=5,
)

print(f"重算完成: {result.total_objects} 个对象")
print(f"耗时: {result.duration_seconds} 秒")
print(f"成功: {result.success_count}")
print(f"失败: {result.failure_count}")

#8. 派生属性与索引

Python

# 派生属性也可以建立索引（用于过滤和排序）
client.schema.create_index({
    "objectType": "Order",
    "properties": ["totalAmount"],
    "type": "BTREE",
    "name": "idx_order_total_amount",
})

# 基于派生属性的查询
orders = client.objects.search("Order", {
    "filters": {
        "totalAmount": {"gte": 10000},  # 大单
    },
    "orderBy": "totalAmount",
    "orderDirection": "DESC",
    "maxResults": 20,
})

#9. 错误处理

Python

# 派生属性计算失败的处理策略

"riskyDerived": {
    "type": "DOUBLE",
    "derived": True,
    "derivation": {
        "reducer": "CUSTOM",
        "expression": "self.revenue / self.unitsSold",  # 可能除以零
        "errorHandling": {
            "onDivisionByZero": "RETURN_NULL",  # 除以零返回 null
            "onNullInput": "RETURN_NULL",       # 输入为 null 时返回 null
            "onOverflow": "RETURN_MAX",          # 溢出返回最大值
            "onError": "RETURN_DEFAULT",         # 其他错误返回默认值
            "defaultValue": 0,
        },
    },
}

#10. 实战案例：客户 360 视图

Python

# 客户的完整派生属性集合

client.schema.create_object_type({
    "name": "Customer",
    "primaryKey": "customerId",
    "properties": {
        "customerId": {"type": "STRING", "required": True},
        "name": {"type": "STRING", "required": True},
        "email": {"type": "STRING"},
        "registeredAt": {"type": "TIMESTAMP"},

        # 1. 订单统计
        "totalOrders": {
            "type": "INTEGER",
            "derived": True,
            "derivation": {"reducer": "COUNT", "sourceRelation": "Customer_hasOrder_Order"},
        },
        "totalSpent": {
            "type": "DOUBLE",
            "derived": True,
            "derivation": {"reducer": "SUM", "sourceRelation": "Customer_hasOrder_Order", "sourceProperty": "totalAmount"},
        },
        "avgOrderAmount": {
            "type": "DOUBLE",
            "derived": True,
            "derivation": {"reducer": "AVG", "sourceRelation": "Customer_hasOrder_Order", "sourceProperty": "totalAmount"},
        },

        # 2. 时间维度
        "firstOrderDate": {
            "type": "DATE",
            "derived": True,
            "derivation": {"reducer": "MIN", "sourceRelation": "Customer_hasOrder_Order", "sourceProperty": "orderDate"},
        },
        "lastOrderDate": {
            "type": "DATE",
            "derived": True,
            "derivation": {"reducer": "MAX", "sourceRelation": "Customer_hasOrder_Order", "sourceProperty": "orderDate"},
        },
        "daysSinceLastOrder": {
            "type": "INTEGER",
            "derived": True,
            "derivation": {"reducer": "CUSTOM", "expression": "dateDiff(now(), self.lastOrderDate, 'DAYS')"},
        },

        # 3. 客户分层
        "loyaltyTier": {
            "type": "STRING",
            "derived": True,
            "derivation": {
                "reducer": "CUSTOM",
                "expression": """
                    if(self.totalSpent >= 100000, 'PLATINUM',
                       if(self.totalSpent >= 50000, 'GOLD',
                          if(self.totalSpent >= 10000, 'SILVER',
                             'BRONZE')))
                """,
            },
        },

        # 4. 活跃度评分
        "activityScore": {
            "type": "DOUBLE",
            "derived": True,
            "derivation": {
                "reducer": "CUSTOM",
                "expression": """
                    let recency = min(self.daysSinceLastOrder / 90.0, 1.0);
                    let frequency = min(self.totalOrders / 50.0, 1.0);
                    let monetary = min(self.totalSpent / 100000.0, 1.0);
                    (1 - recency) * 0.3 + frequency * 0.3 + monetary * 0.4
                """,
            },
        },

        # 5. 产品偏好
        "topCategory": {
            "type": "STRING",
            "derived": True,
            "derivation": {
                "evaluationMode": "SQL_BACKED",
                "sqlQuery": """
                    SELECT category as value
                    FROM order_line_items oli
                    JOIN orders o ON oli.order_id = o.order_id
                    JOIN products p ON oli.product_id = p.product_id
                    WHERE o.customer_id = :objectId
                    GROUP BY category
                    ORDER BY COUNT(*) DESC
                    LIMIT 1
                """,
            },
        },
    },
})

# 使用派生属性进行客户分析
vip_customers = client.objects.search("Customer", {
    "filters": {
        "loyaltyTier": {"in": ["PLATINUM", "GOLD"]},
        "activityScore": {"gte": 0.7},
        "daysSinceLastOrder": {"lte": 30},
    },
    "orderBy": "totalSpent",
    "orderDirection": "DESC",
})

for c in vip_customers:
    print(f"客户: {c.name}")
    print(f"  等级: {c.loyaltyTier}")
    print(f"  累计消费: ¥{c.totalSpent:,.2f}")
    print(f"  活跃度: {c.activityScore:.2f}")
    print(f"  最近下单: {c.daysSinceLastOrder} 天前")
    print(f"  偏好品类: {c.topCategory}")

#11. 性能优化

Code

派生属性性能优化策略：

┌─────────────────────────────────────────────────┐
│                性能优化矩阵                       │
├────────────────┬────────────────────────────────┤
│  优化策略       │  适用场景                       │
├────────────────┼────────────────────────────────┤
│  延迟求值+缓存  │  大多数场景（默认推荐）          │
│  增量计算       │  SUM/COUNT/AVG 可增量更新       │
│  SQL 下推       │  复杂聚合、大数据量             │
│  预物化         │  读多写少的派生属性             │
│  批量重算       │  数据修复、初始化              │
│  并行重算       │  多个无依赖的派生属性           │
└────────────────┴────────────────────────────────┘

增量计算示例（SUM）：
  旧值：totalAmount = 1000
  新增 LineItem.lineTotal = 200
  新值：totalAmount = 1000 + 200 = 1200

  无需重新遍历所有 LineItem！
  时间复杂度从 O(N) 降到 O(1)

#Key Takeaways

派生属性消灭了计算逻辑的重复——一次定义，所有消费者（前端、后端、报表、管道）看到同一个结果，数据一致性有了 Schema 级保障。
9 种 Reducer 覆盖全场景：SUM/AVG/COUNT/MIN/MAX 是基础聚合，FIRST/LAST 处理排序取值，CONCAT 拼接文本，CUSTOM 支持任意复杂表达式。
三种求值模式的选择：REALTIME 保证实时性（小数据量），DEFERRED 平衡新鲜度和性能（推荐），SQL_BACKED 处理复杂计算（大数据量）。
表达式引擎支持丰富语法——数学运算、条件判断、日期计算、字符串操作、跨关系引用，足够表达绝大多数业务计算。
派生属性可建立索引——计算结果可以被过滤和排序，像普通属性一样参与查询。

#Next Article

下一篇 S4-08 派生属性依赖 DAG 将深入"级联计算"的底层机制——当一个属性的变更引发多层派生属性重算时，如何用拓扑排序确保计算顺序正确、如何检测循环依赖、如何批量重算数百万对象。

#ontology #derived-property #reducer #expression-eval #sql-backed #computed-column #data-consistency