“

Series: S12 Developer Tutorials · Article 20 | Level: Intermediate | Reading Time: 15 min

Performance Tuning Practical Guide

#Introduction

coomia-dip is optimized for medium-scale scenarios by default, but when data grows to tens of millions of records, concurrent users reach hundreds, and query complexity increases, targeted performance tuning becomes necessary. This guide covers practical tuning experience across the full stack: OQL query optimization, Doris table design, gRPC connection pools, JVM tuning, caching strategies, and load testing.

#1. Performance Diagnosis Methodology

#1.1 Golden Signals

Before tuning, identify the bottleneck by monitoring four golden signals:

Latency: Request response time distribution
Throughput: Requests processed per second
Error Rate: Percentage of failed requests
Saturation: Resource utilization (CPU/Memory/Disk I/O)

#1.2 Request Path Layers

Code

Client -> API Gateway -> Control Layer -> OQL Engine -> Doris
                                           |
                                           +-> Query Planning -> Execution

Each layer can be a bottleneck. Use distributed tracing (OpenTelemetry) to identify which layer is slow.

#2. OQL Query Optimization

#2.1 Slow Query Analysis

Python

slow_queries = platform.admin.get_slow_queries(threshold_ms=1000, limit=20)
for q in slow_queries:
    print(f"{q.duration_ms}ms: {q.query[:200]}")
    print(f"  Scanned: {q.scanned_rows}, Returned: {q.returned_rows}")

#2.2 Index Optimization

Python

platform.ontology.create_index(
    object_type="Order",
    index_name="idx_status_created",
    fields=["status", "createdAt"],
    index_type="BITMAP",
)

#2.3 Query Rewrite Tips

SQL

-- Bad: Full table scan due to function on column
SELECT * FROM Order WHERE YEAR(createdAt) = 2025
-- Good: Range query uses index
SELECT * FROM Order WHERE createdAt >= '2025-01-01' AND createdAt < '2026-01-01'

-- Bad: Deep pagination
SELECT * FROM Order ORDER BY createdAt DESC LIMIT 100 OFFSET 10000
-- Good: Cursor-based pagination
SELECT * FROM Order WHERE createdAt < :cursor ORDER BY createdAt DESC LIMIT 100

-- Bad: SELECT *
SELECT * FROM Order WHERE status = 'ACTIVE'
-- Good: Select only needed columns
SELECT orderId, totalAmount FROM Order WHERE status = 'ACTIVE'

#3. Doris Performance Tuning

#3.1 Table Design

SQL

CREATE TABLE order_optimized (
    _rid VARCHAR(64) NOT NULL,
    order_id VARCHAR(64),
    total_amount DOUBLE,
    status VARCHAR(32),
    created_at DATETIME NOT NULL
) ENGINE=OLAP
UNIQUE KEY(_rid)
PARTITION BY RANGE(created_at) (
    PARTITION p202501 VALUES [('2025-01-01'), ('2025-02-01'))
)
DISTRIBUTED BY HASH(_rid) BUCKETS 16
PROPERTIES (
    "dynamic_partition.enable" = "true",
    "dynamic_partition.time_unit" = "MONTH",
    "bloom_filter_columns" = "customer_id,status"
);

#3.2 Materialized Views

SQL

CREATE MATERIALIZED VIEW mv_daily_stats AS
SELECT DATE(created_at) AS d, status, COUNT(*) AS cnt, SUM(total_amount) AS rev
FROM order_optimized GROUP BY DATE(created_at), status;

#4. gRPC Performance Tuning

#4.1 Connection Pool Configuration

Python

channel_options = [
    ("grpc.max_send_message_length", 50 * 1024 * 1024),
    ("grpc.keepalive_time_ms", 30000),
    ("grpc.keepalive_timeout_ms", 10000),
    ("grpc.enable_retries", True),
    ("grpc.max_reconnect_backoff_ms", 30000),
]

#4.2 Server Thread Pool

Java

Server server = ServerBuilder.forPort(50051)
    .executor(Executors.newFixedThreadPool(availableProcessors() * 2))
    .maxInboundMessageSize(50 * 1024 * 1024)
    .build();

#5. JVM Tuning (Control Layer / Data Layer)

Bash

JAVA_OPTS="
  -Xms4g -Xmx8g
  -XX:+UseZGC -XX:+ZGenerational
  -XX:MaxGCPauseMillis=10
  -XX:+HeapDumpOnOutOfMemoryError
  -Xlog:gc*:file=/var/log/gc.log:time,uptime:filecount=10,filesize=100m
"

#6. Caching Strategy

#6.1 Multi-Level Cache

L1: In-process cache (Caffeine/LRU, microsecond access)
L2: Distributed cache (Redis, sub-millisecond)
L3: Query result cache (Doris Query Cache)

Python

platform.admin.set_cache_config({
    "schema_cache": {"type": "LOCAL", "ttl_seconds": 300},
    "object_cache": {"type": "REDIS", "ttl_seconds": 60, "max_entries": 100000},
    "query_result_cache": {"type": "DORIS", "ttl_seconds": 30},
})

#7. Load Testing & Continuous Monitoring

#7.1 Load Testing with Locust

Python

from locust import HttpUser, task, between

class CoomiaDipUser(HttpUser):
    wait_time = between(0.1, 0.5)

    @task(5)
    def query_orders(self):
        self.client.post("/api/oql", json={
            "query": "SELECT orderId FROM Order WHERE status='ACTIVE' LIMIT 20"
        })

    @task(1)
    def execute_action(self):
        self.client.post("/api/actions/UpdateStatus", json={"orderId": "o-1"})

#7.2 Performance Regression Gates

Run benchmarks before every release
Block release if P99 latency regresses > 20% vs previous version
Continuously monitor production performance trends

#7.3 Performance Targets

Operation	P50	P95	P99	TPS
Object read	< 5ms	< 20ms	< 50ms	> 5000
Object write	< 10ms	< 50ms	< 100ms	> 2000
Simple OQL	< 20ms	< 100ms	< 200ms	> 1000
Complex JOIN	< 100ms	< 500ms	< 1s	> 200

#Summary

This guide covered coomia-dip performance tuning: diagnosis methodology, OQL query optimization, Doris table design and materialized views, gRPC connection pools and retry configuration, JVM GC tuning, multi-level caching, and load testing. Remember: performance tuning is iterative -- identify the bottleneck first, optimize specifically, then validate the results.

Previous: [S12-19] Production Deployment Checklist