Back to Blog

nsjail Sandbox: Security Isolation for Function Runtime

User-defined functions run inside the platform -- without isolation, malicious or buggy code could compromise the entire system. coomia-dip uses Google's open-source nsjail for process-level sandbox isolation, employing four isolation layers: Linux Namespaces, seccomp-bpf, cgroups, and chroot to confine each function execution within strict security boundaries. This article details nsjail configuration strategies, FunctionRuntime integration, resource monitoring, escape prevention, and comparative analysis with gVisor/Docker.

CoomiaPublished on September 10, 202510 min read
Share this articleTwitter / X

Series: S5 Intelligent Decisions · Article 18 | Level: Advanced | Reading Time: 20 min

nsjail Sandbox: Security Isolation for Function Runtime

#TL;DR

User-defined functions run inside the platform -- without isolation, malicious or buggy code could compromise the entire system. coomia-dip uses Google's open-source nsjail for process-level sandbox isolation, employing four isolation layers: Linux Namespaces, seccomp-bpf, cgroups, and chroot to confine each function execution within strict security boundaries. This article details nsjail configuration strategies, FunctionRuntime integration, resource monitoring, escape prevention, and comparative analysis with gVisor/Docker.

#1. Why nsjail

#1.1 Sandbox Comparison

DimensionnsjailgVisorDocker
Cold start5-10ms50-100ms200-500ms
Memory overhead~5MB~30MB~50MB
Best forShort-lived functionsLong-running servicesFull applications
Config complexityMediumLowLow
Syscall filteringseccomp-bpfKernel replacementseccomp

#1.2 Four Isolation Layers

Code
nsjail Four-Layer Security Isolation:

  Layer 1: Linux Namespaces
  - PID NS: Isolate process tree
  - NET NS: Isolate network stack
  - MNT NS: Isolate filesystem mounts
  - UTS NS: Isolate hostname
  - IPC NS: Isolate IPC
  - USER NS: Isolate user/group IDs

  Layer 2: seccomp-bpf
  - Allow only whitelisted syscalls
  - Block: ptrace, mount, reboot
  - Block: socket (optional)

  Layer 3: cgroups v2
  - CPU time limits
  - Memory caps
  - PID count limits
  - I/O bandwidth limits

  Layer 4: chroot / pivot_root
  - Read-only root filesystem
  - Minimal mount points
  - No /proc, /sys access

#2. nsjail Configuration

#2.1 Python Function Sandbox Config

PROTOBUF
# nsjail config: python-sandbox.cfg

name: "python-function-sandbox"
description: "Sandbox for Python user-defined functions"

mode: ONCE
time_limit: 30
rlimit_as_type: HARD
rlimit_as: 512
rlimit_cpu_type: HARD
rlimit_cpu: 10
rlimit_fsize_type: HARD
rlimit_fsize: 10
rlimit_nofile_type: HARD
rlimit_nofile: 64

clone_newnet: true
clone_newuser: true
clone_newns: true
clone_newpid: true
clone_newipc: true
clone_newuts: true

uidmap {
    inside_id: "1000"
    outside_id: "65534"
    count: 1
}

gidmap {
    inside_id: "1000"
    outside_id: "65534"
    count: 1
}

mount {
    src: "/usr"
    dst: "/usr"
    is_bind: true
    rw: false
}

mount {
    src: "/lib"
    dst: "/lib"
    is_bind: true
    rw: false
}

mount {
    src: "/lib64"
    dst: "/lib64"
    is_bind: true
    rw: false
}

mount {
    dst: "/tmp"
    fstype: "tmpfs"
    rw: true
    options: "size=50m"
}

mount {
    dst: "/dev"
    fstype: "tmpfs"
    rw: false
}

seccomp_string: "POLICY sandbox {"
seccomp_string: "  ALLOW {"
seccomp_string: "    read, write, close, fstat, lseek,"
seccomp_string: "    mmap, mprotect, munmap, brk,"
seccomp_string: "    ioctl, access, pipe, select,"
seccomp_string: "    clone, execve, wait4, exit_group,"
seccomp_string: "    openat, getdents64, fcntl,"
seccomp_string: "    futex, set_tid_address, set_robust_list,"
seccomp_string: "    rt_sigaction, rt_sigprocmask,"
seccomp_string: "    getpid, getuid, getgid, gettid,"
seccomp_string: "    arch_prctl, prlimit64, getrandom"
seccomp_string: "  }"
seccomp_string: "  DENY {"
seccomp_string: "    ptrace, mount, reboot, swapon,"
seccomp_string: "    init_module, delete_module,"
seccomp_string: "    socket, connect, bind, listen"
seccomp_string: "  }"
seccomp_string: "}"

cgroup_mem_max: 268435456
cgroup_pids_max: 32
cgroup_cpu_ms_per_sec: 500

#3. nsjail Integration Layer

#3.1 NsjailExecutor

Python
from __future__ import annotations
import subprocess
import json
import tempfile
import os
import time
from dataclasses import dataclass
from typing import Any


@dataclass
class NsjailConfig:
    config_path: str
    nsjail_binary: str = "/usr/bin/nsjail"
    work_dir: str = "/tmp/nsjail-work"
    log_level: str = "WARNING"


@dataclass
class ExecutionResult:
    stdout: str
    stderr: str
    exit_code: int
    duration_ms: float
    memory_peak_mb: float = 0.0
    cpu_time_ms: float = 0.0
    killed_by_signal: int = 0
    oom_killed: bool = False


class NsjailExecutor:
    """nsjail sandbox executor"""

    def __init__(self, config: NsjailConfig):
        self._config = config
        self._verify_nsjail()

    def _verify_nsjail(self) -> None:
        result = subprocess.run(
            [self._config.nsjail_binary, "--version"],
            capture_output=True, text=True,
        )
        if result.returncode != 0:
            raise RuntimeError("nsjail not found or not executable")

    def execute(self, command: list[str],
                env: dict[str, str] | None = None,
                stdin_data: str | None = None,
                extra_mounts: list[tuple[str, str, bool]] | None = None,
                timeout_seconds: float = 30.0) -> ExecutionResult:
        start = time.monotonic()

        cmd = [
            self._config.nsjail_binary,
            "--config", self._config.config_path,
            "--log_level", self._config.log_level,
        ]

        if extra_mounts:
            for src, dst, rw in extra_mounts:
                cmd.extend(["--bindmount", f"{src}:{dst}"])

        if env:
            for key, value in env.items():
                cmd.extend(["--env", f"{key}={value}"])

        cmd.append("--")
        cmd.extend(command)

        try:
            result = subprocess.run(
                cmd, input=stdin_data,
                capture_output=True, text=True,
                timeout=timeout_seconds,
            )

            duration = (time.monotonic() - start) * 1000
            return ExecutionResult(
                stdout=result.stdout,
                stderr=result.stderr,
                exit_code=result.returncode,
                duration_ms=duration,
            )

        except subprocess.TimeoutExpired:
            duration = (time.monotonic() - start) * 1000
            return ExecutionResult(
                stdout="", stderr="Execution timeout",
                exit_code=-1, duration_ms=duration,
                killed_by_signal=9,
            )

#3.2 FunctionRuntime Integration

Python
class SandboxedPythonRuntime:
    """nsjail-based Python runtime"""

    def __init__(self, nsjail_config: NsjailConfig):
        self._executor = NsjailExecutor(nsjail_config)
        self._venv_cache: dict[str, str] = {}

    async def initialize(self, spec) -> None:
        venv_dir = f"/var/fn-venvs/{spec.function_id}/{spec.version}"
        os.makedirs(venv_dir, exist_ok=True)

        subprocess.run(
            ["python3", "-m", "venv", venv_dir],
            check=True, timeout=30,
        )

        if spec.dependencies:
            allowed = self._filter_allowed_deps(spec.dependencies)
            pip = os.path.join(venv_dir, "bin", "pip")
            subprocess.run(
                [pip, "install", "--no-cache-dir"] + allowed,
                check=True, timeout=120,
            )

        self._venv_cache[spec.function_id] = venv_dir

    async def execute(self, spec, inputs: dict) -> dict:
        wrapper = f"""
import json, sys

{spec.source_code}

inputs = json.loads(sys.stdin.read())
result = {spec.entry_point}(**inputs)
print(json.dumps(result))
"""
        with tempfile.NamedTemporaryFile(
            mode="w", suffix=".py", delete=False, dir="/tmp"
        ) as f:
            f.write(wrapper)
            script_path = f.name

        try:
            venv_dir = self._venv_cache.get(spec.function_id)

            result = self._executor.execute(
                command=[
                    os.path.join(venv_dir, "bin", "python") if venv_dir else "/usr/bin/python3",
                    "/sandbox/script.py",
                ],
                stdin_data=json.dumps(inputs),
                extra_mounts=[
                    (script_path, "/sandbox/script.py", False),
                ] + ([(venv_dir, venv_dir, False)] if venv_dir else []),
                timeout_seconds=spec.resource_limits.max_execution_time_ms / 1000,
            )

            if result.exit_code != 0:
                raise RuntimeError(f"Sandbox execution failed: {result.stderr}")
            if result.oom_killed:
                raise MemoryError("Function killed due to OOM")

            return json.loads(result.stdout)
        finally:
            os.unlink(script_path)

    def _filter_allowed_deps(self, deps: list[str]) -> list[str]:
        allowlist = {
            "numpy", "pandas", "scipy", "scikit-learn",
            "requests", "httpx", "pydantic", "ortools",
            "jsonschema", "pyyaml", "python-dateutil",
        }
        allowed = []
        for dep in deps:
            name = dep.split("==")[0].split(">=")[0].split("<=")[0].lower()
            if name in allowlist:
                allowed.append(dep)
        return allowed

#4. Resource Monitoring

Python
class CgroupMonitor:
    """cgroups v2 resource monitor"""

    def __init__(self, cgroup_path: str = "/sys/fs/cgroup"):
        self._base = cgroup_path

    def get_memory_usage(self, cgroup_name: str) -> dict:
        path = os.path.join(self._base, cgroup_name)
        current = self._read_int(os.path.join(path, "memory.current"))
        peak = self._read_int(os.path.join(path, "memory.peak"))
        limit = self._read_int(os.path.join(path, "memory.max"))

        return {
            "current_mb": current / (1024 * 1024),
            "peak_mb": peak / (1024 * 1024),
            "limit_mb": limit / (1024 * 1024) if limit < 2**62 else "unlimited",
            "usage_pct": (current / limit * 100) if limit < 2**62 else 0,
        }

    def get_cpu_usage(self, cgroup_name: str) -> dict:
        path = os.path.join(self._base, cgroup_name)
        stat_content = self._read_file(os.path.join(path, "cpu.stat"))
        stats = {}
        for line in stat_content.strip().split("\n"):
            key, value = line.split()
            stats[key] = int(value)

        return {
            "user_usec": stats.get("user_usec", 0),
            "system_usec": stats.get("system_usec", 0),
            "total_usec": stats.get("usage_usec", 0),
        }

    def _read_int(self, path: str) -> int:
        try:
            with open(path) as f:
                content = f.read().strip()
                return int(content) if content != "max" else 2**63
        except (FileNotFoundError, ValueError):
            return 0

    def _read_file(self, path: str) -> str:
        try:
            with open(path) as f:
                return f.read()
        except FileNotFoundError:
            return ""

#5. Security Hardening

#5.1 seccomp-bpf Policy Builder

Python
class SeccompPolicyBuilder:
    """seccomp-bpf policy builder"""

    SAFE_SYSCALLS = {
        "read", "write", "close", "fstat", "lseek",
        "mmap", "mprotect", "munmap", "brk",
        "ioctl", "access", "pipe", "select", "poll",
        "clone", "fork", "vfork", "execve",
        "wait4", "exit", "exit_group",
        "openat", "getdents64", "fcntl",
        "futex", "set_tid_address", "set_robust_list",
        "rt_sigaction", "rt_sigprocmask", "rt_sigreturn",
        "getpid", "getppid", "getuid", "getgid", "gettid",
        "arch_prctl", "prlimit64", "getrandom",
        "clock_gettime", "clock_getres", "nanosleep",
    }

    DANGEROUS_SYSCALLS = {
        "ptrace", "mount", "umount2", "reboot",
        "swapon", "swapoff",
        "init_module", "delete_module", "finit_module",
        "kexec_load", "kexec_file_load",
        "pivot_root", "chroot",
        "setns", "unshare",
    }

    @classmethod
    def build_policy(cls, allow_network: bool = False) -> str:
        allowed = set(cls.SAFE_SYSCALLS)
        if allow_network:
            allowed.update({"socket", "connect", "bind", "listen",
                           "accept", "sendto", "recvfrom",
                           "setsockopt", "getsockopt"})

        lines = [
            "POLICY sandbox {",
            "  ALLOW {",
            "    " + ", ".join(sorted(allowed)),
            "  }",
            "  DENY {",
            "    " + ", ".join(sorted(cls.DANGEROUS_SYSCALLS)),
            "  }",
            "}",
        ]
        return "\n".join(lines)

#5.2 Escape Detection

Python
class EscapeDetector:
    """Sandbox escape detection"""

    @staticmethod
    def scan_source_code(code: str) -> list[str]:
        import re
        dangers = []
        patterns = [
            (r"import\s+os\b", "Importing os module"),
            (r"import\s+subprocess\b", "Importing subprocess module"),
            (r"import\s+ctypes\b", "Importing ctypes module"),
            (r"__import__\s*\(", "Using __import__"),
            (r"eval\s*\(", "Using eval"),
            (r"exec\s*\(", "Using exec"),
            (r"open\s*\(.*/etc/", "Accessing /etc"),
            (r"open\s*\(.*/proc/", "Accessing /proc"),
            (r"os\s*\.\s*system", "Using os.system"),
            (r"subprocess\s*\.\s*", "Using subprocess"),
        ]
        for pattern, description in patterns:
            if re.search(pattern, code):
                dangers.append(description)
        return dangers

#6. Sandbox Pool

Python
import asyncio
from collections import deque


class SandboxPool:
    """Sandbox pool: warm and reuse sandbox environments"""

    def __init__(self, nsjail_config: NsjailConfig, pool_size: int = 10):
        self._config = nsjail_config
        self._pool_size = pool_size
        self._available: deque[NsjailExecutor] = deque()
        self._in_use: set[int] = set()
        self._lock = asyncio.Lock()

    async def initialize(self) -> None:
        for _ in range(self._pool_size):
            executor = NsjailExecutor(self._config)
            self._available.append(executor)

    async def acquire(self) -> NsjailExecutor:
        async with self._lock:
            if self._available:
                executor = self._available.popleft()
                self._in_use.add(id(executor))
                return executor
            else:
                executor = NsjailExecutor(self._config)
                self._in_use.add(id(executor))
                return executor

    async def release(self, executor: NsjailExecutor) -> None:
        async with self._lock:
            self._in_use.discard(id(executor))
            if len(self._available) < self._pool_size:
                self._available.append(executor)

#7. Performance Optimization

#7.1 Cold Start Breakdown

Code
Sandbox Startup Latency Breakdown:

  Operation            | Baseline | Optimized
  ---------------------|----------|----------
  nsjail process spawn | 2ms      | 2ms
  Namespace creation   | 3ms      | 3ms
  chroot setup         | 1ms      | 1ms
  seccomp load         | 0.5ms    | 0.5ms
  cgroup config        | 0.5ms    | 0.5ms (pre-created)
  Python interpreter   | 200ms    | 50ms (pyc precompiled)
  Dependency loading   | 300ms    | 0ms (venv warmed)
  ---------------------|----------|----------
  Total                | ~507ms   | ~57ms

#8. Monitoring Dashboard

Code
nsjail Sandbox Monitor:

  Active Sandboxes: 23/50
  +---------------------------+
  | Python  ============ 15   |
  | Node.js ====         5    |
  | WASM    ==           3    |
  +---------------------------+

  Resource Usage:
  +---------------------------+
  | Avg Memory: 45MB / 256MB  |
  | Avg CPU:    120ms / 5000ms|
  | OOM Kills:  2 (0.3%)      |
  | Timeouts:   5 (0.8%)      |
  +---------------------------+

  Security Events (24h):
  +---------------------------+
  | seccomp violations:  3    |
  | escape attempts:     0    |
  | code scan warnings: 12    |
  +---------------------------+

#9. Hybrid Sandbox Strategy

Python
class HybridSandboxManager:
    """Hybrid sandbox: nsjail + gVisor on demand"""

    def __init__(self, nsjail_config: NsjailConfig, gvisor_config: dict):
        self._nsjail = NsjailExecutor(nsjail_config)
        self._gvisor_config = gvisor_config

    def select_sandbox(self, spec) -> str:
        if spec.resource_limits.network_enabled:
            return "gvisor"
        if spec.resource_limits.max_execution_time_ms > 60000:
            return "gvisor"
        if spec.resource_limits.max_memory_mb > 512:
            return "gvisor"
        return "nsjail"

#10. Practical Example

Python
# Scenario: Safely execute user-defined function in nsjail sandbox

config = NsjailConfig(
    config_path="/etc/nsjail/python-sandbox.cfg",
    nsjail_binary="/usr/bin/nsjail",
)

runtime = SandboxedPythonRuntime(config)

spec = FunctionSpec(
    function_id="fn-risk-score",
    name="risk_scorer",
    language=FunctionLanguage.PYTHON,
    version="1.0.0",
    entry_point="calculate_risk",
    source_code="""
import math

def calculate_risk(credit_score, income, debt_ratio):
    base = credit_score / 850
    income_factor = math.log(income + 1) / math.log(1000001)
    risk = 1.0 - (base * 0.5 + income_factor * 0.3 + (1 - debt_ratio) * 0.2)
    return {"risk_score": round(risk, 4), "risk_level": "high" if risk > 0.7 else "low"}
""",
    dependencies=["numpy"],
    resource_limits=ResourceLimits(
        max_memory_mb=128,
        max_execution_time_ms=5000,
        network_enabled=False,
    ),
)

issues = EscapeDetector.validate_before_execution(spec)
assert len(issues) == 0

await runtime.initialize(spec)
result = await runtime.execute(spec, {
    "credit_score": 650, "income": 200000, "debt_ratio": 0.45,
})
# result = {"risk_score": 0.3821, "risk_level": "low"}

#Key Takeaways

  1. Four-layer isolation (Namespaces + seccomp + cgroups + chroot) provides process-level security
  2. 5-10ms cold start is far superior to Docker/gVisor for short-lived functions
  3. seccomp-bpf whitelist precisely controls allowed syscalls, blocking privilege escalation
  4. cgroups v2 limits memory, CPU, and PID count to prevent resource exhaustion attacks
  5. Static code scanning detects dangerous patterns before execution
  6. Sandbox pool warm-up reduces cold start overhead through reuse
  7. Hybrid strategy uses nsjail for short functions, gVisor for long-running and network-dependent scenarios

#Next Article

Next up: S5-19 WASM Runtime: WebAssembly in the Decision Engine details how coomia-dip uses WASM for near-native-speed secure function execution.

tags: #nsjail #sandbox #security #namespaces #seccomp #cgroups #isolation #coomia-dip