nsjail Sandbox: Security Isolation for Function Runtime
User-defined functions run inside the platform -- without isolation, malicious or buggy code could compromise the entire system. coomia-dip uses Google's open-source nsjail for process-level sandbox isolation, employing four isolation layers: Linux Namespaces, seccomp-bpf, cgroups, and chroot to confine each function execution within strict security boundaries. This article details nsjail configuration strategies, FunctionRuntime integration, resource monitoring, escape prevention, and comparative analysis with gVisor/Docker.
“Series: S5 Intelligent Decisions · Article 18 | Level: Advanced | Reading Time: 20 min
nsjail Sandbox: Security Isolation for Function Runtime
#TL;DR
User-defined functions run inside the platform -- without isolation, malicious or buggy code could compromise the entire system. coomia-dip uses Google's open-source nsjail for process-level sandbox isolation, employing four isolation layers: Linux Namespaces, seccomp-bpf, cgroups, and chroot to confine each function execution within strict security boundaries. This article details nsjail configuration strategies, FunctionRuntime integration, resource monitoring, escape prevention, and comparative analysis with gVisor/Docker.
#1. Why nsjail
#1.1 Sandbox Comparison
| Dimension | nsjail | gVisor | Docker |
|---|---|---|---|
| Cold start | 5-10ms | 50-100ms | 200-500ms |
| Memory overhead | ~5MB | ~30MB | ~50MB |
| Best for | Short-lived functions | Long-running services | Full applications |
| Config complexity | Medium | Low | Low |
| Syscall filtering | seccomp-bpf | Kernel replacement | seccomp |
#1.2 Four Isolation Layers
nsjail Four-Layer Security Isolation:
Layer 1: Linux Namespaces
- PID NS: Isolate process tree
- NET NS: Isolate network stack
- MNT NS: Isolate filesystem mounts
- UTS NS: Isolate hostname
- IPC NS: Isolate IPC
- USER NS: Isolate user/group IDs
Layer 2: seccomp-bpf
- Allow only whitelisted syscalls
- Block: ptrace, mount, reboot
- Block: socket (optional)
Layer 3: cgroups v2
- CPU time limits
- Memory caps
- PID count limits
- I/O bandwidth limits
Layer 4: chroot / pivot_root
- Read-only root filesystem
- Minimal mount points
- No /proc, /sys access
#2. nsjail Configuration
#2.1 Python Function Sandbox Config
# nsjail config: python-sandbox.cfg
name: "python-function-sandbox"
description: "Sandbox for Python user-defined functions"
mode: ONCE
time_limit: 30
rlimit_as_type: HARD
rlimit_as: 512
rlimit_cpu_type: HARD
rlimit_cpu: 10
rlimit_fsize_type: HARD
rlimit_fsize: 10
rlimit_nofile_type: HARD
rlimit_nofile: 64
clone_newnet: true
clone_newuser: true
clone_newns: true
clone_newpid: true
clone_newipc: true
clone_newuts: true
uidmap {
inside_id: "1000"
outside_id: "65534"
count: 1
}
gidmap {
inside_id: "1000"
outside_id: "65534"
count: 1
}
mount {
src: "/usr"
dst: "/usr"
is_bind: true
rw: false
}
mount {
src: "/lib"
dst: "/lib"
is_bind: true
rw: false
}
mount {
src: "/lib64"
dst: "/lib64"
is_bind: true
rw: false
}
mount {
dst: "/tmp"
fstype: "tmpfs"
rw: true
options: "size=50m"
}
mount {
dst: "/dev"
fstype: "tmpfs"
rw: false
}
seccomp_string: "POLICY sandbox {"
seccomp_string: " ALLOW {"
seccomp_string: " read, write, close, fstat, lseek,"
seccomp_string: " mmap, mprotect, munmap, brk,"
seccomp_string: " ioctl, access, pipe, select,"
seccomp_string: " clone, execve, wait4, exit_group,"
seccomp_string: " openat, getdents64, fcntl,"
seccomp_string: " futex, set_tid_address, set_robust_list,"
seccomp_string: " rt_sigaction, rt_sigprocmask,"
seccomp_string: " getpid, getuid, getgid, gettid,"
seccomp_string: " arch_prctl, prlimit64, getrandom"
seccomp_string: " }"
seccomp_string: " DENY {"
seccomp_string: " ptrace, mount, reboot, swapon,"
seccomp_string: " init_module, delete_module,"
seccomp_string: " socket, connect, bind, listen"
seccomp_string: " }"
seccomp_string: "}"
cgroup_mem_max: 268435456
cgroup_pids_max: 32
cgroup_cpu_ms_per_sec: 500
#3. nsjail Integration Layer
#3.1 NsjailExecutor
from __future__ import annotations
import subprocess
import json
import tempfile
import os
import time
from dataclasses import dataclass
from typing import Any
@dataclass
class NsjailConfig:
config_path: str
nsjail_binary: str = "/usr/bin/nsjail"
work_dir: str = "/tmp/nsjail-work"
log_level: str = "WARNING"
@dataclass
class ExecutionResult:
stdout: str
stderr: str
exit_code: int
duration_ms: float
memory_peak_mb: float = 0.0
cpu_time_ms: float = 0.0
killed_by_signal: int = 0
oom_killed: bool = False
class NsjailExecutor:
"""nsjail sandbox executor"""
def __init__(self, config: NsjailConfig):
self._config = config
self._verify_nsjail()
def _verify_nsjail(self) -> None:
result = subprocess.run(
[self._config.nsjail_binary, "--version"],
capture_output=True, text=True,
)
if result.returncode != 0:
raise RuntimeError("nsjail not found or not executable")
def execute(self, command: list[str],
env: dict[str, str] | None = None,
stdin_data: str | None = None,
extra_mounts: list[tuple[str, str, bool]] | None = None,
timeout_seconds: float = 30.0) -> ExecutionResult:
start = time.monotonic()
cmd = [
self._config.nsjail_binary,
"--config", self._config.config_path,
"--log_level", self._config.log_level,
]
if extra_mounts:
for src, dst, rw in extra_mounts:
cmd.extend(["--bindmount", f"{src}:{dst}"])
if env:
for key, value in env.items():
cmd.extend(["--env", f"{key}={value}"])
cmd.append("--")
cmd.extend(command)
try:
result = subprocess.run(
cmd, input=stdin_data,
capture_output=True, text=True,
timeout=timeout_seconds,
)
duration = (time.monotonic() - start) * 1000
return ExecutionResult(
stdout=result.stdout,
stderr=result.stderr,
exit_code=result.returncode,
duration_ms=duration,
)
except subprocess.TimeoutExpired:
duration = (time.monotonic() - start) * 1000
return ExecutionResult(
stdout="", stderr="Execution timeout",
exit_code=-1, duration_ms=duration,
killed_by_signal=9,
)
#3.2 FunctionRuntime Integration
class SandboxedPythonRuntime:
"""nsjail-based Python runtime"""
def __init__(self, nsjail_config: NsjailConfig):
self._executor = NsjailExecutor(nsjail_config)
self._venv_cache: dict[str, str] = {}
async def initialize(self, spec) -> None:
venv_dir = f"/var/fn-venvs/{spec.function_id}/{spec.version}"
os.makedirs(venv_dir, exist_ok=True)
subprocess.run(
["python3", "-m", "venv", venv_dir],
check=True, timeout=30,
)
if spec.dependencies:
allowed = self._filter_allowed_deps(spec.dependencies)
pip = os.path.join(venv_dir, "bin", "pip")
subprocess.run(
[pip, "install", "--no-cache-dir"] + allowed,
check=True, timeout=120,
)
self._venv_cache[spec.function_id] = venv_dir
async def execute(self, spec, inputs: dict) -> dict:
wrapper = f"""
import json, sys
{spec.source_code}
inputs = json.loads(sys.stdin.read())
result = {spec.entry_point}(**inputs)
print(json.dumps(result))
"""
with tempfile.NamedTemporaryFile(
mode="w", suffix=".py", delete=False, dir="/tmp"
) as f:
f.write(wrapper)
script_path = f.name
try:
venv_dir = self._venv_cache.get(spec.function_id)
result = self._executor.execute(
command=[
os.path.join(venv_dir, "bin", "python") if venv_dir else "/usr/bin/python3",
"/sandbox/script.py",
],
stdin_data=json.dumps(inputs),
extra_mounts=[
(script_path, "/sandbox/script.py", False),
] + ([(venv_dir, venv_dir, False)] if venv_dir else []),
timeout_seconds=spec.resource_limits.max_execution_time_ms / 1000,
)
if result.exit_code != 0:
raise RuntimeError(f"Sandbox execution failed: {result.stderr}")
if result.oom_killed:
raise MemoryError("Function killed due to OOM")
return json.loads(result.stdout)
finally:
os.unlink(script_path)
def _filter_allowed_deps(self, deps: list[str]) -> list[str]:
allowlist = {
"numpy", "pandas", "scipy", "scikit-learn",
"requests", "httpx", "pydantic", "ortools",
"jsonschema", "pyyaml", "python-dateutil",
}
allowed = []
for dep in deps:
name = dep.split("==")[0].split(">=")[0].split("<=")[0].lower()
if name in allowlist:
allowed.append(dep)
return allowed
#4. Resource Monitoring
class CgroupMonitor:
"""cgroups v2 resource monitor"""
def __init__(self, cgroup_path: str = "/sys/fs/cgroup"):
self._base = cgroup_path
def get_memory_usage(self, cgroup_name: str) -> dict:
path = os.path.join(self._base, cgroup_name)
current = self._read_int(os.path.join(path, "memory.current"))
peak = self._read_int(os.path.join(path, "memory.peak"))
limit = self._read_int(os.path.join(path, "memory.max"))
return {
"current_mb": current / (1024 * 1024),
"peak_mb": peak / (1024 * 1024),
"limit_mb": limit / (1024 * 1024) if limit < 2**62 else "unlimited",
"usage_pct": (current / limit * 100) if limit < 2**62 else 0,
}
def get_cpu_usage(self, cgroup_name: str) -> dict:
path = os.path.join(self._base, cgroup_name)
stat_content = self._read_file(os.path.join(path, "cpu.stat"))
stats = {}
for line in stat_content.strip().split("\n"):
key, value = line.split()
stats[key] = int(value)
return {
"user_usec": stats.get("user_usec", 0),
"system_usec": stats.get("system_usec", 0),
"total_usec": stats.get("usage_usec", 0),
}
def _read_int(self, path: str) -> int:
try:
with open(path) as f:
content = f.read().strip()
return int(content) if content != "max" else 2**63
except (FileNotFoundError, ValueError):
return 0
def _read_file(self, path: str) -> str:
try:
with open(path) as f:
return f.read()
except FileNotFoundError:
return ""
#5. Security Hardening
#5.1 seccomp-bpf Policy Builder
class SeccompPolicyBuilder:
"""seccomp-bpf policy builder"""
SAFE_SYSCALLS = {
"read", "write", "close", "fstat", "lseek",
"mmap", "mprotect", "munmap", "brk",
"ioctl", "access", "pipe", "select", "poll",
"clone", "fork", "vfork", "execve",
"wait4", "exit", "exit_group",
"openat", "getdents64", "fcntl",
"futex", "set_tid_address", "set_robust_list",
"rt_sigaction", "rt_sigprocmask", "rt_sigreturn",
"getpid", "getppid", "getuid", "getgid", "gettid",
"arch_prctl", "prlimit64", "getrandom",
"clock_gettime", "clock_getres", "nanosleep",
}
DANGEROUS_SYSCALLS = {
"ptrace", "mount", "umount2", "reboot",
"swapon", "swapoff",
"init_module", "delete_module", "finit_module",
"kexec_load", "kexec_file_load",
"pivot_root", "chroot",
"setns", "unshare",
}
@classmethod
def build_policy(cls, allow_network: bool = False) -> str:
allowed = set(cls.SAFE_SYSCALLS)
if allow_network:
allowed.update({"socket", "connect", "bind", "listen",
"accept", "sendto", "recvfrom",
"setsockopt", "getsockopt"})
lines = [
"POLICY sandbox {",
" ALLOW {",
" " + ", ".join(sorted(allowed)),
" }",
" DENY {",
" " + ", ".join(sorted(cls.DANGEROUS_SYSCALLS)),
" }",
"}",
]
return "\n".join(lines)
#5.2 Escape Detection
class EscapeDetector:
"""Sandbox escape detection"""
@staticmethod
def scan_source_code(code: str) -> list[str]:
import re
dangers = []
patterns = [
(r"import\s+os\b", "Importing os module"),
(r"import\s+subprocess\b", "Importing subprocess module"),
(r"import\s+ctypes\b", "Importing ctypes module"),
(r"__import__\s*\(", "Using __import__"),
(r"eval\s*\(", "Using eval"),
(r"exec\s*\(", "Using exec"),
(r"open\s*\(.*/etc/", "Accessing /etc"),
(r"open\s*\(.*/proc/", "Accessing /proc"),
(r"os\s*\.\s*system", "Using os.system"),
(r"subprocess\s*\.\s*", "Using subprocess"),
]
for pattern, description in patterns:
if re.search(pattern, code):
dangers.append(description)
return dangers
#6. Sandbox Pool
import asyncio
from collections import deque
class SandboxPool:
"""Sandbox pool: warm and reuse sandbox environments"""
def __init__(self, nsjail_config: NsjailConfig, pool_size: int = 10):
self._config = nsjail_config
self._pool_size = pool_size
self._available: deque[NsjailExecutor] = deque()
self._in_use: set[int] = set()
self._lock = asyncio.Lock()
async def initialize(self) -> None:
for _ in range(self._pool_size):
executor = NsjailExecutor(self._config)
self._available.append(executor)
async def acquire(self) -> NsjailExecutor:
async with self._lock:
if self._available:
executor = self._available.popleft()
self._in_use.add(id(executor))
return executor
else:
executor = NsjailExecutor(self._config)
self._in_use.add(id(executor))
return executor
async def release(self, executor: NsjailExecutor) -> None:
async with self._lock:
self._in_use.discard(id(executor))
if len(self._available) < self._pool_size:
self._available.append(executor)
#7. Performance Optimization
#7.1 Cold Start Breakdown
Sandbox Startup Latency Breakdown:
Operation | Baseline | Optimized
---------------------|----------|----------
nsjail process spawn | 2ms | 2ms
Namespace creation | 3ms | 3ms
chroot setup | 1ms | 1ms
seccomp load | 0.5ms | 0.5ms
cgroup config | 0.5ms | 0.5ms (pre-created)
Python interpreter | 200ms | 50ms (pyc precompiled)
Dependency loading | 300ms | 0ms (venv warmed)
---------------------|----------|----------
Total | ~507ms | ~57ms
#8. Monitoring Dashboard
nsjail Sandbox Monitor:
Active Sandboxes: 23/50
+---------------------------+
| Python ============ 15 |
| Node.js ==== 5 |
| WASM == 3 |
+---------------------------+
Resource Usage:
+---------------------------+
| Avg Memory: 45MB / 256MB |
| Avg CPU: 120ms / 5000ms|
| OOM Kills: 2 (0.3%) |
| Timeouts: 5 (0.8%) |
+---------------------------+
Security Events (24h):
+---------------------------+
| seccomp violations: 3 |
| escape attempts: 0 |
| code scan warnings: 12 |
+---------------------------+
#9. Hybrid Sandbox Strategy
class HybridSandboxManager:
"""Hybrid sandbox: nsjail + gVisor on demand"""
def __init__(self, nsjail_config: NsjailConfig, gvisor_config: dict):
self._nsjail = NsjailExecutor(nsjail_config)
self._gvisor_config = gvisor_config
def select_sandbox(self, spec) -> str:
if spec.resource_limits.network_enabled:
return "gvisor"
if spec.resource_limits.max_execution_time_ms > 60000:
return "gvisor"
if spec.resource_limits.max_memory_mb > 512:
return "gvisor"
return "nsjail"
#10. Practical Example
# Scenario: Safely execute user-defined function in nsjail sandbox
config = NsjailConfig(
config_path="/etc/nsjail/python-sandbox.cfg",
nsjail_binary="/usr/bin/nsjail",
)
runtime = SandboxedPythonRuntime(config)
spec = FunctionSpec(
function_id="fn-risk-score",
name="risk_scorer",
language=FunctionLanguage.PYTHON,
version="1.0.0",
entry_point="calculate_risk",
source_code="""
import math
def calculate_risk(credit_score, income, debt_ratio):
base = credit_score / 850
income_factor = math.log(income + 1) / math.log(1000001)
risk = 1.0 - (base * 0.5 + income_factor * 0.3 + (1 - debt_ratio) * 0.2)
return {"risk_score": round(risk, 4), "risk_level": "high" if risk > 0.7 else "low"}
""",
dependencies=["numpy"],
resource_limits=ResourceLimits(
max_memory_mb=128,
max_execution_time_ms=5000,
network_enabled=False,
),
)
issues = EscapeDetector.validate_before_execution(spec)
assert len(issues) == 0
await runtime.initialize(spec)
result = await runtime.execute(spec, {
"credit_score": 650, "income": 200000, "debt_ratio": 0.45,
})
# result = {"risk_score": 0.3821, "risk_level": "low"}
#Key Takeaways
- Four-layer isolation (Namespaces + seccomp + cgroups + chroot) provides process-level security
- 5-10ms cold start is far superior to Docker/gVisor for short-lived functions
- seccomp-bpf whitelist precisely controls allowed syscalls, blocking privilege escalation
- cgroups v2 limits memory, CPU, and PID count to prevent resource exhaustion attacks
- Static code scanning detects dangerous patterns before execution
- Sandbox pool warm-up reduces cold start overhead through reuse
- Hybrid strategy uses nsjail for short functions, gVisor for long-running and network-dependent scenarios
#Next Article
Next up: S5-19 WASM Runtime: WebAssembly in the Decision Engine details how coomia-dip uses WASM for near-native-speed secure function execution.
tags: #nsjail #sandbox #security #namespaces #seccomp #cgroups #isolation #coomia-dip