15 Commits (master)

Author	SHA1	Message	Date
Pavel Punsky	8c7d8fcb86	Enable --udp-recvmmsg by default on Linux (#1930) ... ## Summary Flips the Linux default for `--udp-recvmmsg` from **off** to **on**. Operators opt out with `--udp-recvmmsg=false` (or `=0`). > **Stacked on #1929.** This depends on the recvmmsg-scoping change in #1929 and is based on that branch, so the diff shows only the default-on change. GitHub will auto-retarget the base to `master` once #1929 merges. Merge #1929 first. ## Why this is now safe The original objection to default-on (recorded in `docs/PerformanceIterationLog.md`) was the **per-session-relay-socket prealloc tax**: `--udp-recvmmsg` applied the 16-buffer batch path to every connected relay socket, which only ever carries one flow, so the churn ate the listener-side win. #1929 scoped recvmmsg to **shared fan-in sockets only** (`udp_recvmmsg_eligible`: the client listener, plus the per-thread shared relay socket under `--multiplex-peer`). Per-session relay sockets now stay on the single-recv path regardless of the flag, so that tax is gone. The one socket touched by default — the client listener — is a genuine fan-in point: - batches whenever client concurrency is non-trivial (measured `avg_batch ≈ 16` under load), and - costs little when idle (few packets ⇒ few prealloc cycles). ## What changed - `mainrelay.c`: `turn_params.udp_recvmmsg` default `false → true` (Linux only). - Removed the now-dead `--multiplex-peer` auto-enable block and the `udp_recvmmsg_set_explicitly` tracking it relied on; multiplex-peer gets its recvmmsg window from the default. The opt-out flows through the normal `get_bool_value` path. - Help text, `man/man1/turnserver.1`, `examples/etc/turnserver.conf`, `CLAUDE.md`, and `docs/PerformanceIterationLog.md` updated for the new default + opt-out. Per-session relay sockets and DTLS session sockets are unchanged. ## Validation - **Format:** clang-format 15.0.7 clean. - **macOS:** build + ctest 6/6 + `run_tests.sh` pass. - **Linux (Docker, clean build):** ctest 5/5; `run_tests.sh`, `run_tests_conf.sh`, `run_tests_multiplex_peer.sh` all pass (no FAIL). - **Runtime proof (loopback, `--udp-recvmmsg-log`):** - Default, no flag: recvmmsg active, `calls=13714 packets=219306 avg_batch=15.99`. - `--udp-recvmmsg=false`: zero recvmmsg activity — opt-out confirmed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	18 hours ago
Pavel Punsky	b17f5c482f	Add --udp-sendmmsg-log to observe egress sendmmsg/UDP-GSO batching (#1927) ... ## Summary Adds a Linux-only `--udp-sendmmsg-log` flag (mirroring `--udp-recvmmsg-log`) that logs per-relay-thread **egress** batch statistics every 10 s: flush count, total datagrams, average batch occupancy, UDP-GSO engagement (`gso_flushes`/`gso_datagrams`/`gso_frac`), and a per-flush occupancy histogram. ## Motivation `--multiplex-peer` enables `sendmmsg`/UDP-GSO coalescing on the egress path, but there was no way to see whether it actually coalesces anything. While investigating this I confirmed a non-obvious property worth documenting: - Per-session UDP **client** sockets are children of the listener (`parent_s`), and `udp_send_fd()` returns the shared **listener fd** for all of them. Combined with the `recvmmsg`-driven batch window, relay→client **downlink** sends to *different clients* already coalesce into one `sendmmsg` on the listener fd — the listener fd is effectively a shared client-facing send socket. (In non-multiplex mode each allocation has its own relay socket, so a `recvmmsg` drain spans only one session and the downlink batch is a singleton — cross-client batching genuinely requires `--multiplex-peer`.) - UDP-GSO only engages when destination **and** segment size match across a batch, so at low per-flow packet rates (VoIP-style, tens of pps per flow) it rarely fires. This flag makes both effects measurable instead of assumed. ## Example Captured under a `--multiplex-peer --udp-gso` load: ``` udp-sendmmsg stats: flushes=21 datagrams=27 avg_batch=1.29 gso_flushes=0 \ gso_datagrams=0 gso_frac=0.000 hist_1=17 hist_2=2 hist_3_4=2 hist_5_8=0 hist_9_16=0 hist_17_32=0 ``` - `avg_batch` — mean datagrams per flush (1.0 = no coalescing) - `gso_frac` — fraction of datagrams sent via UDP-GSO (~0 = GSO not earning its keep) - `hist_*` — per-flush occupancy histogram Both rise with aggregate pps and sit near the floor on lightly loaded servers. ## Implementation - Counters on `ioa_engine` (behind `#if defined(__linux__)`), bumped once per flush in `udp_sendmmsg_flush()` and once per `recvmmsg` call — no per-datagram cost. - New `--udp-sendmmsg-log` CLI flag + 10 s periodic logger in the engine timer, mirroring the existing recvmmsg stats path. - Docs: a new "Egress Batching (sendmmsg / UDP-GSO) and Observability" section in `docs/multiplex-peer.md`. ## Testing - macOS: build + `ctest` (6/6) + `run_tests.sh` pass (instrumentation is `#if __linux__`, validating flag plumbing + the non-Linux stub path). - Linux (clean Ubuntu 24.04 Docker): build + `ctest` (4/4) + `run_tests.sh` / `run_tests_conf.sh` / `run_tests_multiplex_peer.sh` pass, and a `--multiplex-peer --udp-gso --udp-sendmmsg-log` load run emitted the stats line above.	22 hours ago
Pavel Punsky	e0c37a3aa0	Multiplexpeer (#1916) ... ## Summary Adds **`--multiplex-peer`**, a non-standard relay mode that replaces the per-allocation peer-side port bind with **one shared IPv4+IPv6 UDP socket pair per relay thread**. Sessions are demultiplexed by exact peer IP:port in a per-thread `mp_table`. This lifts the ~16 k allocation cap that the default 49152-65535 relay port range imposes, and dramatically reduces kernel-level UDP receive-buffer drops under high pps. Design and trade-offs: [docs/multiplex-peer.md](docs/multiplex-peer.md). ## What changes ### Server (turnserver) - **`--multiplex-peer`** (cross-platform) — enable the shared per-thread relay sockets. Replaces the per-session port bind. Implies sendmmsg batching on Linux and default-enables `--udp-recvmmsg` (override with `--udp-recvmmsg=0`). Incompatible with EVEN-PORT — those Allocates are rejected with 400. - **`--multiplex-peer-port <port>`** (cross-platform, default 3480) — base port; thread `i` binds `<base>+2i` (IPv4) and `<base>+2i+1` (IPv6). A 4-thread server consumes 8 ports. - **`--udp-gso`** (Linux-only CLI) — UDP-GSO (`UDP_SEGMENT` cmsg) on the relay send path. Requires `--multiplex-peer` (which is what enables the sendmmsg batching GSO piggybacks on); passing `--udp-gso` alone is a silent no-op. - **CLI surface tightened**: `--udp-recvmmsg`, `--udp-recvmmsg-log`, `--udp-gso` and their fields are now `#if defined(__linux__)` — absent from `--help`, rejected with `unrecognized option`, and the code paths compile out on macOS/Windows. - **Windows portability**: `SO_REUSEPORT` in `mp_open_socket` wrapped in `#ifdef` (MSVC's Winsock doesn't define it; REUSEPORT was defensive anyway because the per-thread port layout is unique by construction). - **`--sock-buf-size` honoured at startup**: the shared multiplex-peer relay socket now calls `set_ioa_socket_buf_size` in `mp_open_socket` so the configured rcvbuf is in effect from the moment the socket exists, not deferred to the first Allocate. ### turnutils_uclient (loadgen) - **`--no-even-port`** — force `ep = -1` on Allocate. The default path randomly attaches EVEN-PORT (with no-R bit) even under `-c`, which `--multiplex-peer` strictly rejects with 400; this flag makes alloc-flood runs against multiplex-peer deterministic. - **Legacy `timer_handler` now wraps the per-tick send batch with `uclient_send_batch_begin/_end`** — without this, runs with `--sender-threads 0` (the default for `-m < 4`) silently fell through every send to plain `send(2)`. strace A/B: 205 k `sendto` → 61 k `sendmsg` (GSO) + 4 k `sendmmsg` + small `sendto` residual for control. ## Measured impact (3-droplet DigitalOcean, c-4 / 4 vCPU, 8 concurrent UDP streams, 45 s) | | baseline | `--udp-recvmmsg` | `--multiplex-peer` | `--multiplex-peer --udp-gso` | |---|---:|---:|---:|---:| | Server NIC rx pps (UDP relay both legs) | 350 k | 334 k | 326 k | 294 k | | Server `UdpInDatagrams` pps | 279 k | 292 k | 300 k | 294 k | | **Server `UdpRcvbufErrors` pps** | **71 k** | 42 k | 26 k | **0.3 k (−99.6 %)** | | **`turnserver` process CPU** | **387 %** | 205 % | 283 % | **133 % (−65 %)** | | Server host idle | 22 % | 49 % | 41 % | **68 %** | Same loadgen-side packet rate (~2 M pps reported by uclient `send_pps` after the legacy-path batching fix). Iteration log: [docs/PerformanceIterationLog.md](docs/PerformanceIterationLog.md). ## Test plan - [x] `ctest --test-dir build` — 3/3 pass (test_ioaddr, test_stun_msg, test_http_server) on macOS + Linux. - [x] `examples/run_tests.sh` — 4 protocols + 4 threaded + load-gen smoke on Linux; 4 protocols on macOS. - [x] `examples/run_tests_conf.sh` — same coverage, conf-driven. - [x] `examples/run_tests_multiplex_peer.sh` — UDP/TCP/TLS/DTLS via `--multiplex-peer --multiplex-peer-port=35000` on macOS + Linux. - [x] Flag matrix smoke on macOS: `--multiplex-peer`, `--multiplex-peer-port=42000`, `--multiplex-peer --udp-gso` (no-op), `uclient --no-even-port`, `uclient --listener-threads N --sender-threads M` — all pass; `--udp-recvmmsg` / `--udp-gso` correctly rejected with `unrecognized option`. - [x] Flag matrix smoke on Linux (Docker): same + `--udp-recvmmsg` accepted, `--multiplex-peer` auto-enables `--udp-recvmmsg`, `--udp-recvmmsg=0` overrides the auto-enable. - [x] Windows compile fix verified — `SO_REUSEPORT` no longer referenced unconditionally. - [x] 3-droplet perf matrix completed; per-hop UDP counters captured. ## Docs updated - New: [docs/multiplex-peer.md](docs/multiplex-peer.md) - [README.turnserver](README.turnserver): full entries for `--multiplex-peer`, `--multiplex-peer-port`, `--udp-gso`; clarified `--udp-recvmmsg` auto-enable semantics. - [README.turnutils](README.turnutils): added `--no-even-port`, plus previously-undocumented `--listener-threads` / `--sender-threads` loadgen pool flags. - [examples/etc/turnserver.conf](examples/etc/turnserver.conf): commented `udp-recvmmsg`, `udp-recvmmsg-log`, `udp-gso`, `multiplex-peer`, `multiplex-peer-port` keys with one-paragraph descriptions and pointer to `docs/multiplex-peer.md`. - Man pages regenerated via `./make-man.sh`. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2 weeks ago
Pavel Punsky	5959ecfb13	Add UDP-GSO send path (--udp-gso) (#1907) ... ## Summary - New `--udp-gso` flag (Linux, requires `--udp-sendmmsg`) collapses same-destination, same-size sendmmsg batches into a single `sendmsg` with a `UDP_SEGMENT` cmsg, so the kernel allocates one super-skb that traverses the network stack once and is segmented at egress instead of running `udp_sendmsg → ip_finish_output → __dev_queue_xmit` per datagram. - Also wraps the relay-side `recvmmsg` callback loop in `udp_sendmmsg_batch_begin/end` so peer→client sends triggered inside a recv batch can also coalesce — without that wrapping the relay path issues one `sendto` per delivered datagram. - Sticky-disable on `EINVAL/ENOPROTOOPT` for older kernels/NICs that lack UDP-GSO; one warning logged, then transparent fallback to the existing `sendmmsg` and `udp_send` paths. ## Why The `--udp-recvmmsg` and `--udp-sendmmsg` follow-ups confirmed (see [docs/PerformanceIterationLog.md](docs/PerformanceIterationLog.md)) that on the relay flood workload the dominant cost is the per-datagram kernel TX path. mmsg-style batching reduces only the syscall entry/exit, not the per-skb stack traversal — UDP-GSO collapses both. ## Result DigitalOcean nyc1 c-4, 30 s alternating A/B, `-Y packet -m 1`, eth1 TX as the authoritative server forwarding metric: | Variant | eth1 RX | eth1 TX | sys CPU | idle CPU | |---|---:|---:|---:|---:| | baseline (no flags) | 322,091 | 127,445 | 22.9 % | 67.5 % | | `--udp-recvmmsg --udp-sendmmsg --udp-gso` | 266,068 | **257,996** | 15.0 % | 78.7 % | | baseline (no flags) | 309,475 | 125,573 | 20.9 % | 70.7 % | | `--udp-recvmmsg --udp-sendmmsg --udp-gso` | 275,992 | **225,366** | 14.9 % | 74.3 % | Mean server forwarding rate: **126.5 k → 241.7 k pps (+91 %, 1.91×)**, mean system CPU **21.9 % → 14.9 %** — about **2.8× CPU efficiency** (TX pps per system-CPU-%). Full perf-children comparison and methodology in the new section of [docs/PerformanceIterationLog.md](docs/PerformanceIterationLog.md). ## Notes for reviewers - `--udp-gso` is opt-in and requires `--udp-sendmmsg` (the help text states the dependency). Without `--udp-sendmmsg` the batch state never accumulates and GSO has nothing to flush. - GSO eligibility resets on every `_begin/_end`. Mixed-destination, mixed-size, or oversize batches transparently fall back through `sendmmsg` / `udp_send`. - Rebased onto current `master`; the recvmmsg dependency is already merged via #1906. ## Test plan - [x] `cmake --build build --target turnserver` (RelWithDebInfo + ASan local builds clean) - [x] `ctest --test-dir build --output-on-failure` — 3/3 unit tests pass - [x] `examples/run_tests.sh` — TCP/TLS/UDP pass; DTLS pre-existing failure on macOS environment, unrelated to this change - [x] DigitalOcean A/B perf validation captured above - [ ] Reviewer to confirm CI green on Linux build/test/CodeQL --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	3 weeks ago
Pavel Punsky	a5005c4193	Relay recvmmsg (#1906) ... ## Summary Extends the existing Linux-only `--udp-recvmmsg` flag from the UDP listener socket to also cover **connected per-session UDP relay sockets**, so steady-state client→relay and peer→relay traffic on plain UDP is read in batches of up to 16 datagrams per `recvmmsg(2)` instead of one `recvmsg` per packet. DTLS sessions still go through the SSL read path and are unchanged. The flag stays **opt-in**: receive-side batching works correctly, but on the current `m=1` / `m=100` benchmarks throughput is flat to slightly negative — the bottleneck has moved past receive (see results below). ## What's in the change - **Shared receive helpers** (`src/apps/relay/ns_ioalib_engine_impl.c`, `src/apps/relay/ns_ioalib_impl.h`): - `ioa_parse_udp_recvmsg_cmsg()` — single TTL/TOS/`IP_RECVERR` cmsg parser used by both `udp_recvfrom()` and the new batch path. Replaces the duplicated parser previously inlined in `dtls_listener.c` and `udp_recvfrom()`. - `ioa_init_recvmmsg_hdr()` — single initializer for `mmsghdr`/`iovec`/cmsg/source-address fields, also used by the listener. - New `IOA_UDP_RECVMMSG_MAX_BATCH = 16` constant; both listener and relay paths now share it. - **Connected relay batch read** (`socket_udp_read_batch_recvmmsg` in `ns_ioalib_engine_impl.c`): called from `socket_input_worker` for non-SSL UDP sockets when `--udp-recvmmsg` is on. Allocates per-message `stun_buffer_list_elem`s, calls `recvmmsg(MSG_DONTWAIT)`, dispatches each datagram through the existing `read_cb` path, and falls back cleanly on `ENOSYS`/`EINVAL`/`EOPNOTSUPP` (auto-disables the flag) and on `EAGAIN`/short-batch (releases unused buffers). - **Per-engine scratch state**: the `mmsghdr[16]` / `iovec[16]` / cmsg / src-addr arrays live on `ioa_engine`, not on every socket — keeps memory flat at thousands of allocations. - **TTL/TOS-sized cmsg buffers** in the listener: the listener previously over-allocated `64 KiB` per slot; it now uses the same TTL+TOS sizing as the relay path. - **Opt-in occupancy stats** behind a new `--udp-recvmmsg-log` flag: every 10 s the relay logs `udp-recvmmsg stats: calls=… packets=… avg_batch=… wouldblock=… unavailable=… no_buffer=… hist_1=… hist_2=… hist_3_4=… hist_5_8=… hist_9_16=…`. Counters are always tracked (cheap); the periodic log is gated by the new flag so default operation is silent. - **CLI plumbing**: `--udp-recvmmsg-log` long option in `mainrelay.c`/`mainrelay.h`, `cli_print_flag` entry in `turn_admin_server.c`, doc updates in `README.turnserver`. - **Docs**: `docs/PerformanceIterationLog.md` records the iteration steps, validation, and two rounds of DigitalOcean A/B numbers. `CLAUDE.md` load-test instructions updated to mention the new flag and the `tot_recv_msgs` / `tot_recv_bytes` workaround.	3 weeks ago
Pavel Punsky	69bc0e7351	Load generator mode in turnutils_uclient (#1894) ... ## Summary Adds load-generator modes to `turnutils_uclient` for repeatable TURN server performance testing: - Adds `-Y packet|alloc|invalid` load modes. - Supports packet flood, allocation flood, and invalid-packet flood workflows. - Adds unique local client ports for allocation flood mode. - Removes default packet pacing in load-generator modes unless explicitly set. - Adds helper scripts under `examples/loadtest/`. - Documents load-test usage in `README.turnutils`, `man/man1/turnutils.1`, `CLAUDE.md`, and `docs/PerformanceIterationLog.md`. The performance log captures DigitalOcean benchmark methodology, A/B lessons, hot-path findings, and future optimization candidates.	4 weeks ago
Pavel Punsky	69aa0e4401	Keep only NEV_UDP_SOCKET_PER_THREAD network engine (#1849) ... Remove the two engine implementations (NEV_UDP_SOCKET_PER_SESSION and NEV_UDP_SOCKET_PER_ENDPOINT) and all the dispatch/selection logic around them. NEV_UDP_SOCKET_PER_THREAD is now the sole, unconditional implementation. - mainrelay.h: removed _NET_ENG_VERSION enum, typedef, and net_engine_version / net_engine_version_txt struct fields - mainrelay.c: removed NE_TYPE_OPT CLI option, set_network_engine(), per-endpoint branch in print_features(), and all remaining net_engine_version references - netengine.c: removed run_udp_listener_thread(), setup_socket_per_endpoint_udp_listener_servers() (~190 lines), setup_socket_per_session_udp_listener_servers() (~90 lines); simplified setup_barriers(), setup_relay_server(), run_general_relay_thread(), setup_general_relay_servers(), and setup_server() by eliminating all engine-type conditionals - turn_admin_server.c: replaced dynamic engine version lookups with hardcoded values (3 / "UDP thread per CPU core") in CLI and HTTPS status handlers --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2 months ago
Captain Picarl	a6028ddd8a	Improve PostgreSQL.md Clarity (#1833) ... A rewrite and reformat of the PostgreSQL.md file: - Applies some minor formatting to the document. - Applies consistency between the 'turn' user and 'coturn' database names. - Adds some warnings about issues I encountered during my own installation process (public schema error) - Adds a minor example of restarting a service with SystemD	3 months ago
Gustavo Garcia	ce56a78a7a	Add modules diagram to developer doc (#1685) ... Add basic diagram with modules of coturn inside Developer.md. I also wanted to move the existing Flow chart to mermaid so that our doc doesn't depend on external tools but i couldn't find the time to do it properly (horizontal arrows are hard to represent with mermaid) so i postponed that part. This is the result of the mermaid diagram: 	1 year ago
dominiquefournier	460cfa38af	Update SQLite.md (#1429) ... Markdown is now proper	2 years ago
Kang Lin	d33112f84a	Update FlowChart (#1377)	2 years ago
Kang Lin	05900adb2e	Doc: add flowchart (#1328) ... The flowchart will be kept updated: - Use [drawio](https://app.diagrams.net/) to edit. 1. Open [drawio](https://app.diagrams.net/) in brower 2. Menu → File → Open from ... → Device: Select [FlowChart.html](FlowChart.html) 3. Edit flow chart 4. Export to svg: Menu → File → Export as... → SVG...	2 years ago
vuittont60	da6bc6b0c4	Fix typos (#1345)	2 years ago
Kang Lin	294f897ccd	CMake: find prometheus(fix #1304) (#1315) ... fix #1304	3 years ago
Pavel Punsky	85c11b69e3	Move and split documentation files (#1096) ... This is not a long term documentation works - this is a temp proposal to modernize existing documentation - Create docs folder - Move existing documentation files into docs - Split one massive INSTALL files into smaller files (no text editing was done except small markdown header formatting) - Rename files to be markdowns for nicer rendering - Bad md rendering is expected in many places	3 years ago