What you need¶

The exercises assume you already have a BlueField-3 in front of you, wired and configured a particular way. This page states those assumptions exactly, and how to satisfy them.

To run the tutorial for other people, see How it was built.

1. A card you can log into¶

One BlueField-3, with a shell on its Arm cores and sudo. Everything is built and run there — no second machine, no host-side component.

2. Two ports that reach each other¶

The one topological requirement: a frame leaving p1 must arrive at p0. A direct cable between the two ports satisfies it; so does a fabric that carries the traffic around. Nothing else about the wiring matters.

     ns1                                    ns0
     10.0.0.2                               10.0.0.1
     mlx5_3  (SF on PF1)                    mlx5_2  (SF on PF0)
        |                                      ^
        | client sends                         | server receives
        v                                      |
       p1  ────────────  wire  ──────────────► p0
       PF1 / mlx5_1                            PF0 / mlx5_0
       PCC controller runs here                DOCA Flow marks packets here

A sub-function (SF) on each port plays sender and receiver, each in its own network namespace so the kernel cannot short-circuit 10.0.0.1 ↔ 10.0.0.2 locally and skip the wire.

3. That exact configuration¶

Every command in the guides hardcodes these names, so the layout is not negotiable:

	Device	Netns	IP	Role
SF, sfnum 0 on PF0	`mlx5_2` / `enp3s0f0s0`	`ns0`	`10.0.0.1`	receiver (server)
SF, sfnum 0 on PF1	`mlx5_3` / `enp3s0f1s0`	`ns1`	`10.0.0.2`	sender (client)
SF, sfnum 4 on PF0	`mlx5_4` / `enp3s0f0s4`	`ns0_1`	`10.0.0.11`	second receiver — path steering only

Plus ovsbr1/ovsbr2 bridging each PF, and 4 GB of hugepages. One script establishes all of it:

sudo ./admin/local_scripts/setup_roce_loopback.sh

It imposes this layout rather than adapting — it deletes every OVS bridge and SF on both ports first. RDMA device indices are handed out in probe order, so starting from zero is the only way to reliably land on mlx5_2/mlx5_3. None of it survives a reboot; re-run after every boot.

4. Firmware, for Part 2¶

PCC will not start otherwise:

USER_PROGRAMMABLE_CC=1 — must be the current value, not merely staged. That takes a genuine power-cycle; the BlueField-3 has its own power rail, so a reboot will not apply it.
DPA_AUTHENTICATION=0 — the factory default, but some cards ship hardened.

5. Software¶

DOCA 2.x or 3.x installed under /opt/mellanox/doca, plus meson, ninja and dpacc. Also libpcap-dev and perftest (ib_write_bw). Cards used for the tutorial spanned DOCA 2.7–3.4, which is why the exercises exist in two parallel trees — build the one matching your card (admin/local_scripts/print_doca_version.sh tells you which).

meson setup build doca-3     # or doca-2
ninja -C build

Flow programs land in build/doca-flow/, the PCC controller in build/doca-pcc-ecn/. Path steering is a separate meson project nested one level down.

Where to get it¶

Repository
…-participants	Start here. Exercises, guides, traffic scripts. No solutions — the `TODO`s are yours.
sigcomm26-tutorial-bluefield	The organizers' repo: same exercises plus answer keys, fleet tooling, guide sources, bring-up notes.

Both are frozen. MIT licensed, except parts of the PCC exercises derived from NVIDIA's DOCA samples, which stay BSD-3-Clause — listed in THIRD-PARTY-NOTICES.md.