opnsense-firewall

Overview

Runtime Architecture

Internet
│
ISP Router (WAN)
│
OPNsense (Router-on-a-Stick)
│
├── VLAN 10 — WAN
├── VLAN 20 — Server Subnet (10.10.1.0/24)
└── VLAN 30 — PC / Client Subnet (10.10.2.0/24)
│
Managed Switch (TP-Link TL-SG105E)
│
├── Proxmox Nodes (VLAN 20)
└── Personal PC (VLAN 30)

Infrastructure Architecture

└── OPNsense Router
    ├── LAN (VLAN 20 — Server Subnet)
    ├── OPT1 (VLAN 30 — PC Subnet)
    └── WAN (VLAN 10 — Upstream ISP)

Challenges

• •
  Repeated Loss of Internet and Server AccessDuring early VLAN and interface configuration, I repeatedly lost internet connectivity and access to backend servers after applying network changes. Troubleshooting was difficult because I couldn't immediately tell if the failure was caused by routing, DNS, or the firewall.
• •
  Unstructured Configuration ApproachI initially configured VLANs, DHCP, and firewall rules all at once without testing each layer individually. This made it nearly impossible to isolate the root cause when something broke, since multiple untested changes were stacked on top of each other.
• •
  DNS Pointing to a Stale GatewayAfter migrating to the OPNsense based topology, one of the Proxmox nodes still had its DNS resolver pointed at an old gateway IP from a previous network setup. Internet routing (ICMP) worked fine, which made the failure misleading it looked like a connectivity issue when it was actually a DNS resolution issue pointing to an unreachable host.
• •
  VLAN Tagging Confusion Between Host and Container LevelWhen assigning VLAN 20 to a new container, the VLAN interface visible at the Proxmox host level (vmbr0.20) was not selectable as a bridge target for containers only the underlying physical bridge (vmbr0) was. This caused confusion about where VLAN tagging actually needed to be applied.
• •
  Firewall Rule Not Taking EffectA block rule intended to prevent servers from initiating connections back to client devices didn't work as expected when created as a Floating Rule without an explicitly assigned interface, and when placed below the default allow rule instead of above it.

Solutions

• •
  Adopted a Phased, Layer-by-Layer Configuration ApproachInstead of configuring routing, VLANs, DHCP, and firewall rules simultaneously, I rebuilt the network in isolated phases verifying basic connectivity at each layer before adding the next. This became the core best practice going forward: test one variable at a time, confirm it works, then move on.
• •
  Diagnosed DNS Failures SystematicallyUsed a structured diagnostic sequence (cat /etc/resolv.conf → ping <external IP> → nslookup <domain> <specific DNS server>) to isolate whether a failure was routing level or DNS-level, rather than guessing. This distinguished 'host unreachable' (misconfigured DNS server) from actual connectivity loss.
• •
  Corrected DNS Configuration at the Right LayerFixed the stale DNS entry permanently through Proxmox's System DNS settings (the actual source of truth that regenerates /etc/resolv.conf) instead of editing the resolver file directly, which would have been overwritten.
• •
  Enabled VLAN-Aware BridgingIdentified that the physical bridge (vmbr0) needed the 'VLAN aware' option enabled before per container VLAN tagging would function correctly, resolving the confusion between host level VLAN interfaces and container assignable bridges.
• •
  Restructured Firewall Rule PlacementRebuilt the block rule as a standard interface rule (rather than a Floating Rule) with an explicit interface assignment, and repositioned it above the default allow rule since OPNsense evaluates rules top down and stops at the first match.

What I Learned

• •A methodical, phase by phase approach to network configuration is far more reliable than configuring everything at once isolating variables makes root cause diagnosis dramatically faster.
• •Firewall rules are additive, not exclusive: each rule only matches specific traffic, so allowing one destination doesn't implicitly block others every intended restriction needs its own explicit rule.
• •Rule order and interface scope are as critical as the rule logic itself in a top-down, first match firewall engine.
• •DNS failures and routing failures can look identical from the outside ('no internet') but require completely different diagnostic paths.
• •VLAN tagging can exist at multiple levels (host interface vs. bridge vs. container) understanding where a tag is applied is essential to avoid double tagging or missing segmentation entirely.
• •Building network segmentation from scratch gave me a much deeper appreciation for why enterprise environments enforce least privilege access between subnets by default.