Unable to access VMs from outside the host

[harshl]

Hey all,

I have an issue that is driving me nuts and I think I am getting outside of my ability to properly fix it. I have an issue on an Ubuntu 22.04.4 server doing virtualization for a GNS3 environment. I don't believe that it being a GNS3 environment is really relevant except to say their guide is how I arrived where I am at. I did a base install of Ubuntu 22.04.4 and then followed this guide with a couple of modifications along the way:
https://docs.gns3.com/docs/getting-s...llation/linux/ (top Ubuntu section)
The only thing I had to do different from that guide was install the GPG keys a little different since the method they reference has been deprecated, but that all went fine.

So with that context, here is the root issue. I am unable to ping into VMs on the Ubuntu host from anywhere outside of the Ubuntu host itself. I know the issue is on the host itself, because if I add the following to IPTables, it works, but I don't know how to make this stick through a reboot.

Again, in an attempt to provide clarity, the only thing I need to do to obtain my desired outcome is add this one single line below (iptables -I LIBVIRT_FWI 2 -j ACCEPT). Nothing else is broken, and nothing else is needed. I just need to know how to make that entry persist through a reboot.

iptables -I LIBVIRT_FWI 2 -j ACCEPT

According to this: https://libvirt.org/firewall.html and this: https://libvirt.org/formatnwfilter.html, there are other firewall auto configurations at play here that may be overriding that?

Regarding the two libvirt articles above, I did some looking and it looks like there are already rules to allow this traffic, but it isn't working. I'm stumped.
#virsh nwfilter-list

 UUID                                   Name
-----------------------------------------------------------------
 85246ab3-9fb5-4806-a5d9-427a6ce49cbd   allow-arp
 6484405f-6207-463b-a446-17653cab27b1   allow-dhcp
 53f0ebb9-eed3-4d83-aa27-1d968be5ee2a   allow-dhcp-server
 80f3302a-6759-4fdf-a4f7-031f74d4f4a4   allow-dhcpv6
 80953f56-390e-4acf-9ab6-55d59f8ad44c   allow-dhcpv6-server
 cb585ff6-5b65-4e88-82bc-1d0624f9531a   allow-incoming-ipv4
 3f72c727-912d-4d2d-925e-a99bcfd6d388   allow-incoming-ipv6
 037901c9-ec86-4510-b093-7d71cd3c7a39   allow-ipv4
 5109e791-64a9-4fa8-a45a-b80b08b619af   allow-ipv6
 627fd14f-28c0-4748-88f0-29e62b6627b5   clean-traffic
 534467e9-a32c-407b-ae8d-facccaf8866c   clean-traffic-gateway
 c573809e-bb84-4272-b9ff-2cdaaa028a33   no-arp-ip-spoofing
 7b1b5650-c72b-4fc3-a6d6-a883fa98f5ea   no-arp-mac-spoofing
 51966ed9-4e22-4c58-bbce-36756977c779   no-arp-spoofing
 403501b0-2f61-4fa9-807f-79005da9597f   no-ip-multicast
 1898b7ce-5710-41ef-8e22-168fd38e530e   no-ip-spoofing
 0496a176-5f5e-4a84-aa30-fa868029ae7c   no-ipv6-multicast
 37c749c6-8df2-4791-a9e9-eddc17487f1f   no-ipv6-spoofing
 3b75ae93-4192-4ba5-b165-bcee0b2403e6   no-mac-broadcast
 04e72613-0335-4e7c-89bd-f3dbe45c6b46   no-mac-spoofing
 49f53b92-2463-4388-873d-47ed6f575020   no-other-l2-traffic
 654184aa-c9af-4981-ac73-f8641e0b4e24   no-other-rarp-traffic
 9ddff19c-3a88-4676-808d-b623fee24a23   qemu-announce-self
 43f355c1-bda5-4494-b925-1101c02ee636   qemu-announce-self-rarp

and

#cat /etc/libvirt/nwfilter/allow-incoming-ipv4.xml

<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
  virsh nwfilter-edit allow-incoming-ipv4
or other application using the libvirt API.
-->

<filter name='allow-incoming-ipv4' chain='ipv4' priority='-700'>
  <uuid>cb585ff6-5b65-4e88-82bc-1d0624f9531a</uuid>
  <rule action='accept' direction='in' priority='500'/>
</filter>

#ufw status

Status: inactive

Any guidance on how to resolve this sure would be appreciated, thanks!

[Tadaen_Sylvermane]

You can write a script fired by a systemd service to configure iptables on boot. I know there is a better way but this is what I ended up doing. Easier for me to mess with a script than anything else.

My script for reference. https://gitlab.com/jmgibson1981/home...ref_type=heads

This was fired at boot via systemd. I never did mess with the iptables save stuff. Supposedly one should be using nftables instead but I am and was to lazy to change it. As I don't use that script anymore it's on my git just for reference.

[TheFu]

If you want a VM to be on the same subnet as the host, use a network bridge. There are examples for how to set up a bridge on an Ubuntu Server in the netplan.io/examples website. Inside the VM, networking is all the same, but you'll need to connect the bridge to the VM networking inside libvirt. I usually do that using virt-manager into the VM-host from my workstation.

I don't know what GNS is.

[harshl]

Thank you, Tadaen_Sylvermane, I think I will likely go that direction for now, the firewalling in libvirt is a mess.

Thank you for the reference, I really appreciate it! My script will be much simpler as I literally just need to add one IPTables line to make it work as desired.

[harshl]

If you want a VM to be on the same subnet as the host, use a network bridge. There are examples for how to set up a bridge on an Ubuntu Server in the netplan.io/examples website. Inside the VM, networking is all the same, but you'll need to connect the bridge to the VM networking inside libvirt. I usually do that using virt-manager into the VM-host from my workstation.

I don't know what GNS is.

Thanks TheFu, but all of that is already working fine. Per my OP, I need only add one IPTables line to get the desired outcome. I just can't figure out how to make that line persist through reboots, so for now, I will just script the entry of the line after bootup.

Thanks.

[TheFu]

Thanks TheFu, but all of that is already working fine. Per my OP, I need only add one IPTables line to get the desired outcome. I just can't figure out how to make that line persist through reboots, so for now, I will just script the entry of the line after bootup.

Thanks.

I don't have any firewall rules on my VM hosts specific to the VMs. Any traffic for the IP is sent to the VM IP where any firewalling desired happens, just like with any other physical host.

But there are 50,000 different solutions. Perhaps I'm just lazy. OTOH, I do passthru a NIC to internet-facing VMs, so the host doesn't actually have access to that traffic, unless it is on the LAN with the host, which is what I thought you wanted.

[MAFoElffen]

I do Network bridges on my KVM Hosts. It is not that hard, and it is the accepted answer for that. I find that easier than trying to make IP Tables work. I really can't understand the reasoning of thinking the opposite on that.

There is a third way that most people do not think of:
If your remote KVM host is on the same net as the local machine, and the local machine also has KVM installed with Network Manager, then you can connect to remote KVM host's VM through qemu+ssh://UserName@IP_Address/system...

[TheFu]

I do Network bridges on my KVM Hosts. It is not that hard, and it is the accepted answer for that. I find that easier than trying to make IP Tables work. I really can't understand the reasoning of thinking the opposite on that.

Me neither. Bridges have been the recommended solution since at least 2008.

There is a third way that most people do not think of:
If your remote LVM host is on the same net as the local machine, and the local machine also has KVM installed with Network Manager, then you can connect to remote KVM host's VM through qemu+ssh://UserName@IP_Address/system...

"LVM?" Perhaps you mean "VM"?

I use the qemu+ssh:// method with virt-viewer to connect to my VMs.

    /usr/bin/virt-viewer --connect qemu+ssh://hadar/system regulus &

hadar is the VM host.
regulus is the guest VM.

    /usr/bin/virt-viewer --connect qemu+ssh://istar/system deneb &

istar is the VM host.
deneb is the guest VM.

To setup the VM, I use virt-manager, but that isn't the normal way for connections. There's a virt-viewer client for the main 3 platforms.

[MAFoElffen]

@TheFu -- I meant "KVM Host", instead of "LVM Host". Was in a hurry and they (L & K) are 1 key from each other = typo. Thank you for the catch.

[harshl]

Thanks for the responses guys. Unless I am massively misunderstanding something, I do have a bridge network, and it is working fine accept that one of the firewall technologies, that was present right out of a base install of 22.04.4 server is blocking traffic inbound to VMs on the other side of it.

Interface virbr0 is the main interface that I have attached my VMs to that need to communicate outside of the host. From the outside, production networks, I am routing to it from my production firewall, which is also working fine as I can reach the VMs once I add that IPTables line. Why that line is needed I have no idea and is a part of the mystery here.

Any thoughts on why that line is necessary to make this work? Can I provide any particular output that would help diagnose this?

To be abundantly clear, I have not installed, modified or added any firewalling technologies to this host besides that one line that makes it work. Everything else is as it was from a base install.