On Tue, Sep 08, 2020 at 08:19:21 -0400, Alexander Wels wrote: > Hi, > > I am working on a project, and one of the requirements is that I can > hotplug a large number of disks. Think several dozens to potentially 100+. > The guest OS will be some linux flavor so it should have no problem with a > large number of disks. > > During my experimentation I quickly ran into the "No available PCI slots" > limit. I found https://libvirt.org/pci-hotplug.html explaining to me why > the limit is there. Now I could add a bunch of pcie-root-ports ahead of > time, but there is likely a limit to that as well. So before I go down some > crazy paths, I figured I would ask on here if there is a better plan than > adding a bunch of root ports for adding a large number of hotplugged disks? I strongly suggest using a virtio-scsi controller and plugging disks into it. You can have a fairly large number of LUNs per controller, so that will severely decrease the amount of required PCI slots. You can obviously add more controllers if you need even more LUNs.

On Tue, Sep 08, 2020 at 08:19:21AM -0400, Alexander Wels wrote: > Hi, > > I am working on a project, and one of the requirements is that I can > hotplug a large number of disks. Think several dozens to potentially 100+. > The guest OS will be some linux flavor so it should have no problem with a > large number of disks. > > During my experimentation I quickly ran into the "No available PCI slots" > limit. I found https://libvirt.org/pci-hotplug.html explaining to me why > the limit is there. Now I could add a bunch of pcie-root-ports ahead of > time, but there is likely a limit to that as well. So before I go down some > crazy paths, I figured I would ask on here if there is a better plan than > adding a bunch of root ports for adding a large number of hotplugged disks? While you can add extra PCI bridges, I'd suggest using virtio-scsi instead These blog posts should serve as warning about some of the limits you'll hit: https://rwmj.wordpress.com/2017/04/25/how-many-disks-can-you-add-to-a-vir... https://rwmj.wordpress.com/2017/04/28/how-many-disks-can-you-add-to-a-vir...

Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|