Re: large number of hot plug disks
On Tue, Sep 08, 2020 at 02:38:17PM +0200, Peter Krempa wrote:
On Tue, Sep 08, 2020 at 08:19:21 -0400, Alexander Wels wrote:
> Hi,
>
> I am working on a project, and one of the requirements is that I can
> hotplug a large number of disks. Think several dozens to potentially 100+.
> The guest OS will be some linux flavor so it should have no problem with a
> large number of disks.
>
> During my experimentation I quickly ran into the "No available PCI slots"
> limit. I found https://libvirt.org/pci-hotplug.html explaining to me why
> the limit is there. Now I could add a bunch of pcie-root-ports ahead of
> time, but there is likely a limit to that as well. So before I go down some
> crazy paths, I figured I would ask on here if there is a better plan than
> adding a bunch of root ports for adding a large number of hotplugged disks?
I strongly suggest using a virtio-scsi controller and plugging disks
into it. You can have a fairly large number of LUNs per controller, so
that will severely decrease the amount of required PCI slots. You can
obviously add more controllers if you need even more LUNs.
A single virtio-scsi controller can support ~4 million LUNs. IOW you'll
face a wall of other problems before you get anywhere near needing a
second virtio-scsi controller due to LUN exhaustion :-)
Only real reason for multiple virtio-scsi controllers that I see is
if you want to setup different host CPU affinity for each controller
for I/O thread performance tuning
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|