Re: Updating authentication for a Ceph (RBD) disk in a live domain
On Tue, Aug 20, 2024 at 21:40:07 -0000, will.gorman(a)joyent.com wrote:
Is it possible to update and change the <auth/> for an RBD
network disk while the domain the disk is attached to is running and without
detaching/reattaching the disk? For example if I have a disk attached like the
following:
<disk type='network' device='disk'>
<driver name='qemu' type='raw'/>
<auth username='someuser'>
<secret type='ceph' usage='someuser key'/>
</auth>
<source protocol='rbd' name='somepool/someimage'>
<host name='127.0.0.1' port='3300'/>
</source>
<target dev='sdd' bus='scsi'/>
<alias name='scsi0-0-0-3'/>
<address type='drive' controller='0' bus='0'
target='0' unit='3'/>
</disk>
If I want to change the auth to
<auth username='someotheruser'>
<secret type='ceph' usage='someotheruser key'/>
</auth>
can I do that without either attaching/detaching the disk or stopping/restarting the
domain?
No this is not possible currently.
Could you please elaborate what you want to achieve?
The problem with changing credentials is that it can create problems if
clients need to reauthenticate during the credential changeover process
as based on how you roll them out either the client or server may have
incorrect credentials. This would then cause problems for the VM as if
the need to re-auth occurs it would not be able to access the disk any
more.
Example 1:
You update credentials on the server first. The client needs to
reauthenticate shortly before you manage update the VM's credential.
VM breaks.
Example 2:
To mitigate problem from example 1, you update credentials on the VM
first. The client right away needs to reauth and the server wasn't yet
updated.
(Disclaimer: I don't know how instant is the auth propagation in
ceph/rbd)
Until now this feature wasn't needed and thus wasn't yet explored as it
has the above possible drawbacks.
I've tried `virsh update-device domain disk.xml --live
--persistent` using xml identical to the current disk except for the auth and it says
"Device updated successfully" but when I check the domain with `dumpxml` I can
still see the original auth settings for the disk.
Unfortunately the checkers which decide what the API behind 'virsh
update-device' can update are not perfect, so this is a bug that your
request was not rejected, at least for now.