Re: [libvirt] [PATCH RFC 0/8] Sparse streams

** NOT TO BE MERGED UPSTREAM ** This is merely an RFC. What's the problem? We have APIs for transferring disk images from/to host. Problem is, disk images can be sparse files. Our code is, however, unaware of that fact so if for instance the disk is one big hole of 8GB all those bytes have to: a) be read b) come through our event loop. This is obviously very inefficient way. How to deal with it? The way I propose (and this is actually what I like you to comment on) is to invent set of new API. We need to make read from and write to a stream sparseness aware. The new APIs are as follows: int virStreamSendOffset(virStreamPtr stream, unsigned long long offset, const char *data, size_t nbytes); int virStreamRecvOffset(virStreamPtr stream, unsigned long long *offset, char *data, size_t nbytes); The SendOffset() is fairly simple. It is given an offset to write @data from so it basically lseek() to @offset and write() data. The RecvOffset() has slightly complicated design - it has to be aware of the fact that @offset it is required to read from fall into a hole. If that's the case it sets @offset to new location where data starts. Are there other ways possible? Sure! If you have any specific in mind I am happy to discuss it. For instance one idea I've heard (from Martin) was instead of SendOffset() and RecvOffset() we may just invent our variant of lseek(). What's left to be done? Basically, I still don't have RPC implementation. But before I dive into that I thought of sharing my approach with you - because it may turn out that a different approach is going to be needed and thus my work would render useless.