Hello again,
TL;DR:
Today I noticed that my LXC container was out of space in "/dev". It took me a few minutes to figure out why. There were only 23 files there. The "/dev" fs quota was 64K, and all was taken up.
The problem was that "syslog-ng" was writing "console" messages to "/dev/tty12". But "/dev/tty12" was not a device node, so the kernel "(2)open"ed it as a file and dutifully wrote to it.
I have corrected my syslog-ng config, but I was wondering....
Is there any legitimate reason to EVER have a regular file in "/dev"? If not, can libvirt or Linux be modified so that the filesystem can be mounted in such a way to prevent a regular file from ever being created there? Kind of like an inverse of the "nodev" mount option seen in various filesystems (ext3, nfs). IMHO, I would rather have syslog-ng (or other tool) fail to open a regular file in "/dev", than for it is succeed and then fill up the small fs.
Thoughts?
Boring stuff:
Sep 30 14:06:47 localhost syslog-ng[440]: Error suspend timeout has elapsed, attempting to write again; fd='16'
Sep 30 14:06:47 localhost syslog-ng[440]: I/O error occurred while writing; fd='16', error='No space left on device (28)'
Sep 30 14:06:47 localhost syslog-ng[440]: Suspending write operation because of an I/O error; fd='16', time_reopen='60'