Hello again,
TL;DR:
Today I noticed that my LXC container was out of space in "/dev". It
took me a few minutes to figure out why. There were only 23 files there.
The "/dev" fs quota was 64K, and all was taken up.
The problem was that "syslog-ng" was writing "console" messages
to
"/dev/tty12". But "/dev/tty12" was not a device node, so the kernel
"(2)open"ed it as a file and dutifully wrote to it.
I have corrected my syslog-ng config, but I was wondering....
Is there any legitimate reason to EVER have a regular file in "/dev"?
If not, can libvirt or Linux be modified so that the filesystem can be
mounted in such a way to prevent a regular file from ever being created
there? Kind of like an inverse of the "nodev" mount option seen in various
filesystems (ext3, nfs). IMHO, I would rather have syslog-ng (or other
tool) fail to open a regular file in "/dev", than for it is succeed and
then fill up the small fs.
Thoughts?
Boring stuff:
Sep 30 14:06:47 localhost syslog-ng[440]: Error suspend timeout has
elapsed, attempting to write again; fd='16'
Sep 30 14:06:47 localhost syslog-ng[440]: I/O error occurred while writing;
fd='16', error='No space left on device (28)'
Sep 30 14:06:47 localhost syslog-ng[440]: Suspending write operation
because of an I/O error; fd='16', time_reopen='60'
This is what I see inside the container:
djenkins@dwj-hfax-dev ~/src/HylaFAX+ $ find /dev | wc -l
23
dwj-hfax-dev ~ # ls -l /dev/tty*
crw-rw-rw- 1 root root 5, 0 Sep 30 13:44 /dev/tty
lrwxrwxrwx 1 root root 10 Sep 30 13:03 /dev/tty1 -> /dev/pts/0
-rw------- 1 root root 65536 Sep 30 14:12 /dev/tty12
### AHHH!!! There is the problem. "syslog-ng" is writing to
"/dev/tty12",
but it is a file and not a real device.
djenkins@dwj-hfax-dev ~/src/HylaFAX+ $ du -sh /dev
64K /dev
djenkins@dwj-hfax-dev ~/src/HylaFAX+ $ df -h /dev
Filesystem Size Used Avail Use% Mounted on
devfs 64K 64K 0 100% /dev