POiSON WORLD

Troubleshooting filesystem issues can be a complicated task, and it is made almost impossible if the system is unable to create logs or if you’re unable to access the logs.

There are few instances in which this might occur. This article will focus on a specific instance: when the file system is remounted as read-only.

Read-only Filesystem Issue / Errors

By default, Linux will mount the filesystems as read+write, but some times, due to failures, they can be automatically remounted as read-only. When that happens, well, you can only read your files, not write to them.

There a couple of different reasons why an issue like this might occur. One of the more common reasons is when the system boots after a crash (or loses power) where it is unable to shutdown correctly. The system will try to auto correct during boot, but if it is unable to do so, it will mount in read-only mode.

A real-world example might be if your hosting provider has an infrastructure failure. Your servers come online, but all of a sudden the file-system is in a read-only mode.

Working with Read-only Filesystem Logs

Building on the above scenario, let’s assume your server filesystem is in read-only mode. What do you do? How do you identify that is what is happening?

First, keep an eye out for “cannot create file: read-only file system” warnings in terminal. Such an example might occur if you try to do something like this:

root@test-server:~# tail /va-bash: cannot create temp file for here-document: Read-only file system
-bash: cannot create temp file for here-document: Read-only file system

This is a good indicator that a problem exists. A good way to confirm is by trying to touch a file to see if works:

root@test-server:~# touch test
touch: cannot touch 'test': Read-only file system

If it does not allow you to create the file you know where the issue is, you’re likely working a read-only file system issue.

Anther way to test is using mount. You can type the command “mount” and the output will give you something like this: , you can confirm if the server is in read-only (ro) mode:

# mount
..
/dev/sda4 on / type ext4 (ro,relatime,data=ordered)

In the entry above you see the reference to ro which is the abbreviation for read-only.

Solving the issue can be as simple as rebooting the server, but if that does not work you will have to investigate further.