Enhanced file change notification for linux

News

I am currently working on combining parts of the inotify patch from John McCutchan with my patch. This will break compatibility with the old dnotify mechanism and instead create a pollable device. But the old dnotify mechanism was a mess anyway. Even linus thinks so :-)

There is a similar project called inotify from John McCutchan. Please check it out at kerneltrap external link . It does some things better than my patch (device instead of arcane signals) and somethings worse (no recursive notification).

The Problem

There are many situations where a program wants to get notified when a file has been changed or accessed by another program. The most obvious case is a file manager or open/save dialog that has to present accurate and up-to-date information about the contents of a directory to the user. Another case would be metadata like the locatedb that has to be consistent with the current contents of the file system. A third example would be security tools like tripwire. Automatic synchronization tools like rsync could also benefit very much from such a mechanism.

Limitations of the current Mechanism

The current linux kernel contains a mechanism to watch for changes in a directory. This dnotify mechanism (see Documentation/dnotify.txt for details) is useful, but very limited in its scope.

It does not work recursively, so to watch for file changes in a large directory tree like /home it would be nessecary to watch each directory in the tree separately. Since watching for changes in a directory requires a file handle, this would require holding thousands of file handles.
It does not pass any information about what file has changed, so whenever something has changed, the whole directory has to be rescanned to determine what file has changed.
It does not pass any information about where the file has changed, so in the case of small changes to very large files it would be nessecary to compare the new file to the old file to find out what exactly has changed.

Enhanced dnotify mechanism

The patch enhances the dnotify mechanism in two very important ways.

It adds an additional flag to make dnotify work recursively. To watch for changes in /home and any of its subdirectories, you would just have to watch /home recursively. A limitation is that this does not work over mount boundaries, so to watch for changes on the whole system you would have to watch each mount point.
It optionally passes additional information for each change. The amount of additional information depends on the kind of change. For example for a write access, the information contains the pid of the writing process, the file that has changed, the offset and the size of the area that has been written to.

How does it work?

The enhanced dnotify mechanism works on dcache level. The dcache is an internal tree structure that presents a view into each file system. It only contains references to the most recently used files, so it is quite small even for large file systems.

When a program wants to watch for changes for a directory or file, it does an ioctl like in the original dnotify mechanism. But there are some additional flags for the ioctl:

DN_RECURSIVE means that all subdirectories of the watched directory will be watched. A limitation is that this does not work over mount boundaries, so if you want to watch for changes on the whole system you will have to watch each mount point.
DN_EXTENDED means that extended information for the type of change is gathered. The information you get depends on the kind of change that happened. For example for a read access you get information about the file that has changed and the offset and size of the changed region.

As in the original dnotify mechanism, whenever one of the watched files changes the userspace program gets a signal. The program can then do another ioctl to find out what exactly has happened. The information passed to the userspace program is in a very compact form, but the program can reconstruct the path of the file and other interesting information. See the userspace program for how this works.

What can be watched

The current implementation can be used for the following events:

/*
 * Types of directory notifications that may be requested.
 */
#define DN_ACCESS       0x00000001      /* File accessed */
#define DN_MODIFY       0x00000002      /* File modified */
#define DN_CREATE       0x00000004      /* File created */
#define DN_DELETE       0x00000008      /* File removed */
#define DN_RENAME       0x00000010      /* File renamed */
#define DN_ATTRIB       0x00000020      /* File changed attibutes */
#define DN_REGISTER     0x00000040
#define DN_UNREGISTER   0x00000080
#define DN_MMAP         0x00000100
#define DN_MUNMAP       0x00000200
#define DN_MOUNT        0x00000400
#define DN_UNMOUNT      0x00000800

#define DN_EXTENDED     0x20000000      /* Extended notify info */
#define DN_RECURSIVE    0x40000000      /* Recursive notification */
#define DN_MULTISHOT    0x80000000      /* Don't remove notifier */

Limitations

To make the impact on the performance and on the linux file system code as small as possible, I had to make some compromises.

Just like the original dnotify mechanism, the mechanism does not work for hard links. I tried to make it work for hard links, but since inode numbers are not unique and persistent for all file systems this would be quite hard to do efficiently, and I think it would not be worth it.
File changes on mmap'ed files will not not be noticed. However, you can get an event when a file is mmap'ed in the first place, so you have to assume that all files that are currently mmap'ed can change at any time.

Downloads

I have divided the dnotify mechanism into a stub that has to be compiled into the kernel and a module that contains the bulk of the code. That way, I can try new things without rebooting every five minutes. This separation will no longer be nessecary once the mechanism is stable. I also wrote a small user space utility to test the mechanism.

The patch for the stub (this has to be compiled into the kernel): dnotify-0.1.patch

The module containing the bulk of the mechanism (compile with ./make and install with insmod dnotify.o) dnotify-module-0.1.tgz

The userspace tool (compile with g++ -o dnotify dnotify.cpp) dnotify-user-0.1.tgz

The userspace tool watches for all changes in the directory and all its subdirectories. So if you let it watch the root directory you will get a lot of output. You can use grep to watch only for certain changes

Examples:

./dnotify / | grep "access"

will watch for all read access in / and its subdirectories.

./dnotify /mnt | grep "mount"

will watch for mounts in /mnt and its subdirectories.