SourceXR

C/C++ Cross-Reference Tool

Watching File System Events with inotify

Ever wanted to watch file modifications? To read new data from a file as soon as they arrive? To follow several files all at once without polling? To watch for file system events and handle some network connections as the same time without blocking? All of this is possible thanks to the inotify_*() system call family that we describe in this article.

inotify System Calls

The inotify API is rather straightforward:

  • an initialization call: inotify_init()
  • configuration calls with inotify_add_watch() and inotify_rm_watch()
  • the notification call through read()
  • and the termination with close()

Once set up, the API allows the caller to watch for any file system event:

  • file creation, removal, modification, for example: you can be notified when a file is opened
  • directory creation, removal, modification, unmounting
  • file system metadata modification (file attributes)

When an event occurs, the read() system call is used to retrieve data describing the event. The format of the returned structure is the following (excerpt from the inotify(7) man page):

struct inotify_event {
    int wd;          /* Watch descriptor */
    uint32_t mask;   /* Mask of events */
    uint32_t cookie; /* Unique cookie associating related
                        events (for rename(2)) */
    uint32_t len;    /* Size of name field */
    char name[];     /* Optional null-terminated name */
};

But apart from these minimal requirements, the interesting part is that the descriptor returned by inotify_init can be used within a reactor (select, poll, epoll) allowing an application to watch for events without blocking.

Sample Application

This sample application is watching for a file in which logs are written to and sends them as soon as they arrive remotely to a server. It is watching several files at the same time and does it (of course!) in a non-blocking way using the epoll reactor.

The files to watch are supplied on the command line and the remote peer location as well.

First the subroutines that will help us.

The connection is established using the following function:

int connect_peer (const char *ip, unsigned short port ) {
    sockaddr_in addr;
    addr.sin_family = AF_INET;
    addr.sin_port = htons (port);
    inet_aton (ip, &addr.sin_addr);

    int sockfd = socket (AF_INET, SOCK_STREAM, 0);
    if (sockfd == -1) {
        return sockfd;
    }
    // blocking connect, then set non-blocking mode
    int r = connect (sockfd, reinterpret_cast<struct sockaddr *> (&addr),
                     sizeof (sockaddr_in));
    if (r == -1) {
        close (sockfd);
        return r;
    }
    int flags = fcntl (sockfd, F_GETFL, 0);
    fcntl (sockfd, F_SETFL, flags | O_NONBLOCK);
    return sockfd;
}

The reactor manipulations are performed through the following functions:

int add_fd_to_reactor (int epollfd, int fd, uint32_t events) {
    epoll_event event;
    event.events = events;
    event.data.fd = fd; // user data
    return epoll_ctl (epollfd, EPOLL_CTL_ADD, fd, &event);
}
int mod_fd_reactor (int epollfd, int fd, uint32_t events) {
    epoll_event event;
    event.events = events;
    event.data.fd = fd; // user data
    return epoll_ctl (epollfd, EPOLL_CTL_MOD, fd, &event);
}

And finally the files are watched and set up using the following routine:

int add_watch (int argc, char **argv, int inotifyfd,
               std::vector<int>& fds,
               std::map<int, int>& watches) {
    // add watch for all files specified on the command line:
    for (int i = 3; i < argc; ++i) {
        // we're only interested when data is appended to the file
        // other flags that are worth looking at could be:
        // IN_CLOSE, IN_DELETE_SELF, IN_MOVE_SELF for example when
        // watching files that are rotated
        int r = inotify_add_watch (inotifyfd, argv[i], IN_MODIFY);
        if (r == -1) {
            std::cerr << strerror (errno) << "\n";
            close_fds (fds, watches);
            return 1;
        }
        // open the file and seek to the end
        int fd = open (argv[i], O_RDONLY);
        if (fd == -1) {
            std::cerr << strerror (errno) << "\n";
            close_fds (fds, watches);
            return 1;
        }
        lseek (fd, 0, SEEK_END);

        // enable non-blocking mode on files
        int flags = fcntl (fd, F_GETFL, 0);
        fcntl (fd, F_SETFL, flags | O_NONBLOCK);

        fds.push_back (fd);
        // map the watched event to the file file descriptor
        watches.insert (std::make_pair (r, fd));
    }
    return 0;
}

Now the full program. We set up the reactor, the connection to the remote peer (which, to simplify, we perform in blocking mode) and the inotify watch of the supplied files.

int main (int argc, char** argv) {

    if (argc < 4) {
        std::cerr << "Use " << argv[0] << " remote_ip port files...\n";
        return 1;
    }

    struct sigaction sa;
    sa.sa_handler = SIG_IGN;
    sigemptyset (&sa.sa_mask);
    sa.sa_flags = 0;
    sigaction (SIGPIPE, &sa, 0);

    int infd = inotify_init1 (IN_NONBLOCK); // from 2.6.27
    // otherwise use inotify_init()
    // followed by a call to fcntl()
    if (infd == -1) {
        std::cerr << strerror (errno) << "\n";
        return 1;
    }

    std::map<int, int> watches;
    std::vector<int> fds;
    fds.push_back (infd);

    if (add_watch (argc, argv, infd, fds, watches) == -1) {
        close_fds (fds, watches);
        return 1;
    }

    // establish connection to remote peer
    const char *ip = argv[1];
    const unsigned short port = atoi (argv[2]);
    int confd = connect_peer (ip, port);
    if (confd == -1) {
        std::cerr << strerror (errno) << "\n";
        close_fds (fds, watches);
        return 1;
    }
    fds.push_back (confd);

    const int size = 10;
    int epollfd = epoll_create (size);
    if (epollfd == -1) {
        std::cerr << strerror (errno) << "\n";
        close_fds (fds, watches);
        return 1;
    }
    fds.push_back (epollfd);

    epoll_event events[size];

    // add inotify fd to reactor
    if (add_fd_to_reactor (epollfd, infd, EPOLLIN) == -1) {
        std::cerr << strerror (errno) << "\n";
        close_fds (fds, watches);
        return 1;
    }

    // add peer connection to reactor
    if (add_fd_to_reactor (epollfd, confd, EPOLLIN) == -1) {
        std::cerr << strerror (errno) << "\n";
        close_fds (fds, watches);
        return 1;
    }

The main loop is wait for events in epoll_wait. When it returns, we demultiplex the pending events based on the fd field which we supplied when we registered the descriptor.

If it is an inotify event, we map the event to the file using the map that we filled earlier, read the data that has been appended to the file and we enable the ouput on the connection to the peer.

Otherwise, we write the buffer to the peer and we disable the notification if we are done. If the peer closes the connection, we exit.

    std::string outputBuffer;

    bool done = false;
    while (!done) {

        const int infinity = -1;
        // and wait for events
        int r = epoll_wait (epollfd, events, size, infinity);
        if (r == -1) {
            close_fds (fds, watches);
            return 1;
        }

        // demultiplex events
        int i = 0;
        while (i < r) {
            if (events[i].data.fd == infd) {
                // a file was modified
                // inotify_event size is dynamic, we use a buffer to read the
                // pending events
                const size_t size = 4096;
                char buffer[size];
                int i = read (infd, buffer, size);
                if (i != -1) {
                    int k = 0;
                    while (k < i) {
                        inotify_event event =
                            *reinterpret_cast<inotify_event *> (buffer + k);
                        char filebuffer[size];
                        if (event.mask & IN_MODIFY) {
                            int r = read (watches[event.wd], filebuffer, size);
                            if ((r == -1)
                                && (errno != EAGAIN)) {
                                std::cerr << strerror (errno) << "\n";
                                close_fds (fds, watches);
                                return 1;
                            }
                            outputBuffer.append (filebuffer, r);
                            // enable write on the connection
                            mod_fd_reactor (epollfd, confd, EPOLLIN|EPOLLOUT);
                        }
                        else {
                            std::cout << "Unhandled inotify event: "
                                      << event.mask << "\n";
                        }
                        k += sizeof (inotify_event) + event.len;
                    }
                }
                else {
                    std::cerr << strerror (errno) << "\n";
                    close_fds (fds, watches);
                    return 1;
                }
            }
            else if (events[i].data.fd == confd) {
                if (events[i].events & EPOLLOUT) {
                    ssize_t r = write (confd, outputBuffer.data (),
                                       outputBuffer.size ());
                    if (r == -1) {
                        std::cerr << strerror (errno) << "\n";
                        close_fds (fds, watches);
                        return 1;
                    }
                    else {
                        if (static_cast<size_t> (r) == outputBuffer.size ()) {
                            outputBuffer.clear();
                            // disable write events, nothing left to send
                            mod_fd_reactor (epollfd, confd, EPOLLIN);
                        }
                        else { // remaining data, move buffer
                            std::string b (outputBuffer.begin () + r,
                                           outputBuffer.end ());
                            outputBuffer = b;
                        }
                    }
                }
                if (events[i].events & EPOLLIN) {
                    const size_t s = 128;
                    char buffer[s];
                    ssize_t r = read (confd, buffer, s);
                    if ((r == -1)
                        && (errno != EAGAIN)) {
                        std::cerr << strerror (errno) << "\n";
                        close_fds (fds, watches);
                        return 1;
                    }
                    if (r == 0) {
                        std::cout << "Remote peer disconnected, exiting\n";
                        done = true;
                    }
                }
            }
            ++i;
        }
    }

In order to test this small program, you can use netcat to provide the remote peer: launch it with the following command line:

netcat -l -p port

where port is the port value that you supplied to the program.

Then launch the program and write some data to the watched files: netcat will print what it receives.

The source code is available here.

Comments !