Driver porting: The seq_file interface

There are numerous ways for a device driver (or other kernel component) to provide information to the user or system administrator. One very useful technique is the creation of virtual files, in /proc or elsewhere. Virtual files can provide human-readable output that is easy to get at without any special utility programs; they can also make life easier for script writers. It is not surprising that the use of virtual files has grown over the years.

Creating those files correctly has always been a bit of a challenge, however. It is not that hard to make a /proc file which returns a string. But life gets trickier if the output is long - anything greater than an application is likely to read in a single operation. Handling multiple reads (and seeks) requires careful attention to the reader's position within the virtual file - that position is, likely as not, in the middle of a line of output. The Linux kernel is full of /proc file implementations that get this wrong.

The 2.6 kernel contains a set of functions (implemented by Alexander Viro) which are designed to make it easy for virtual file creators to get it right. This interface (called "seq_file") is not strictly a 2.6 feature - it was also merged into 2.4.15. But 2.6 is where the feature is starting to see serious use, so it is worth describing here.

The seq_file interface is available via <linux/seq_file.h>. There are three aspects to seq_file:

• An iterator interface which lets a virtual file implementation step through the objects it is presenting.
• Some utility functions for formatting objects for output without needing to worry about things like output buffers.
• A set of canned file_operations which implement most operations on the virtual file.

We'll look at the seq_file interface via an extremely simple example: a loadable module which creates a file called /proc/sequence. The file, when read, simply produces a set of increasing integer values, one per line. The sequence will continue until the user loses patience and finds something better to do. The file is seekable, in that one can do something like the following:

    dd if=/proc/sequence of=out1 count=1
    dd if=/proc/sequence skip=1 out=out2 count=1

Then concatenate the output files out1 and out2 and get the right result. Yes, it is a thoroughly useless module, but the point is to show how the mechanism works without getting lost in other details. (Those wanting to see the full source for this module can find it here).

The iterator interface

Modules implementing a virtual file with seq_file must implement a simple iterator object that allows stepping through the data of interest. Iterators must be able to move to a specific position - like the file they implement - but the interpretation of that position is up to the iterator itself. A seq_file implementation that is formatting firewall rules, for example, could interpret position N as the Nth rule in the chain. Positioning can thus be done in whatever way makes the most sense for the generator of the data, which need not be aware of how a position translates to an offset in the virtual file. The one obvious exception is that a position of zero should indicate the beginning of the file.

The /proc/sequence iterator just uses the count of the next number it will output as its position.

Four functions must be implemented to make the iterator work. The first, called start() takes a position as an argument and returns an iterator which will start reading at that position. For our simple sequence example, the start() function looks like:

static void *ct_seq_start(struct seq_file *s, loff_t *pos)
{
	loff_t *spos = kmalloc(sizeof(loff_t), GFP_KERNEL);
	if (! spos)
		return NULL;
	*spos = *pos;
	return spos;
}

The entire data structure for this iterator is a single loff_t value holding the current position. There is no upper bound for the sequence iterator, but that will not be the case for most other seq_file implementations; in most cases the start() function should check for a "past end of file" condition and return NULL if need be.

For more complicated applications, the private field of the seq_file structure can be used. There is also a special value whch can be returned by the start() function called SEQ_START_TOKEN; it can be used if you wish to instruct your show() function (described below) to print a header at the top of the output. SEQ_START_TOKEN should only be used if the offset is zero, however.

The next function to implement is called, amazingly, next(); its job is to move the iterator forward to the next position in the sequence. The example module can simply increment the position by one; more useful modules will do what is needed to step through some data structure. The next() function returns a new iterator, or NULL if the sequence is complete. Here's the example version:

static void *ct_seq_next(struct seq_file *s, void *v, loff_t *pos)
{
	loff_t *spos = (loff_t *) v;
	*pos = ++(*spos);
	return spos;
}

The stop() function is called when iteration is complete; its job, of course, is to clean up. If dynamic memory is allocated for the iterator, stop() is the place to return it.

static void ct_seq_stop(struct seq_file *s, void *v)
{
	kfree (v);
}

Finally, the show() function should format the object currently pointed to by the iterator for output. It should return zero, or an error code if something goes wrong. The example module's show() function is:

static int ct_seq_show(struct seq_file *s, void *v)
{
	loff_t *spos = (loff_t *) v;
	seq_printf(s, "%Ld\n", *spos);
	return 0;
}

We will look at seq_printf() in a moment. But first, the definition of the seq_file iterator is finished by creating a seq_operations structure with the four functions we have just defined:

static struct seq_operations ct_seq_ops = {
	.start = ct_seq_start,
	.next  = ct_seq_next,
	.stop  = ct_seq_stop,
	.show  = ct_seq_show
};

This structure will be needed to tie our iterator to the /proc file in a little bit.

It's worth noting that the interator value returned by start() and manipulated by the other functions is considered to be completely opaque by the seq_file code. It can thus be anything that is useful in stepping through the data to be output. Counters can be useful, but it could also be a direct pointer into an array or linked list. Anything goes, as long as the programmer is aware that things can happen between calls to the iterator function. However, the seq_file code (by design) will not sleep between the calls to start() and stop(), so holding a lock during that time is a reasonable thing to do. The seq_file code will also avoid taking any other locks while the iterator is active.

Formatted output

The seq_file code manages positioning within the output created by the iterator and getting it into the user's buffer. But, for that to work, that output must be passed to the seq_file code. Some utility functions have been defined which make this task easy.

Most code will simply use seq_printf(), which works pretty much like printk(), but which requires the seq_file pointer as an argument. It is common to ignore the return value from seq_printf(), but a function producing complicated output may want to check that value and quit if something non-zero is returned; an error return means that the seq_file buffer has been filled and further output will be discarded.

For straight character output, the following functions may be used:

int seq_putc(struct seq_file *m, char c);
int seq_puts(struct seq_file *m, const char *s);
int seq_escape(struct seq_file *m, const char *s, const char *esc);

The first two output a single character and a string, just like one would expect. seq_escape() is like seq_puts(), except that any character in s which is in the string esc will be represented in octal form in the output.

There is also a function for printing filenames:

int seq_path(struct seq_file *m, struct vfsmount *mnt, 
	     struct dentry *dentry, char *esc);

Here, mnt and dentry indicate the file of interest, and esc is a set of characters which should be escaped in the output. This function is more suited to filesystem code than device drivers, however.

Making it all work

So far, we have a nice set of functions which can produce output within the seq_file system, but we have not yet turned them into a file that a user can see. Creating a file within the kernel requires, of course, the creation of a set of file_operations which implement the operations on that file. The seq_file interface provides a set of canned operations which do most of the work. The virtual file author still must implement the open() method, however, to hook everything up. The open function is often a single line, as in the example module:

static int ct_open(struct inode *inode, struct file *file)
{
	return seq_open(file, &ct_seq_ops);
};

Here, the call to seq_open() takes the seq_operations structure we created before, and gets set up to iterate through the virtual file.

On a successful open, seq_open() stores the struct seq_file pointer in file->private_data. If you have an application where the same iterator can be used for more than one file, you can store an arbitrary pointer in the private field of the seq_file structure; that value can then be retrieved by the iterator functions.

The other operations of interest - read(), llseek(), and release() - are all implemented by the seq_file code itself. So a virtual file's file_operations structure will look like:

static struct file_operations ct_file_ops = {
	.owner   = THIS_MODULE,
	.open    = ct_open,
	.read    = seq_read,
	.llseek  = seq_lseek,
	.release = seq_release
};

The final step is the creation of the /proc file itself. In the example code, that is done in the initialization code in the usual way:

static int ct_init(void)
{
	struct proc_dir_entry *entry;

	entry = create_proc_entry("sequence", 0, NULL);
	if (entry)
		entry->proc_fops = &ct_file_ops;
	return 0;
}

module_init(ct_init);

And that is pretty much it.

The extra-simple version

For extremely simple virtual files, there is an even easier interface. A module can define only the show() function, which should create all the output that the virtual file will contain. The file's open() method then calls:

int single_open(struct file *file, 
                int (*show)(struct seq_file *m, void *p), 
		void *data);

When output time comes, the show() function will be called once. The data value given to single_open() can be found in the private field of the seq_file structure. When using single_open(), the programmer should use single_release() instead of seq_release() in the file_operations structure to avoid a memory leak.

---

(Log in to post comments)

Driver porting: The seq_file interface

Posted Nov 14, 2003 13:13 UTC (Fri) by subscriber laf0rge [Link]

<!-- body -->After using this article as an example to port the /proc/net/ip_conntrack interface over to seq_file, and about 5 hours of crashing/rebooting/debugging, I have to admit that there are some shortcomings in it.

Some hints for other people, so they don't fall into the same pits as I did:

1) If you allocate something in ct_seq_start(), the place to free it is _NOT_ in ct_seq_stop(). This is because ct_seq_stop() is even called if ct_seq_start() returns an error (Like ERR_PTR(-ENOMEM)). You would then end up calling kfree(ERR_PTR(-ENOMEM)) which your mm subsystem doesn't really like. I am now kfree()ing in ct_seq_next(), just before it returns with NULL at the end of the table.

2) If you take a lock in ct_seq_start(), do it unconditionally as the first thing. Even if ct_seq_start() fails, ct_seq_stop() is called. In ct_seq_stop() you have no idea of knowing if ct_seq_start() failed or not - so you will unconditionally unlock.
<!-- comment button -->

<!-- notification -->

Driver porting: The seq_file interface

Posted Nov 15, 2003 1:53 UTC (Sat) by subscriber giraffedata [Link]

<!-- body -->I am now kfree()ing in ct_seq_next(), just before it returns with NULL at the end of the table

Seems like that would be a problem if the user chooses not to read all the way to EOF.

This just sounds like a basic bug in the seq_file interface. If ct_seq_start() fails, it should be ct_seq_start's responsibility to not change any state, and thus ct_seq_stop doesn't need to be, and should not be, called. After all, does a POSIX program call close(-1) when open() fails? <!-- comment button -->

<!-- notification -->

Driver porting: The seq_file interface

Posted Jan 30, 2004 23:08 UTC (Fri) by guest gcc [Link]

<!-- body -->Users should be clear about the difference between a position and an iterator (this confused me for a while). The position is your means of communication with the seq_file interface, telling it where you are, and for it to tell you where to go.

The iterator is an opaque value, about which seq_file understands nothing, except that you return it from your start and next handlers, and it will give it back to you in next, stop and show.

Examples of iterators given in the article are a simple index (using the pointer as an integer, i.e. it doesn't really point to anything), or a pointer to a linked list entry or an array member. However, if you have to navigate some more complex structure, such as a hash table, it can be useful to have a struct which holds several values, for example:

struct my_pos {              
	my_hash_table *table;              
	int hash;              
	my_ht_entry *entry;              
}              

This allows a single value (your iterator, void *v) to tell you everything you need to know in order to walk the hashtable. In this case, as in the /proc/sequence example, v remains the same throughout the walk.

On the other hand, the position, loff_t *pos, is a strange beast. If you start reading the /proc file from the beginning, then start will be called with position 0, 1, 2, etc. until it returns NULL. At this point the user gets EOF, and they can then seek around and start reading again, which results in your start handler being called again.

However, if the user skips more than the length of the file, (e.g. dd if=/proc/my_file bs=1G skip=1, then your code will be called with position 0 twice! The user doesn't get what they were expecting (an empty file), but the whole file, as if the seek never happened. strace shows that the seek appears to succeed. I think this behavious is wrong and constitutes a mental health hazard.

If you want to use the same set of operations for multiple files in /proc, you need a way for them to distinguish what object they operate on. The article mentions s->private, but you can only use this after you have called seq_open, because the seq_file object doesn't exist before that. On the 2.6 kernel, it appears that a good place to store this private data is in the struct inode *inode which is passed to your open handler:

static int my_open (struct inode *inode, struct file          
*file)

In particular, 2.6 has a function called PROC_I which converts your struct inode into a struct proc_inode, which has a member called pde, which is your struct proc_dir_entry.

So you can create the proc_dir_entry, initialise its data member with a pointer to your private data, and then in the open handler you can get to it from inode->pde. For example:

// module init              
static int __init init()              
{              
	my_proc_entry = create_proc_entry(filename, 0, NULL);              
	/* ... check for failure ... */              
	my_proc_entry->data = my_data;              
}              
              
static int my_open (struct inode *inode, struct file *file)              
{              
	struct seq_file *s;       
	result = seq_open(file, &seq_ops);       
	/* ... check for failure ... */       
	/* file->private_data was initialised by seq_open */       
	s = (struct seq_file *)file->private_data;       
	my_data = PROC_I(inode)->pde->data;              
	s->private = my_data;              
}              
              
static int my_seq_start (...) {              
	my_data = s->private;              
}              

I'm still looking for a good way to do this on 2.4, which lacks PROC_I and struct proc_inode. Can anyone help? <!-- comment button -->

<!-- notification -->

Driver porting: The seq_file interface

Posted Apr 1, 2005 17:40 UTC (Fri) by guest guest [Link]

<!-- body -->This article describes how to generate an infinite series of integers. It would have been much more practical if this article had talked about generating finite series of integers ... I had trouble getting the seq_file iterator to stop calling my functions.

I read this article and http://www.kernelnewbies.org/documents/seq_file_howto.txt. Nevertheless, I still wasted a lot of time trying to get it to work.

My notes:

you write the routines your_start, your_stop, your_next, and your_show

The pos parameter to your_start and your_next is an index into the set of items that you wish to enumerate. your_next must increment this index, as in ++*pos.

your_start must know the number of items in your set so that it can check the validity of *pos. your_start will be called again, even after your_next returns NULL and your_stop has been called.

Let's say that you want to count from 0 through 9. your_start is called with *pos==0. It returns a pointer to your iterator data structure. your_show is called, then your_next is called. This happens until your_next returns NULL or ERR_PTR(some-negative-error-code). your_stop is then called. At this point, your_start will be called *again*, with *pos==10 ... seems strange to me, but that is the behavior.

The iterator pointer returned by your_start/your_next can be whatever you want, including integer values. However, be advised that the values in the range -1000 <= n <= 0 are interpreted as error conditions/NULL. Therefore, you probably want to return a pointer.

The kernelnewbies.org seq_file_howto.txt document says your_start should return error codes like EACCES if the *pos position is out of bounds. Note that if you want to do this, then you must return -EACCES, not EACCES. Maybe that was obvious to experienced kernel hackers, but it certainly was not obvious to me. Do this by returning ERR_PTR(-EACCES). If this value is not negative, then the seq_file code will assume that you have returned a valid iterator pointer and it will call your_show.

It is probably easier to simply return NULL on error conditions (like *pos out of bounds) and not worry about it. When I tried to worry about it and do the right thing, it just caused trouble for me.

I spent some time looking at the seq_file code. Frankly, I was quite disappointed ... the implementation looked pretty ugly to me.

Michael