Messages in this thread | | | Subject | Re: [RFC PATCH 1/3] Unified trace buffer | From | Peter Zijlstra <> | Date | Wed, 24 Sep 2008 18:11:25 +0200 |
| |
On Wed, 2008-09-24 at 08:47 -0700, Martin Bligh wrote: > Thanks for creating this so quickly ;-) > > >> We can record either the fast way of reserving a part of the buffer: > >> > >> event = ring_buffer_lock_reserve(buffer, event_id, length, &flags); > >> event->data = record_this_data; > >> ring_buffer_unlock_commit(buffer, event, flags); > > > > This can, in generic, not work. Due to the simple fact that we might > > straddle a page boundary. Therefore I think its best to limit our self > > to the write interface below, so that it can handle that. > > I'm not sure why this is any harder to deal with in write, than it is > in reserve? We should be able to make reserve handle this just > as well?
No, imagine the mentioned case where we're straddling a page boundary.
A----| |----B ^------|
So when we reserve we get a pointer into page A, but our reserve length will run over into page B. A write() method will know how to check for this and break up the memcpy to copy up-to the end of A and continue into B.
You cannot expect the reserve/commit interface users to do this correctly - it would also require one to expose too much internals, you'd need to be able to locate page B for starters.
> If you use write rather than reserve, you have to copy all the data > twice for every event.
Well, once. I'm not seeing where the second copy comes from.
> > On top of that foundation build an eventbuffer, which knows about > > encoding/decoding/printing events. > > > > This too needs to be a flexible layer - > > That would be nice. However, we need to keep at least the length > and timestamp fields common so we can do parsing and the mergesort?
And here I was thinking you guys bit encoded the event id into the timestamp delta :-)
> +struct ring_buffer_event { > + unsigned long long counter; > + short type; > + short length; > + char body[]; > +} __attribute__((__packed__)) > > So type would move into the body here?
All of it would, basically I have no notion of an event in the ringbuffer API. You write $something and your read routine would need to be smart enough to figure it out.
The trivial case is a fixed size entry, in which case you always know how much to read. A slightly more involved but still easy to understand example might be a 7bit encoding and using the 8th bit for continuation.
Another option is to start out with a fixed sized header that contains a length field.
But the raw ringbuffer layer, the one concerned with fiddling the pages and writing/reading thereto need not be aware of anything else.
> > as I suspect the google guys > > will want their ultra-compressed events back. > > Is useful when gathering GB of data across 10,000 machines ;-) > Also reduces general overhead for everyone to keep events small.
Exactly - which is why a flexible encoding layer makes sense to me - aside from the abstraction itself.
| |