On Documents vs. Streams

The other day I was trying to figure out what was making my desktop computer run so slow. This was pretty unusual on Linux. Way back when I used to use Windows, I'd encounter such things due to running too many programs, so of course I looked at what was running.

But these days I don't really run that many intensive applications. I hardly ever use spreadsheets, presentation tools, word processors, or so on like I used to. Really the only things visible in my task bar were Firefox and Terminal. Killing off Firefox made no difference. And even more oddly, rebooting had no effect - within minutes my load average was up in the 20-30's.

Of course, the culprit was that I had a bunch of automatic cron jobs scheduled to occur at the same time. Reordering the timings got the system back on track.

But this led to an epiphany moment.

This was my *desktop* machine. The last time I'd seen this cron job I/O blocking madness was on servers. What was I doing with so many cron jobs on a *desktop*??

I looked at the individual cron jobs, and found that each and every one of them had something to do with my job. Disabling any one of them would reduce my work productivity. I could no longer do without one of these silly cron jobs than I could have done without Microsoft Excel a decade ago.

And then, as I looked around my desktop at the other things running on it - internet radio, IRC chat, mutt - that I noticed that I hardly ever use traditional "document-centric" applications. Everything running had something to do with dealing with _streams_ of information.

The realization sank in at that point, that our computer UI paradigm really stinks for what we actually use our computers for.

The prevailing UI paradigm today is built around the notion of document authoring. It expects that the main thing you do is create spreadsheets, word documents, presentations, and so on. There is a task bar to remind you of what documents you're editing, there is cross-application cut and paste so you can put pieces of one document into another. You can place documents on your desktop surface itself, so you can organize your work. You can define which applications to use for which types of docs. You can set up a default printer to put your documents to hard copy. You can set up system-wide fonts to use in documents. You can put icons to apps and even documents onto your panel. And on and on.

Occasionally, some of that is actually useful to me. But most of the time, for most of the work I do, it's all irrelevant. To pick one example, I used absolutely none of that in order to write this blog post.

Really, what I mostly do today is stream management. And I suspect this is true for the vast majority of people. I don't deal with writing documents, but with changes to documents. I put comments onto things. I slap patches onto things. I tweak the states of things. Once in a rare while I may author a completely new thingee, but even there I usually end up working with it as a stream of changes that I build up over time (and usually in collaboration with a few other people who stream changes to me).

Thinking about user interfaces this way, it occurs to me that there are a LOT of different kinds of streams. Streams of emails. Streams of spam to take *out* of my email stream. Streaming radio broadcasts. Streams of bug reports. Streams of software updates to apply. Streams of chatter on IRC. Streams of youtube video URL's from friends and family to check out. Streams of updates to my weather applet. Even my todo list is more like a stream of things coming in and going out, than like a static document I can print and save and be done with.

Each one of those silly cron jobs bogging down my computer were critically important for me, because THEY were the tools I used for helping me stay atop all these streams. They summarized my bug streams in various ways. They filtered my email streams into more organized sub-streams. They flagged tasks (and took care of certain tasks for me). They took care of updating the tools I used day to day.

It's weird that for all the document tools at my fingertips, it's a stodgy old *server* tool that is helps my productivity the most. I wonder how non-technical people not knowledgeable about crontab and scripting deal with such things?

This all got me to a key question... Since the purpose of our desktop UI is to make our work easier and more efficient, then if today's knowledge workers are, like me, more stream-oriented than document-oriented, then doesn't it stand to reason that we ought to re-think our UI design to optimize it for making stream management easier and more efficient? How would such optimization be done? How would such a UI look and feel? What kinds of toolkits would be needed?

Posted in | Submitted by bryce on Tue, 2008-06-24 23:10.
bryce's blog

I did have the luck to play with an OLPC at a Ubuntu conference last year and I was pretty blown away by its interface. The idea that it could do away with 'save'/'load' was quite an eye opener, although it was a concept I'd run across a few other times before then. But it was the first time I'd seen an entire UI designed without 'load/save' as part of its core metaphor. Playing with that OLPC made me wonder if there's more of today's existing UI metaphors we can question.

bryce | Thu, 2008-06-26 07:25

Check out the "Sugar" interface designed by the One Laptop Per Child Foundation. Instead of a file manager, it uses a journal.

(Sugar also has interesting collaboration and presence features built into the main UI as well as most applications.)

Matt Brubeck (not verified) | Thu, 2008-06-26 04:23

There's a good analogy in the comments on reddit: "Documents are a "thing". Streams aren't a thing, they are bunch of things presented serially. Streams of documents, streams of events, streams of bits. Saying documents vs. streams is like saying cars vs. highway." "Yes, but, from what I understood from what he is saying is.. He used to drive those cars, now he simply watches the highway, sometimes talking to the drivers."

bryce | Wed, 2008-06-25 21:54

Your blog-post is linked to by Reddit. You might want to check the comments there as well..

http://www.reddit.com/info/6owbv/comments/

Meneer R (not verified) | Wed, 2008-06-25 21:38

Unfortunately parts of this concept are patented. Google up "David Gelernter" and lifestream. Mirrorworlds Technology and Scopeware are his two prior companies that folded after some fairly poor implementations of this. But he still holds the patents on lifestreaming technology. (see:
http://cs-www.cs.yale.edu/homes/freeman/lifestreams.html
http://www.sigchi.org/chi96/proceedings/videos/Fertig/etf.htm )

There's kind of an explosion of the concept lately, see:
http://lifestreamblog.com/lifeinlines-and-lifeblob-two-new-lifestreaming-services/

Here's to hoping that none of the newer Web 2.0 implementations of the idea get sued by him.

Some of the posters in these comments have some nice new twists to it too. I hate patents :-(

-David Mercer
Tucson, AZ

Anonymous | Wed, 2008-06-25 19:32

The idea of moving the desktop metaphor to something tracking the streams of data events in our lives is at the heart of David Gelertner's Lifestreams project, which folded as a company, but has a continuing influence. Some of the ideas from it have been showing up in OS X, for instance.

I personally find little use for big software such as word processors, preferring to keep things in text, possibly with lightweight markup, and convert it to whatever end form I need, whether that is a Word document, a web page, presentation slides, PDF, whatever. But that model, while very accessible to me as a programmer (the document as source code), is not yet easily used, or understood, by my wife or kids.

They understand blogs though, and email, and IM, so the pieces are there. I think a great desktop replacement metaphor would help others make the conceptual/contextual leap just like the WIMP (windows, icons, menus, pointer) interface helped break software away from the command-line.

Anonymous | Wed, 2008-06-25 18:37

A relevant toy problem to try some of these ideas out on:

Does SVG (or at least the SVG emitted by Inkscape) play nicely with revision control systems like bzr? If so, is there room for a bzr (or generic RCS) interface in Inkscape to track changes on a collaborative document?

Anonymous | Wed, 2008-06-25 17:07

You have just deftly defined the difference between a typewriter and a pc. Not to be too sarcastic, I think it is well beyond time that the rest of the world began to think like this. Our brains do not like static document models, and neither do we in daily life. This applies not just to computers but to everything. On the one hand it's nice to be able to memorize the applicable laws for your job. On the other it's nice to have instant reference to any new changes so as to incorporate them into your work. Google Apps and others are trying to bring that model mainstream. When you sit down and put your streaming data for the home office on a server, and access only those parts that you want to see you have a home-made version of such on-line apps. I actually recommend it. Push the 'busy work' off to a server and let your laptop relax a bit so you don't stress out.

Anonymous | Wed, 2008-06-25 16:27

It's an interesting concept. One could consider viewing such a "life stream" at different "altitudes".

First the data... Consider for a second a simple database that does nothing but capture your stream. Lets say for the sake of demonstration an Incredibly Basic db table, with 6 columns - id, owner, type, topic, content, created. Every time something came down the line, it would be stored in this simple database.

So what would be a stream? Well, everything...

A new line in IRC
An IM
A Twitter update
A document revision by a collaborator
An Email
A phone call
A facebook profile update
Scheduling of a doctors appointment
A Charge on a credit card
Shipping Status
Shipment Delivery
Democratic Vote

Every moment in your life that could be somehow timestamped would be a line item in the stream.

id = 1
owner = bob@example.com
type = IRC
topic = programming
content = how do i loop through an array?
created = (date-time)

id = 2
owner = me@example.com
type = document revision
topic = letter to dad
content = Happy Father's Day!
created = (date-time)

id = 3
owner = me@example.com
type = checking account 1, decrease
topic = Some Bar
content = $42.57
created = (date-time)

id = 4
owner = me@example.com
type = media: play
topic = television
content = BSG, Season 3, Episode 5
created = (date-time)

id = 5
owner = home@example.com
type = doorbell
topic = ''
content =
created = (date-time)

and so on.

From that point you have a complete history of all interactions and can interact with it at different "altitudes". You can view your stream from "high up" where you see everything happening as it happens. Or in broad scope, viewing all bank transactions or all items owned by soandso@example.com. Or you can get into the details and categorize, where you view the latest draft of item with type document, topic project 1 - file a.

The complexity wouldn't necessarily be in the act of storage, but mostly in the retrieval in any useful manner. That, and getting all systems to actually report to this one system for storage.

Otherwise a major issue might be Privacy (should be hosted in a way that only the owner can access the stream since it holds so much personal information).

Could prove to be quite a powerful system.

enobrev | Wed, 2008-06-25 16:19

Absolutely.

My vision of a desktop environment dedicated to streams is also more task focussed - each incoming stream has priorities, so your current working mode "casual"/"concentrated"/"in the zone", will determine which streams flash to get your attention and which simply pile up waiting for you. Some are marked transient (like digg), so if you don't read them immediately they just fade away. Some are permanent like your emails or smses. All of them can be searched. Streams can be combined (short emails can be displayed and interacted with in the same way as IM and SMS), split, merged.

The point is that I don't care how the stream content got to me, (SMS, RSS, web page, email, postal mail that is then scanned, message from God modulated on cosmic rays), what I care about are it's properties of relevance to the way I interact with it, how long is it, should it interrupt what I'm doing or can it wait, can I respond to it or discuss it with friends, is someone waiting for a response on the other end, does it need to add something to my todo stream. Synchronous or Asynchronous.

All of the applications should be scriptable. I imagine them working a little like the messages in Black and White, sliding inobtrusively down the side of the screen unless I pay attention to them, and the whole desktop designed more along the tabs/panes snappable model common in IDEs.

I don't believe in the "save" model of interaction either. All changes to documents should be permanent as you make them, but you have the ability to checkpoint and roll back to any previous version.

kyb

Anonymous | Wed, 2008-06-25 16:01

Starting the command with ionice -c3 gives the processes idle disk priority for read calls.

If the write queue gets full try increasing /proc/sys/vm/dirty_ratio which will help provided that you have enough RAM to spare.

http://www.westnet.com/~gsmith/content/linux-pdflush.htm has more info

Anonymous | Wed, 2008-06-25 14:58

You're very close to the idea of this blog post from cgwalters: http://cgwalters.livejournal.com/5659.html , just an other angle. I'd suggest getting familiar with it.

Also the comments are very high of value.

Anonymous | Wed, 2008-06-25 14:55

This is an issue that has been weighing on me heavily lately too.

Mostly my interactions with streams are in the form of RSS feeds, reading them, remixing them and producing them again. What I want are tools that make it easy for me to produce feeds based on the media I want to share with my other devices and other people. I'm really looking forward to Microsoft's Live Mesh product because I think it'll be a very easy way for people to make feeds without really trying.

As for the UI changes required for stream oriented thinking perhaps we need a new metaphor other than the desktop one. Perhaps we can look into some experimental or conceptual UI metaphors. I did a cursory google search and found this one based around a timeline. Is that better suited to your workflow?

- closetgeekshow [at] gmail.com
(Please ignore my user account request, I thought I had to do that to comment. )

Anonymous | Wed, 2008-06-25 14:40

yep, I thought of this a long while ago and it's been a central part of my many attempts to design a better User Land.
I really should make this.

Anonymous | Wed, 2008-06-25 13:22

First of all, this means the whole unix architecture has to go out of the window. (it was only a lit bit better than the windows architecture)

If we are at that, let's throw out the C-Calling-Convention as well. Layers on layers, CCC, DBUS, Kernel ..

.NET and JAVA also got it wrong (threads should be natural state of any new object, not something we manage ourselves, but something that is intrinsic to the design of the framework and possible the languages used)

The solution exists though. It's called functional reactive programming. That's the academic term for what is usually meant with 'agent programming'. Every object is an independent 'thread'. We distinguish between 'functions' and 'procedures'. (questions and orders). When we ask another agent a question, we wait for an answer (we block). When we give another agent an order, we continue with our own stuff, he'll get back to us if needed.

The most important feature is that no agent is ever consistently busy. They are either doing a finite amount of computation, or waiting for an answer of another agent, or not doing anything at all. The internal computation steps are not of the turing-level. They can not loop indefinately; they only way to do that, is to sent _yourself_ message. This way you don't even need preemptive multi-tasking.

Then, things like 'physically stored files' could be a side-effect of to-disk-serializing of the state of some agent. (let's call them persistant agents)

The knowledge exists. But the problem with upgrading the linux infrastructure like this, is that we have to kick out the traditional path-oriented device structure, the tradition c-calling-convention, etc.

It's not going to be backwards compatible. And it needs to start at the kernel level.

There are also some added benefits: plug and play comes naturally, scheduling of tasks becomes an integral/central part of the OS (fuck cron).

Technically, reactive-state-proccessing does with dataflow semantics, what object-oriented programming did with imperative semantics. It turns it into lego.

But this lego is intrinsicly concurrent, persistant and referentially transparent. (the last part is extremely interesting because it leads to machiene-provable safety constraints). And when a machiene can prove something does no harm, it could run in real-mode, rather than protected mode.

Anonymous | Wed, 2008-06-25 11:01

I think this is very insightful, indeed; and in the current climate of stormy change, it's exactly the thought that's needed. The new desktop paradigm isn't specifically "in the cloud", with web applications taking over, it's this. The Internet, web applications - they're all just tools to help us with our contemporary work, and we need to make the desktop accomodate those, and everything else.

Anonymous | Wed, 2008-06-25 10:00

hear hear!

"I wonder how non-technical people not knowledgeable about crontab and scripting deal with such things?"

uhm well, not at all, we hopelessly mess up and get lost...
It's a nightmare for semi technical people like me already, the average user is hopelessly inefficient.
it's really terrible, so I really applaud your remarks!

Anonymous | Wed, 2008-06-25 09:31

Frankly I think it's already on the Linux desktop. Workspaces + compiz plugins like scale accomplish stream management.

Vadi | Wed, 2008-06-25 02:58

Sure, if you take a snapshot of a stream, it is essentially a document, and in fact a lot of our tools use this.

In fact, I ran into this myself back when I was a college student writing a research paper with my research partner. We kept trading the document back and forth and were working on it simultaneously, and keeping our work in sync and merged together was a pain in the ass. In truth, we were collaborating in heavy stream mode, but limited by the document-orientation of our tools.

Of course, if I were to do such a thing today, we might write it up using a wiki or a version control system like bzr, or the revisions capabilities in OpenOffice... Of course, what I'd really want is some sort of a mash up of all three of those! (And then something like that, but for spreadsheets... or presentations... or...)

bryce | Wed, 2008-06-25 02:48

It's possible that we wouldn't need to change methods at all, since if there exists a backend capable of handling streams, then your streams of data would appear as documents. Note, however, that I'm currently a student, as well as futzing around with writing and some other stuff, so I'm still mostly document-based. :P

blewis004 | Wed, 2008-06-25 02:02

I thought of this months ago. If you keep thinking about this, you'll steal my idea!!

Seriously though, this is a common problem that's been thought about before. I haven't written about it in detail because I'm not sure how I want to go about it. There's actually several UIs and langauages available for what you're describing. They're called Dataflow Languages. I think they're much better at describing computation than OO. Perhaps we should get together sometime and share notes.

Actually I thought about this in detail on the drive from Manhattan to KC after having watched a lecture by the Adobe Open Source guy give a talk on "one possible future of software development". Its buried somewhere as a Google TechTalk.

jldugger

jldugger | Wed, 2008-06-25 01:10

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
More information about formatting options