?

Log in

No account? Create an account
 

Taking a REST from Email - 410

About Taking a REST from Email

Previous Entry Taking a REST from Email Oct. 8th, 2008 @ 12:09 pm Next Entry

I had a play with installing the RoundCube webmail system on my server yesterday and was pretty impressed. It does seem to do what it says on the tin and act as a AJAX-y web front end to an IMAP server.

This encouraged me to place my web developer hat[0] upon my head and have a ponder. “Surely JavaScript » Some XML-y RPC » PHP » IMAP » My Mail and the associated return path is somehow sub-optimal, requiring, as it does, two separate servers on my email hosting box”, says I to myself. “Not to mention the overhead of said box talking IMAP to itself!” I added.

Astonished that such a sentence could emerge from my mouth, abusing, as it does, a French punctuation symbol, I started Googling around for an IMAP-like protocol which was more RESTful. Perhaps I should explain at this point that RESTful is the Web 2.0 buzzword du jour for ’sensible API for accessing stuff over the web, innit’. The Big Idea I had was making the mail reader truly just a hunk of HTML + JavaScript which could access the mail directly.

I failed to find such a (public) API. Do please link in comments if my Googling was insufficient.

Then I stumbled upon a description of the Maildir on-disk mailbox format which my IMAP server was getting all the mail from and it struck me that, actually, the Maildir format itself is RESTful. It certainly ticks many REST boxes:

  • Authentication is devolved to some higher power. In REST APIs this is generally some HMAC sent with the request or using the Auth mechanism built into HTTP. With Maildir, any authentication/permissions/etc are handled by the filesystem/OS.
  • Data are organised as resources. Each email is a file with an unique ID located by a semantically valid resource locater (i.e. a folder subdirectory in Maildir++ boxes).
  • It is stateless. Maildirs were designed to allow stateless, lock-free[1] concurrent access. Just what a RESTful API craves.

This set me wondering. Assuming I have some appropriately cunning Apache config which lets me access my Maildir via WebDAV under something like http://example.com/~user/mail/…, would this be sufficient to write a mail reader[2] entirely in HTML + JavaScript? I am far to busy to actually investigate this myself but the idea is almost irresistible to my mind.

[0] A funky little beret naturally!

[1] Well, locking handled by the file system anyhow. It has been pointed out that there are some conditions where Maildirs are not the panacea they appear in this regard but for the purposes of this discussion I’ll stick my fingers in my ears about it.

[2] Reading. i.e. without the actual sending mail part.

Update: So it appears that one of the protocols Exchange and Outlook speak for exchanging emails is WebDAV. Great minds…

Leave a comment
From:(Anonymous)
Date:October 8th, 2008 01:03 pm (UTC)

Mail over WebDAV

(Link)
Well, no, not really. Such a WebDAV folder would need to deal with 10.000 mails or so and pure WebDAV itself has no query language. In a ajax web client you would want to issue queries like "give me the 10 last mails", thats not possible with pure WebDAV (as implemented in mod_dav). It might be viable in combination with a local cache (eg using Gears). But you still need to parse the MIME, which is hard to get right and dealing with xy MB files in JavaScript isn't exactly efficient either.

There is DASL which puts a query language on top of WebDAV. This is implemented by a few (more complex) WebDAV servers, but not by plain mod_dav.

Notably Exchange's Outlook Web Access (OWA) works like what you describe. You can query Exchange mailboxes using WebDAV, including support for DASL. HotMail also uses (used?) email-over-HTTP between Outlook and the backend.

Another system which is worth a look when you are into email/REST is ScalableOGo. It maps IMAP4 hierarchies to a HTTP/WebDAV hierarchy. Example:

http://sogo/mail/hh@zideone.com/INBOX/123/4.5.gif

If you GET this URL, you get the image contained in MIME part 4.5 of the message 123 in the INBOX folder of the IMAP4 account.

Have fun,
Helge

[User Picture Icon]
From:filecoreinuse
Date:October 8th, 2008 01:25 pm (UTC)

Re: Mail over WebDAV

(Link)
Such a WebDAV folder would need to deal with 10.000 mails or so and pure WebDAV itself has no query language.


Fair point. But that is a scalability problem, not an actual technical restriction on whether you can do it. I was merely wondering if it could be done :).

Notably Exchange's Outlook Web Access (OWA) works like what you describe. You can query Exchange mailboxes using WebDAV, including support for DASL. HotMail also uses (used?) email-over-HTTP between Outlook and the backend.


So I discovered (see post). It seemed to sit in some uncomfortable ground between stateful and stateless to my initial inspection however.

Another system which is worth a look when you are into email/REST is ScalableOGo. It maps IMAP4 hierarchies to a HTTP/WebDAV hierarchy.


Aha! That looks like almost exactly the sort of system I was thinking about. It still uses IMAP as a glue layer. It seems, in my gut, that talking to the Maildir directly would be better although talking IMAP does mean you can devolve the messiness of parsing email messages.
From:(Anonymous)
Date:October 8th, 2008 01:49 pm (UTC)

Re: Mail over WebDAV

(Link)
Yes, part of the issue is that the mails itself are also (often) structured. Well, you could even do this with plain Apache if you leave out maildir. You would need to write a small delivery script which is invoked by postfix when mail comes in. The script could parse the MIME message and instead of writing a single file, it would write a directory hierarchy, maybe even some index files, something like this:
  /inbox
     123/
        1.1.txt
        2.1.html
        2.2.gif
     123.mime (raw)

In the same run one could create WebDAV properties on the items which contain stuff like subject, read status etc. (which you could then query using a regular PROPFIND)

Not sure whether its worth the effort though. If you need to install an smtpd alongside your Apache, you can as easily install Dovecot or Cyrus. Whats the problem with running the additional daemon ..., Apache and imapd can both auth via PAM etc.
[User Picture Icon]
From:filecoreinuse
Date:October 8th, 2008 01:52 pm (UTC)

Re: Mail over WebDAV

(Link)
Whats the problem with running the additional daemon ..., Apache and imapd can both auth via PAM etc.


No real problem. It just seems somehow that it is a very thick layer of glue.
[User Picture Icon]
From:mas90
Date:October 8th, 2008 04:23 pm (UTC)
(Link)
The problem with accessing Maildirs directly is that any useful indices maintained by your IMAP server and/or MTA become useless. And pretty much everything sucks at accessing folders containing tens of thousands of files without some sort of index. (Perhaps this is just a sign that I should organise my mail better...)

I do however quite like the idea of a webmail app which implements IMAP in JavaScript. Not that the implementation is likely to be pleasant, mind.
[User Picture Icon]
From:mas90
Date:October 8th, 2008 04:23 pm (UTC)
(Link)
*headdesk*

s/MTA/MDA/
[User Picture Icon]
From:cozminsky
Date:October 13th, 2008 11:53 am (UTC)
(Link)
I know courier's sqwebmail accesses the maildir directly, however it does run a daemon that is accessed via a cgi script. I wonder how much their implementation could be put into the web application itself.
(Leave a comment)
Top of Page Powered by LiveJournal.com