⑨ lab ≡ ByteLabs

Plan9/Email

— Igor Böhm

DIY email client — acme(1), upasfs(4), winwatch(1), and rio(1) is all you need to effectively handle multiple IMAP accounts on 9front

⑨ lab ≡ 9front Email screencast…

term% @{ramfs ; cd /tmp ; hget https://9lab.org/vid/plan9/9mail.mp4 > 9mail.mp4 && treason 9mail.mp4}

Overview

The following describes my email setup on 9front, managing three IMAP accounts (i.e. 9lab.org, gmail.com, and bytelabs.org). A nested rio(1) window running winwatch(1) and three acme(1) instances acting as mail clients, one per IMAP account, is what I call my mail client ☺

Setup

Note that some of this stuff is shamelessly stolen and modified from the excellent 9front FQA.

Reading Mail

To access my three IMAP accounts I run the following:

 % cat $home/bin/rc/$user/init/mail
#!/bin/rc
# -- 9lab.org
upas/fs -f /imaps/mail.9lab.org/igor &
# -- gmail.com
upas/fs -f /imaps/imap.gmail.com/boehm.igor@gmail.com -m /mail/gmail.com &
# -- bytelabs.org
upas/fs -f /imaps/imap.bytelabs.org/igor -m /mail/bytelabs.org &
wait
# -- 9lab.org folders
@{
	for(f in (Junk Trash Sent Hacking Finance House Archive)){
		echo open /imaps/mail.9lab.org/igor/$f $f >/mail/fs/ctl &
	}
} &

…from my $home/lib/profile:

 % cat $home/lib/profile
bind -qa $home/bin/rc /bin
bind -qa $home/go/bin /bin
bind -qa $home/bin/$cputype /bin
font=/lib/font/bit/vga/unicode.font
editor=(sam -d -a)
GO111MODULE=on
switch($service){
case terminal
	auth/factotum
	webcookies
	webfs

	# namespace lifting
	srvfs plumbspace.$user.$pid /n
	plumber
	rfork n
	mount -b /srv/plumbspace.$user.$pid /n

	echo -n accelerated > '#m/mousectl'
	echo -n 'res 3' > '#m/mousectl'
	prompt=('term% ' '	')
	fn term%{ $* }
	editor=(sam -a)
	$user/init/mail
	rio -i $user/init/rio/start
	…

Note the invocation of $user/init/mail just before starting rio(1)

Running the above for the first time errors out like this:

upas/fs imap: server certificate 22471E10D5C1E41768048EF5567B27F532F33
    not recognized
upas/fs: opening mailbox: bad server certificate

To add this certificate to your system, type:

echo 'x509 sha1=22471E10D5C1E41768048EF5567B27F532F33' \
    >>/sys/lib/tls/mail

You will have to do this for every account. The following opens additional labels or folders such as Junk:

echo open /imaps/mail.9lab.org/igor/Junk    Junk     >/mail/fs/ctl

It is wise to add the required imap service entries to factotum(4) to avoid having to type in passwords. Some examples from my setup:

% echo 'key proto=pass service=imap server=imap.gmail.com \
	user=boehm.igor@gmail.com !password=☠' >/mnt/factotum/ctl
% echo 'key proto=pass service=imap server=mail.9lab.org \
	user=igor !password=☠' >/mnt/factotum/ctl
% echo 'key proto=pass service=imap server=imap.bytelabs.org \
	user=igor !password=☠' >/mnt/factotum/ctl

Having an acme(1) instance per mail account is achieved with this script:

 % cat bin/rc/amail
#!/bin/rc
rfork n
fn Help{
  echo `{basename $0}^' (9lab|gmail|bytelabs|all)'
}
fn Wait{
  while(! test -e $1){ sleep 4 }  
}
fn Mail{
  label Mail:$1
  bind /dev/null /dev/label
  if(test -e /mnt/plumb/edit)
  	bind /dev/null /mnt/plumb/edit
  if(test -e /mnt/term/mnt/plumb/edit)
  	bind /dev/null /mnt/term/mnt/plumb/edit
  if(test -e $home/bin/rc/a/mail/Move)
  	bind -a $home/bin/rc/a/mail /acme/bin
  acme -l $home/lib/a/dump/mail.$1
}
switch ($#*) {
	case 0
		Help
	case *
		switch($1){
			case 9lab
				Wait /mail/fs/Archive
				Mail 9lab
			case gmail
				upasname=boehm.igor@gmail.com
				Mail gmail
			case bytelabs
				Mail bytelabs
			case *
				Mail all
		}
}

Note how for gmail.com we set the upasname to the gmail address.

The above assumes you have a folder $home/bin/rc/a/mail that contains the below Move script (see Appendix) and binds it to /acme/bin:

  if(test -e $home/bin/rc/a/mail/Move)
  	bind -a $home/bin/rc/a/mail /acme/bin

The above also assumes you have dumped the arrangement of IMAP mailboxes via Dump in acme(1) to $home/lib/a/dump/mail.$1 where $1 is the name of the mailbox. The dumps are then loaded with this command:

  acme -l $home/lib/a/dump/mail.$1

After some swearing, scratching your head, and trying around, you should be able to fire up an acme instance for a mail account like this:

 % amail 9lab.org

The above starts an acme(1) instance with all the IMAP folders for my 9lab.org account.

With these ingredients you can create a script that opens a sub rio(1), and start three acme(1) instances and winwatch(1) to achieve the same setup as in the screencast:

⑨ lab ≡ 9front DIY Email Client…

9front DIY Email Client

Sending Mail

Additional setup is required to be able to send emails. 9lab.org mails are sent via mail.9lab.org and gmail mails are sent via smtp.gmail.com.

First lets add the required passwords to factotum:

% echo 'key proto=pass service=smtp server=mail.9lab.org
	user=igor !password=☠' >/mnt/factotum/ctl
% echo 'key proto=pass service=smtp server=smtp.gmail.com
	user=boehm.igor@gmail.com !password=☠' >/mnt/factotum/ctl

Modify /mail/lib/remotemail to gateway mail through your account:

% cat /mail/lib/remotemail
#!/bin/rc
shift
sender=$1
shift
addr=$1
shift
fd=`{/bin/upas/aliasmail -f $sender}
switch($fd){
case 9lab.org
	addr=tcp!mail.9lab.org!587
	user=()
case gmail.com
	addr=tcp!smtp.gmail.com!ssmtp
	user=(-tu $sender)
case bytelabs.org
	fd=bytelabs.org
	addr=tcp!mail.9lab.org!587
	user=()
case *
	fd=9lab.org
	addr=tcp!mail.9lab.org!587
	user=()
}
exec /bin/upas/smtp -a -h $fd $user $addr $sender $*

Note how bytelabs.org and 9lab.org use the same dialstring as the addr of the smtp server. Furthermore note how the user variable is set. It is subtle but this setup allows one to use gmail and a proper mail service in one configuration.

Before this will work you need to retrieve the certificate hash. This can be done by trying to send an e-mail and then looking for the hash in /sys/log/smtp:

% echo hello | mail -s test your.username@gmail.com

Then look in /sys/log/smtp for the following error:

cert for smtp.gmail.com not recognized:
    sha256=wnu7Uuzq4MlyJHP90+8f2smoh6x3cj0dG5z02jJlX42

Add the certificate to your system:

% echo 'x509 sha256=wnu7Uuzq4MlyJHP90+8f2smoh6x3cj0dG5z02jJlX42' \
    >> /sys/lib/tls/smtp

Make sure to include the following mappings as well:

% cat /mail/lib/fromfiles
# files listed here will be consulted for aliases
names.remote

% cat /mail/lib/names.remote
boehm.igor@gmail.com gmail.com
igor@9lab.org 9lab.org
igor@bytelabs.org bytelabs.org

% cat /mail/lib/names.local
# alias file, listed in /mail/lib/namefiles

# postmaster goes to igor
postmaster	igor

Obviously you have to adapt the above to your mail addresses.

I have also added mail.9lab.org to /lib/ndb/local:

% grep smtp /lib/ndb/local
smtp=mail.9lab.org

Searching Mail

Searching for emails is accomplished via acme(1)’s builtin Look command, and is therefore limited to the contents displayed in a window. To enable more advanced Search for patterns in various fields (e.g. from, subject, body, …) the following script can be used by invoking it in the tag line of a mailbox:

 % cat /acme/bin/Search
#!/bin/rc
# Search: "Search [field] pattern"
rfork n
if(! ~ `{pwd} /mail/fs/*){
 echo Must run in mail directory >[1=2]
 exit 'bad dir'
}
fields=(from to cc bcc date header subject body)
flagfmt=''
args='('^`{echo $fields|sed 's/ /|/g'}^') pattern'
if(! ifs=() eval `{aux/getflags $*} || ~ $#* 0){
	aux/usage
	exit usage
}
F=$1 ; shift   # field
if(! ~ $F $fields){
	aux/usage
	exit usage
}
P=$1 ; shift   # pattern
# -- search for pattern in field
walk -d -n1,1 `{pwd} | sed 's/$/\/'^$F^'/' | xargs -p 10 grep -n -- $P /dev/null

An invocation such as Search from 'rob@plan9.com' will produce all emails from rob@plan9.com in the +Errors window. Plumbing the folder path will open the respective message.

Attachements

The screencast shows how to send and open attachements. It is as simple as typing Attach: /path/to/your/attachement like this:

To: 9front@9front.org
Subject: intro(1): fix typo in BUGS section
Attach: /tmp/man-intro-1.patch

Hello,

Attached is a patch that fixes…

You can also inline attachements using the Include: directive. Read marshal(1) for more information on how mail is formatted and sent.

Screencast

Here is the email session as a screencast on youtube to give you a better idea of how this works in action:

Appendix

Save sent files in Sent

I have not yet tried it but sirjofri has a description of how to automatically save sent files in Sent.

Moving messages between IMAP folders

rsal shared a patch that enables moving of messages between IMAP folders for upasfs. This is useful if you try to keep your INBOX clean by moving mails into their respective folders. With the patch⁽¹⁾ applied you can move messages between folders like this:

 % echo move mbox 9 99 999 Archive > /mail/fs/ctl

That will move message numbers 9, 99, and 999 from your IMAP INBOX known as mbox to upasfs, to the IMAP ARCHIVE folder known as Archive to upasfs.

Note that the patch has been applied to 9front and is therefore part of the base system.

If you are using acme to read your emails the following small script is useful to speed up moving of messages so you don’t have to write the above by hand:

 % cat /acme/bin/Move
#!/bin/rc
rfork n
if(! ~ `{pwd} /mail/fs/*){
 echo Must run in mail directory >[1=2]
 exit 'bad dir'
}
# -- tgt...IMAP mail box name
tgt=$*
if(~ $#tgt 0){
 echo No target folder specified >[1=2]
 exit usage
}
# -- src...extract upas mail box name from tag
# 's/^\/([^ ]+).*$/\1/' … get path excl. initial '/'
# 's/\/$//' … drop final '/'
# 's/[^\/]+\///g' … drop everything before last '/'
# '1q' … stop processing after 1 line
src=`{sed 's/^\/([^ ]+).*$/\1/
s/\/$//
s/[^\/]+\///g
1q' /mnt/acme/$winid/tag}
if(! ~ $#src 1){
 echo Could not determine source folder >[1=2]
 exit usage
}
# -- msgs...extract message numbers from selection in window
msgs=`{ssam 'x/.*\n/ s/^([0-9]*).*\n/\1 /' /mnt/acme/$winid/rdsel}
if(~ $#msgs 0){
 echo No messages selected >[1=2]
 exit usage
}
# -- move $msgs from $src to $tgt
echo move $src $msgs $tgt > /mail/fs/ctl

Now you simply put Move Archive into the tag of the window that contains messages you want to move to Archive. After selecting all message lines you want to move middle click on Move Archive.

patch⁽¹⁾: moving messages between IMAP folders

diff -r 7c895ae504fa sys/man/4/upasfs
--- a/sys/man/4/upasfs	Fri May 28 13:02:58 2021 +0200
+++ b/sys/man/4/upasfs	Mon May 31 13:55:01 2021 +0300
@@ -206,7 +206,7 @@
 .I fs
 to open, close, rename, create or remove new mailboxes,
 and also to
-delete or flag groups of messages atomically.
+delete, flag, or move groups of messages atomically.
 The messages that can be written to this file are:
 .TP 2i
 .PD 0
@@ -260,6 +260,13 @@
 .TP
 .B "flag \fImboxname flags number ...\fP
 flag the given messages.
+.TP
+.B "move \fImboxname number ... target\fP
+Move the given messages from
+.IR mboxname
+to mailbox named
+.IR target.
+At the moment only supported with IMAP mailboxes.
 .PD
 .PP
 The
diff -r 7c895ae504fa sys/src/cmd/upas/fs/dat.h
--- a/sys/src/cmd/upas/fs/dat.h	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/dat.h	Mon May 31 13:55:01 2021 +0300
@@ -174,6 +174,7 @@
 	void	(*decache)(Mailbox*, Message*);
 	int	(*fetch)(Mailbox*, Message*, uvlong, ulong);
 	void	(*delete)(Mailbox*, Message*);
+	char	*(*move)(Mailbox*, Message*, char*);
 	char	*(*ctl)(Mailbox*, int, char**);
 	char	*(*remove)(Mailbox *, int);
 	char	*(*rename)(Mailbox*, char*, int);
@@ -215,6 +216,7 @@
 void		unnewmessage(Mailbox*, Message*, Message*);
 char*		delmessages(int, char**);
 char		*flagmessages(int, char**);
+char*		movemessages(int, char**);
 void		digestmessage(Mailbox*, Message*);

 int		wraptls(int, char*);
diff -r 7c895ae504fa sys/src/cmd/upas/fs/fs.c
--- a/sys/src/cmd/upas/fs/fs.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/fs.c	Mon May 31 13:55:01 2021 +0300
@@ -1242,6 +1242,11 @@
 				return nil;
 			return flagmessages(argc - 1, argv + 1);
 		}
+		if(strcmp(argv[0], "move") == 0){
+			if(argc < 4)
+				return nil;
+			return movemessages(argc - 1, argv + 1);
+		}
 		if(strcmp(argv[0], "remove") == 0){
 			v0 = argv0;
 			flags = 0;
diff -r 7c895ae504fa sys/src/cmd/upas/fs/imap.c
--- a/sys/src/cmd/upas/fs/imap.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/imap.c	Mon May 31 13:55:01 2021 +0300
@@ -1025,6 +1025,29 @@
 }

 static char*
+imap4move(Mailbox *mb, Message *m, char *dest)
+{
+	char *r;
+	Imap *imap;
+
+	imap = mb->aux;
+	imap4cmd(imap, "uid copy %lud %s", (ulong)m->imapuid, dest);
+	r = imap4resp(imap);
+	if(!isokay(r))
+		return r;
+	imap4cmd(imap, "uid store %lud +flags (\\Deleted)", (ulong)m->imapuid);
+	r = imap4resp(imap);
+	if(!isokay(r))
+		return r;
+	imap4cmd(imap, "expunge");
+	r = imap4resp(imap);
+	if(!isokay(r))
+		return r;
+	m->inmbox = 0;
+	return 0;
+}
+
+static char*
 imap4sync(Mailbox *mb)
 {
 	char *err;
@@ -1183,6 +1206,7 @@
 	mb->delete = imap4delete;
 	mb->rename = imap4rename;
 	mb->modflags = imap4modflags;
+	mb->move = imap4move;
 	mb->addfrom = 1;
 	return nil;
 }
diff -r 7c895ae504fa sys/src/cmd/upas/fs/mbox.c
--- a/sys/src/cmd/upas/fs/mbox.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/mbox.c	Mon May 31 13:55:01 2021 +0300
@@ -1137,6 +1137,37 @@
 	return rerr;
 }

+char*
+movemessages(int argc, char **argv)
+{
+	char *err, *dest, *rerr;
+	int i, needwrite;
+	Mailbox *mb;
+	Message *m;
+
+	rerr = 0;
+	for(mb = mbl; mb != nil; mb = mb->next)
+		if(strcmp(*argv, mb->name) == 0)
+			break;
+	if(mb == nil)
+		return "no such mailbox";
+	if(mb->move == nil)
+		return "move not supported";
+	dest = argv[argc - 1];
+	needwrite = 0;
+	for(i = 1; i < argc - 1; i++)
+		for(m = mb->root->part; m; m = m->next)
+			if(strcmp(m->name, argv[i]) == 0){
+				if(err = mb->move(mb, m, dest))
+					rerr = err;
+				else
+					needwrite = 1;
+			}
+	if(needwrite)
+		syncmbox(mb, 1);
+	return rerr;
+}
+
 void
 msgincref(Mailbox *mb, Message *m)
 {
diff -r 7c895ae504fa sys/src/cmd/upas/fs/mdir.c
--- a/sys/src/cmd/upas/fs/mdir.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/mdir.c	Mon May 31 13:55:01 2021 +0300
@@ -294,5 +294,6 @@
 	mb->idxread = idxr;
 	mb->idxwrite = idxw;
 	mb->ctl = mdirctl;
+	mb->move = nil;
 	return nil;
 }
diff -r 7c895ae504fa sys/src/cmd/upas/fs/plan9.c
--- a/sys/src/cmd/upas/fs/plan9.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/plan9.c	Mon May 31 13:55:01 2021 +0300
@@ -411,5 +411,6 @@
 	mb->remove = localremove;
 	mb->rename = localrename;
 	mb->decache = plan9decache;
+	mb->move = nil;
 	return nil;
 }
diff -r 7c895ae504fa sys/src/cmd/upas/fs/pop3.c
--- a/sys/src/cmd/upas/fs/pop3.c	Fri May 28 13:02:58 2021 +0200
+++ b/sys/src/cmd/upas/fs/pop3.c	Mon May 31 13:55:01 2021 +0300
@@ -624,6 +624,7 @@
 	mb->sync = pop3sync;
 	mb->close = pop3close;
 	mb->ctl = pop3ctl;
+	mb->move = nil;
 	mb->addfrom = 1;
 	return nil;
 }

#Plan9 #9front