MYRIGHTS not understood by Mirapoint IMAP4PROXY 3.5.8-GR server

[Christoph Berg <cb@…>]

The following was submitted as Debian bug #384076:

----- Forwarded message from David Everly <deckrider@gmail.com> -----

Date: Mon, 21 Aug 2006 12:27:56 -0600
From: David Everly <deckrider@gmail.com>
Reply-To: David Everly <deckrider@gmail.com>, 384076@bugs.debian.org
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: Bug#384076: mutt MYRIGHTS not understood by Mirapoint IMAP4PROXY
	3.5.8-GR server

I think the fact that MYRIGHTS is not understood by the imap server I
connect to causes mutt to think my folders are read-only.

If anyone knows of a patch I can use for this, I'll be glad to try it.

from mutt -d2:

Connected to imap.mcilink.com:143 on fd=4
4< * OK Mirapoint IMAP4PROXY 3.5.8-GR server ready
IMAP queue drained
4> a0000 CAPABILITY^M
4< * CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS
Handling CAPABILITY
4< a0000 OK Completed
IMAP queue drained
imap_authenticate: Using any available method.
local ip: 166.34.132.36;41513, remote ip:166.38.62.50;143
SASL: No worthy mechs found
imap_auth_sasl: IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS
unavailableSending LOGIN command for c-david.everly@mci.com...
4< a0001 OK LOGIN completed.
IMAP queue drained
4> a0002 LIST "" ""^M
4< * LIST (\Noselect) "/" ""
Root delimiter: /
4< a0002 OK LIST completed.
IMAP queue drained
4> a0003 MYRIGHTS "INBOX"^M
a0004 STATUS "INBOX.Drafts" (MESSAGES)^M
a0005 SELECT "INBOX"^M
4< a0003 BAD Protocol Error: "Unidentifiable command specified".

----- End forwarded message -----
----- Forwarded message from David Everly <deckrider@gmail.com> -----

Date: Mon, 21 Aug 2006 14:15:12 -0600
From: David Everly <deckrider@gmail.com>
Reply-To: David Everly <deckrider@gmail.com>, 384076@bugs.debian.org
To: 384076@bugs.debian.org,
	Adeodato =?UTF-8?Q?Sim=C3=B3 ?= <dato@net.com.org.es>
Subject: Bug#384076: CAPABILITY before and after LOGIN

I just noticed by connecting from telnet that the CAPABILITY results
are different before and after LOGIN.

Before:

CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS

After (note there is no ACL capability here):

CAPABILITY IMAP4 IMAP4rev1 IDLE LOGIN-REFERRALS MAILBOX-REFERRALS
NAMESPACE LITERAL+ UIDPLUS CHILDREN

----- End forwarded message -----
----- Forwarded message from David Everly <deckrider@gmail.com> -----

Date: Mon, 21 Aug 2006 15:35:22 -0600
From: David Everly <deckrider@gmail.com>
Reply-To: David Everly <deckrider@gmail.com>, 384076@bugs.debian.org
To: 384076@bugs.debian.org
Subject: Bug#384076: patch to recheck capabilities after login

--- imap/imap.c	2006-08-21 15:30:26.000000000 -0600
+++ imap/imap.c	2006-08-21 15:32:03.000000000 -0600
@@ -376,6 +376,7 @@
   }
   if (new && idata->state == IMAP_AUTHENTICATED)
   {
+    imap_check_capabilities (idata);
     /* get root delimiter, '/' as default */
     idata->delim = '/';
     imap_cmd_queue (idata, "LIST \"\" \"\"");

----- End forwarded message -----

Christoph
-- 
cb@df7cb.de | http://www.df7cb.de/

>Fix:
Unknown

[Brendan Cully <brendan@…>]

On Thursday, 08 March 2007 at 12:11, Kyle Wheeler wrote:
> On Thursday, March  8 at 07:25 PM, quoth Christoph Berg:
> >I think the fact that MYRIGHTS is not understood by the imap server I
> >connect to causes mutt to think my folders are read-only.

> For what it's worth, in this case, the IMAP server is broken. What causes 
> mutt to think that it can use the MYRIGHTS command is that the server 
> *claims* to be able to understand it, and then later contradicts itself.

Here's a thread about this on the IMAP protocol list:

http://mailman1.u.washington.edu/pipermail/imap-protocol/2006-August/000253.html

It's Mark Crispin's opinion that mutt needs to recheck after
login. It's Arnt Gulbrandsen's opinion (and mine) that the server is
broken:

http://mailman1.u.washington.edu/pipermail/imap-protocol/2006-September/000256.html

I'd rather not add another round trip, but maybe I have to. The simple
solution for both mutt and mirapoint is for the IMAP proxy server to
send a gratuitous capability command after authentication completes.

[Kyle Wheeler <kyle-mutt-dev@…>]

On Thursday, March  8 at 07:25 PM, quoth Christoph Berg:
>I think the fact that MYRIGHTS is not understood by the imap server I
>connect to causes mutt to think my folders are read-only.

For what it's worth, in this case, the IMAP server is broken. What 
causes mutt to think that it can use the MYRIGHTS command is that the 
server *claims* to be able to understand it, and then later 
contradicts itself.

>4> a0000 CAPABILITY^M
>4< * CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS

Notice that the server returned that one of it's capabilities is ACL.

According both to RFC 4314 (which defines the ACL capability) and RFC 
2086 (which was obsoleted by 4313), any server which claims it 
supports the ACL capability MUST support the commands SETACL, 
DELETEACL, GETACL, LISTRIGHTS, and MYRIGHTS.

>I just noticed by connecting from telnet that the CAPABILITY results
>are different before and after LOGIN.
>
>Before:
>
>CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS
>
>After (note there is no ACL capability here):
>
>CAPABILITY IMAP4 IMAP4rev1 IDLE LOGIN-REFERRALS MAILBOX-REFERRALS
>NAMESPACE LITERAL+ UIDPLUS CHILDREN

This is, well, stupid. Not only do they claim ACL support, but QUOTA 
support as well.

The patch you sent works around this, but you really ought to contact 
the authors of the IMAP server and tell them to fix their broken 
server. They shouldn't claim to support capabilities that they do not 
actually support, no matter whether you're authenticated or not.

~Kyle
-- 
Having a family is like having a bowling alley installed in your 
brain.
                                                         -- Martin Mull

[Kyle Wheeler <kyle-mutt-dev@…>]

On Thursday, March  8 at 01:17 PM, quoth David Champion:
>> This is, well, stupid. Not only do they claim ACL support, but 
>> QUOTA support as well.
>
> I'm not sure I understand you.  I surely don't mean to defend Mirapoint 
> (we use them, and I'm not a fan by a long shot) but it is legitimate 
> to offer different capabilities when authenticated than when not, no?

True, but do they actually support ACL and QUOTA commands when you're 
not authenticated? I doubt it. Essentially, they're *lying*.

I'm philosophically more comfortable with not providing the full list 
of capabilities to people who aren't authenticated than I am with 
claiming you can do things you can't simply because the client isn't 
authenticated. I mean, at that point, why not simply include all kinds 
of bogus things in your unauthenticated CAPABILITY response, eh? Like 
MAKESCOFFEE, PERPETUALMOTION, and NUCLEARWEAPONS. The reasons for not 
including those are all the  same: the server doesn't really have them 
(i.e. it is lying) and it's wasting bandwidth by claiming that it 
does.

> I don't see why they *would* in this case, but to me it seems like 
> they're not claiming to support ACL or QUOTA when authenticated. The 
> patch seems to be quite correct.

The patch correctly works *around* it. Nothing more.

I mean, logically, they could technically report that they don't 
support ANY capabilities to people who aren't authenticated. And I can 
respect that somewhat in terms of wanting to play your cards close to 
your chest (i.e. for the same reasons you might not want to make it 
easy to figure out what brand (e.g. dovecot) or version (e.g. 1.0rc26)  
of server you're running, because it gives the untrusted client more 
information than is strictly necessary). Plus, it's technically more 
accurate: NAMESPACE, UIDPLUS, QUOTA, ACL - these things all cannot be 
used by unauthenticated clients, so if you aren't authenticated, it 
makes sense to report that you are not capable of that (RFC 3501 
provides for CAPABILITY to depend on the state of the communication).

So, you've essentially got two possible, respectable, logically 
coherent philosophies on this matter: either untrusted clients need to 
know virtually nothing, or letting untrusted (i.e. unauthenticated) 
clients know your capabilities doesn't hurt. Sitting in between 
doesn't make any sense: if you don't trust your client enough to tell 
them that you support IDLE, why would you trust them enough to tell 
them you support NAMESPACE? And why would you waste bandwidth claiming 
to support things that you don't actually support? There's nothing to 
be gained by lying about that.

~Kyle
-- 
I believe that every human has a finite number of heart-beats. I don't 
intend to waste any of mine running around doing exercises.
                                                      -- Neil Armstrong

[Kyle Wheeler <kyle-mutt-dev@…>]

On Thursday, March  8 at 08:35 PM, quoth Christoph Berg:
> Still, other MUAs probably gracefully accept that broken behavior, 
> maybe mutt should as well?

Probably should... particularly given that it seems to be viewed as 
"okay" (even in the various IMAP RFC's) for the CAPABILITY response to 
change after authentication (and STARTTLS). I still don't think it's 
acceptable to *REMOVE* capabilities when you authenticate, or to claim 
capabilities that you don't actually have regardless of the state of 
the machine. But, if you only trust the most recent CAPABILITY 
response, my argument is kind of a moot point, and mirapoint could 
technically claim all kinds of nonsense in their initial CAPABILITY 
response if they wanted to. :)

Does make you wonder if they actually do support all the things they 
claim in the latter CAPABILITY response, though, or if just most 
clients don't actually use all of them.

~Kyle
-- 
To bathe a cat takes brute force, perseverance, courage of 
conviction---and a cat. The last ingredient is usually hardest to come 
by.
                                                       -- Stephen Baker

[David Champion <dgc@…>]

* On 2007.03.08, in <20070308191120.GO2924@c-76-18-79-168.hsd1.nm.comcast.net>,
*	"Kyle Wheeler" <kyle-mutt-dev@memoryhole.net> wrote:
> >
> >CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS
> >
> >After (note there is no ACL capability here):
> >
> >CAPABILITY IMAP4 IMAP4rev1 IDLE LOGIN-REFERRALS MAILBOX-REFERRALS
> >NAMESPACE LITERAL+ UIDPLUS CHILDREN
> 
> This is, well, stupid. Not only do they claim ACL support, but QUOTA 
> support as well.

I'm not sure I understand you.  I surely don't mean to defend Mirapoint
(we use them, and I'm not a fan by a long shot) but it is legitimate
to offer different capabilities when authenticated than when not, no?

I don't see why they *would* in this case, but to me it seems like
they're not claiming to support ACL or QUOTA when authenticated.
The patch seems to be quite correct.

-- 
 -D.    dgc@uchicago.edu        NSIT    University of Chicago

[David Champion <dgc@…>]

* On 2007.03.08, in <20070308194240.GP2924@c-76-18-79-168.hsd1.nm.comcast.net>,
*	"Kyle Wheeler" <kyle-mutt-dev@memoryhole.net> wrote:
> On Thursday, March  8 at 01:17 PM, quoth David Champion:
> >>This is, well, stupid. Not only do they claim ACL support, but 
> >>QUOTA support as well.
> >
> >I'm not sure I understand you.  I surely don't mean to defend Mirapoint 
> >(we use them, and I'm not a fan by a long shot) but it is legitimate 
> >to offer different capabilities when authenticated than when not, no?
> 
> True, but do they actually support ACL and QUOTA commands when you're 
> not authenticated? I doubt it. Essentially, they're *lying*.

They're lying *if* they don't support ACL and QUOTA when they say they
do.  If they're lying, then I agree the server is broken, but I don't
see that this changes anything -- it it were not lying and it changed
its capability list upon login, then the patch would still be necessary.

Is that a spec violation?  I might be wrong, but I didn't think so.

Anyway, you seem to be assuming that Mirapoint's server does not support
QUOTA or ACL before login, but I'm not sure why that assumption is
valid.


> I'm philosophically more comfortable with not providing the full list 
> of capabilities to people who aren't authenticated than I am with 

Is authentication necessary for mailbox access?  Does anonymous access
require login as an "anonymous" user?  Again, I could be wrong, but I
didn't think so.

-- 
 -D.    dgc@uchicago.edu        NSIT    University of Chicago

[David Champion <dgc@…>]

* On 2007.03.08, in <E1HPPx1-0000ND-Sx@trithemius.gnupg.org>,
*	"Kyle Wheeler" <kyle-mutt-dev@memoryhole.net> wrote:
>  
>  Probably should... particularly given that it seems to be viewed as 
>  "okay" (even in the various IMAP RFC's) for the CAPABILITY response to 
>  change after authentication (and STARTTLS). I still don't think it's 
>  acceptable to *REMOVE* capabilities when you authenticate, or to claim 

It seems to be fairly common, at least -- not just Mirapoint.  For
example, servers may take "STARTTLS" off the capability list after you
start TLS, or take AUTH=XYZ off once you authenticate.  It seems like a
reasonable way of telling the client that they shouldn't do it again.
(Maybe there are client/server combinations that support re-login, but
if a server doesn't, it seems prudent to stop advertising that it's an
available option.)


>  technically claim all kinds of nonsense in their initial CAPABILITY 
>  response if they wanted to. :)

But that's the whole point of CAPABILITY. :) You can claim anything, as
long as you accept responsibility for what compliant clients do with
that.  If it's undefined, a compliant client ignores it and there's no
problem.

-- 
 -D.    dgc@uchicago.edu        NSIT    University of Chicago

[Kyle Wheeler <kyle-mutt-dev@…>]

On Thursday, March  8 at 02:53 PM, quoth David Champion:
> Anyway, you seem to be assuming that Mirapoint's server does not 
> support QUOTA or ACL before login, but I'm not sure why that 
> assumption is valid.

Well, there's a couple reasons. The most obvious is that ACL commands 
are only allowed in the authenticated state (IMAP communications can 
be in one of three states: not authenticated, authenticated, and 
selected), according to the ACL RFC, 4314. See the table in section 4. 
(I'll explain more in a moment.)

But I also find it hard to imagine that a server author would go to 
the trouble of implementing SETACL, DELETEACL, GETACL, LISTRIGHTS, and 
MYRIGHTS (which is not a particularly trivial set of commands) for the 
anonymous user but would then claim the commands were "unidentifiable" 
for an authenticated user. If they really had supported them for only 
certain users, I would think a NO response would be more appropriate 
than a BAD response.

> Is authentication necessary for mailbox access?  Does anonymous 
> access require login as an "anonymous" user?  Again, I could be 
> wrong, but I didn't think so.

Strictly speaking? Mailbox access, for the most part, requires the 
server to be in the authenticated state. RFC 3501 (i.e. the IMAP RFC) 
says that the only commands allowed in the "not authenticated" state 
are CAPABIILTY, NOOP, LOGOUT, STARTTLS, AUTHENTICATE, and LOGIN. 
(Extensions may, of course, define more that can be done while not 
authenticated, but I'm just talking basic IMAP for the moment.) Doing 
things like SELECT, EXAMINE, LIST, and so forth all require the 
authenticated state as a minimum. And, as I mentioned before, ACL 
commands require the authenticated state as well.

So the question then becomes "how does one get into the authenticated 
state?" Obviously, you can do so by using either the LOGIN or 
AUTHENTICATE commands, but what about anonymous users? Well, here's 
what the IMAP RFC has to say:

     Server implementations MAY allow access to certain mailboxes 
     without establishing authentication. This can be done by means of 
     the ANONYMOUS [SASL] authenticator described in [ANONYMOUS]. An 
     older convention is a LOGIN command using the userid "anonymous"; 
     in this case, a password is required although the server may 
     choose to accept any password. The restrictions placed on 
     anonymous users are implementation-dependent.

     Once authenticated (including as anonymous), it is not possible to 
     re-enter the not authenticated state.

The phrasing they choose is a little wonky, but that says (to me) that 
you can have anonymous access, but you need to do it by 
"authenticating" (a poor choice of words) as the anonymous user in 
some way. Someone could argue that their means of establishing 
anonymous authentication is through "connection" (i.e. everyone that 
connects is insta-magically authenticated as an anonymous user), but 
if that were true, then the server would be required by the RFC to 
greet the user with the PREAUTH response, which it didn't do.

Anyway, yes, ACL at least, and probably QUOTA (though the QUOTA RFC 
doesn't say anything about authentication), only makes sense in an 
authenticated state, and the Mirapoint server is giving the client 
every reason to believe that it begins the connection unauthenticated.

Thus, claiming ACL or QUOTA support at the beginning is misleading at 
best and flat wrong at worst. I suppose it's possible that Mirapoint 
implements ACL and QUOTA support for some users and not for others, 
though I (for no good reason) rather doubt that. But either way 
there's no point in advertising it up front if they're going to insist 
that IMAP clients must automatically check CAPABILITY *after* 
authenticating. No matter how god-like their implementation of ACL and 
QUOTA for *some* users (assuming they have one), advertising them 
before authentication is wasting bandwidth (admittedly, not very much) 
for all those users who are not sufficiently deserving of ACL support.

~Kyle
-- 
In any conversation where only one side may be argued, we 
instinctively assume that those who publicly defend the official 
position are motivated by ideology and not by an interest in truth. 
The outlawed position is then assumed to be true, and wins by default.
                                              -- Fr. Andrew Hamilton SJ

[Christoph Berg <cb@…>]

Re: Kyle Wheeler 2007-03-08 <20070308191120.GO2924@c-76-18-79-168.hsd1.nm.comcast.net>
> The patch you sent works around this, but you really ought to contact 
> the authors of the IMAP server and tell them to fix their broken 
> server. They shouldn't claim to support capabilities that they do not 
> actually support, no matter whether you're authenticated or not.

I had feared something like that... sorry for the noise then.

Still, other MUAs probably gracefully accept that broken behavior,
maybe mutt should as well?

Christoph
-- 
cb@df7cb.de | http://www.df7cb.de/

[brendan]

On Thursday, March  8 at 07:25 PM, quoth Christoph Berg:
 On Thursday, 08 March 2007 at 12:11, Kyle Wheeler wrote:
 Re: Kyle Wheeler 2007-03-08 <20070308191120.GO2924@c-76-18-79-168.hsd1.nm.comcast.net>
 On Thursday, March  8 at 01:17 PM, quoth David Champion:
 * On 2007.03.08, in <20070308191120.GO2924@c-76-18-79-168.hsd1.nm.comcast.net>,
 * On 2007.03.08, in <20070308194240.GP2924@c-76-18-79-168.hsd1.nm.comcast.net>,
 On Thursday, March  8 at 08:35 PM, quoth Christoph Berg:
 * On 2007.03.08, in <E1HPPx1-0000ND-Sx@trithemius.gnupg.org>,
 On Thursday, March  8 at 02:53 PM, quoth David Champion:
I've applied a variant of the proposed patch that queues the
command to save a round trip.