mutt quote/dequote headers too many times

My patch is attached.

- Uploading patch-1.5.11cvs.tamovl.namequot.1 (see #2163).

patch-1.5.11cvs.tamovl.namequot.1

mbox file showing missing quotes in the "To:" header in the pager

The problem with this patch is that it sometimes removes useful quotes, e.g. in

To: "foo <foo@domain.invalid>, bar" <bar@domain.invalid>

which is displayed as:

To: foo <foo@domain.invalid>, bar <bar@domain.invalid>

* Sat Jul  5 2008 Mutt <fleas@mutt.org>
> #2014: mutt quote/dequote headers too many times
> 
> Comment (by vinc17):
> 
>  The problem with this patch is that it sometimes removes useful quotes,
>  e.g. in
> 
>    To: "foo <foo@domain.invalid>, bar" <bar@domain.invalid>
> 
>  which is displayed as:
> 
>    To: foo <foo@domain.invalid>, bar <bar@domain.invalid>

Well, I don't think this is my fault. ;)

In short, this problem is caused by the fact that mutt uses
the same decoration (color, italic, bold, or anything) for
both display names and addresses.

(Your "foo <foo@domain.invalid>, bar" is a display name.
And <bar@domain.invalid> is an address. Most of GUI MUAs
such as gmail use bigger/bolder fonts for display names.)

If you prefer having double-quotes around _every_ display
name, you are against mutt's behavior rather than my patch.
(The following comment in rfc822.c was not written by me:
>>	/* if we get something like "Michael R. Elkins"
>>	   remove the quotes */
Hmm, whose is this? :)

If you just want to include double-quotes in your display
name, use a backslash before every double-quote. (See RFC
2822 and you'll find an example like "Giant; \"Big\" Box")



Anyway, I am not a heavy user of mutt these days.
So I've forgotten the detail of the patch.

Replying to TAKAHASHI Tamotsu:

In short, this problem is caused by the fact that mutt uses the same decoration (color, italic, bold, or anything) for both display names and addresses.

In fact it doesn't. Double quotes can be regarded as a decoration and Mutt uses them when necessary.

If you prefer having double-quotes around _every_ display name, you are against mutt's behavior rather than my patch.

No, only when need be, i.e. when there is some ambiguity (which Mutt does without your patch -- but it does other things incorrectly).

> Comment (by vinc17):
> 
>  Replying to [comment:4 TAKAHASHI Tamotsu]:
>  > In short, this problem is caused by the fact that mutt uses
>  > the same decoration (color, italic, bold, or anything) for
>  > both display names and addresses.
> 
>  In fact it doesn't. Double quotes can be regarded as a decoration and Mutt
>  uses them when necessary.
> 
>  > If you prefer having double-quotes around _every_ display
>  > name, you are against mutt's behavior rather than my patch.
> 
>  No, only when need be, i.e. when there is some ambiguity (which Mutt does
>  without your patch -- but it does other things incorrectly).

Oh, I removed the RFC822Specials case from rfc822_write_address_single().
Though I don't remember why I made such a modification,
you may want to revert the chunk of the patch.
(e.g. Delete lines 207-253 of patch-1.5.11cvs.tamovl.namequot.1)

Well, I hope it is not a core part of the patch.
But if you find a side-effect, you may need to check "display",
i.e.:
-    if (strpbrk (addr->personal, RFC822Specials))
+    if (display && strpbrk (addr->personal, RFC822Specials))

HTH,

Replying to TAKAHASHI Tamotsu:

Oh, I removed the RFC822Specials case from rfc822_write_address_single(). Though I don't remember why I made such a modification, you may want to revert the chunk of the patch. (e.g. Delete lines 207-253 of patch-1.5.11cvs.tamovl.namequot.1)

There's a problem with that: when one replies to a message for which quotes are needed, too many quotes are added. For instance, if I reply to a message with address

"foo <foo@domain.invalid>, bar" <bar@domain.invalid>

and postpone the message, I get in the postponed mailbox:

"\"\\\"\\\\\\\"foo <foo@domain.invalid>, bar\\\\\\\"\\\"\"" <bar@domain.invalid>

Well, I hope it is not a core part of the patch. But if you find a side-effect, you may need to check "display", i.e.: - if (strpbrk (addr->personal, RFC822Specials)) + if (display && strpbrk (addr->personal, RFC822Specials))

With this change, I still have the display problem I mentioned on 2008-07-05.

Okay, so this won't be fixed with an easy hack.
This seems a matter of design, i.e. 
(1) when (the timing) to de/quote rfc822 addresses and
(2) how to handle complicated rfc822 addresses.

Let's see what these problems are and what we can do.

======================
(1) Timing
Mutt accepts addresses from two sources:
key (to quote) and file (to dequote).

(1-1) From a key typing,
(a) an address can be quoted or unquoted.
(b) It is then processed by editor,
(c) and then by "compose" menu.
(d) Sometimes it is postponed.
(e) At last it is sent.

There are two points to quote the address.
(e) is not processed many times, but it's too late to see
how the address will be modified actually.
Somewhere between (a) and (d) is processed again and again.
Considering edit_hdr, (c) is the best point IMHO.

So we have to invent a way not to quote already-quoted
addresses. This will be discussed in (2).

(1-2) From a file or a remote mailbox,
(a) an address should be a valid rfc822 address,
  but it can be double-quoted or only backslash-quoted.
(b) It is parsed.
(c) Then given to main (index) mode.
(d) At last it is shown in a pager (builtin or external).

In this case, (d) seems the best point to dequote,
but I'm not sure. I haven't investigated in this area.
Anyway, we have to make sure this won't ruin the (1-1) course.

======================
(2) Quoting scheme
Currently mutt has no way to send a message with a complicated
address. So there's little compatibility problem.

One big problem is how to avoid over-processing.
How can we determine whether an address is quoted or not?
I suggest that we decide with the first character.
i.e. If the address begins with '"', it is already quoted.
Mutt should just pass it through.

So we can dequote an address at (1-2-b) and quote it again
to declare it has already been quoted.

======================
(3) Implementation
Unfortunately, (1-1) and (1-2) share a lot of functions.
So we will have to have a hackish flag or something.
And it may have some hard-to-find side-effects.

======================

I'm going to try the above idea, but it may take a while.
Don't hesitate to make your own patch, everyone.

Lastly, thanks for your testing, Vincent.