Include references header in emails for improved threading

  • Idea
  • Updated 4 weeks ago
  • Under Consideration
It would be great if the email notifications sent by GS were properly threaded in clients.

See: http://cr.yp.to/immhf/thread.html

I'm using Apple's Mail.app and it doesn't detect that all of the emails for a particular GS topic are “threadable”. We have a high traffic instance and receive a lot of email notifications, this would make our lifes easier.

Thanks.

Luke Daley

  • 9 Posts
  • 0 Likes

Posted 2 years ago

  • 4

Caty, Employee

  • 946 Posts
  • 104 Likes
This is a great idea, thanks for writing in! I'm going to share it with our product team and we'll update the status once they've had a chance to review.

Steven Pal, Director, Consumer Products

  • 328 Posts
  • 50 Likes
@Luke - Appreciate you sharing this idea. Do you know what mail clients adhere to the standards referenced in that link you shared?

Kevin Cox

  • 2 Posts
  • 0 Likes
Email threading is defined in RFC 882 https://www.ietf.org/rfc/rfc0822.txt and later refined in http://www.ietf.org/rfc/rfc2822.txt.

Basically check the references and in-reply-to headers. jwz has a good explanation here http://www.jwz.org/doc/threading.html.

Essentially you would just need to add these headers to your emails which could use a message ID based off of some sort of question ID.  Shouldn't be more that a couple of lines of code.

As for client support most.  Microsoft does their own thing as usual but Thunderbird, KMail, evolution, all the command line ones, probably the Apple client and so many more use this standard method.  The gmail web interface likely uses this as well although they appear to heavily reply on subject line.

Steven Pal, Director, Consumer Products

  • 328 Posts
  • 50 Likes
Thanks for the info Kevin!  We were actually talking about this today, so this is quite timely.  I'm really interested to find out if Gmail respects these headers or not.  I know you can search for the message-id that's passed in the headers (more here), but not sure if Gmail uses that to influence threading or not.

Kevin Cox

  • 2 Posts
  • 0 Likes
From what they say it appears that they don't follow them very closely.  For example I believe that if you reply to an email (with the proper headers) but change the subject it still breaks the conversation.  I don't know if they use them as hints but I don't think that would be a big deal because as you say, the subject remains constant anyway.

Also, for inspiration Facebook does this, although they set the References and In-Reply-To header to a single message which doesn't exist (it is actually composed of the group ID and post ID, so it is easy to generate and they don't have to store what they are using for each post).  It appears that most mail clients don't have an issue with this though (I tried a couple) as it is not uncommon to miss the start of a thread.  Ideally they would set to header to that of the original post notification (which the recipient may or may not have).

Source of original notification:
Message-ID: <23cd228579250323ec0c7eb48ce1d0de@async.facebook.com>
In-Reply-To: <10152315854823554-2202808553@groups.facebook.com>
References: <10152315854823554-2202808553@groups.facebook.com>
Source of replies (except for a different Message-ID they all have the same headers).
Message-ID: <940fd6babf273969c952b2b9a1180262@www.facebook.com>
In-Reply-To: <10152315854823554-2202808553@groups.facebook.com> References: <10152315854823554-2202808553@groups.facebook.com>
So while it would be nice to have the original message ID be the one that is referenced by the replies this appears to work pretty well.  Also, since GS supports threaded comments it would be nice to put the replied to comment in the "In-Reply-To" header and append it to the "References" header (you don't need every message ID in "References" but first and last couple are generally nice).

Facebook also keeps the subject line the same except for adding a "Re: " to replies.  This should make subject-dependant threading work as well.

Steven Pal, Director, Consumer Products

  • 328 Posts
  • 50 Likes
Appreciate you doing that investigation Kevin!  Good idea to look at how Facebook does this.  Generating those different id's would be relatively straightforward I imagine as we have unique id's for all of these objects (conversations, replies, comments).  Let me loop in someone else from our team to get their thoughts.

Luke Daley

  • 9 Posts
  • 0 Likes
I can't find a good definitive reference on what clients support this.

Some googling seemed to indicate that everything uses this for threading, except Outlook which uses its own mechanism.

Steven Pal, Director, Consumer Products

  • 328 Posts
  • 50 Likes
@Luke - Thanks for doing some investigation on this. Gmail email threading seems pretty straightforward -- found this StackExchange thread -- How does Gmail decide to thread email messages?. I think given that our email subject lines are the same for replies on a topic (plus they are from the same sender) it basically already works in Gmail. But sounds like other email clients don't automatically exhibit this behavior.

Out of curiosity, does this completely not work in Mac's mail client? Or it partially works? And if it's partially working, when does it not work?