Exim: Failed to get envelope sender in "Return-path:" field #1354

dpc22 · 2022-03-02T15:21:19Z

Version

6.2.66

Installation method

My own rpm, derived from the "official" RHEL 7 rpms

Expected behavior

If someone sends a message to a moderated list, they receive a notification message as you would expect:

Your message to the list 'XXX' has been forwarded to the moderator(s)

If they have also set up an out of office message on the mailbox which receives this notification then (at least when using Exchange Online) this out of office message is send to sympa@domain with an empty SMTP return path (the "<>" below):

2022-02-24 10:09:06 +0000 1nNB3a-0008az-Hh
        <= <>
        H=ppsw-30.csi.cam.ac.uk [131.111.8.130]:50470
        I=[131.111.8.15]:25 P=esmtp S=12985 
        for [email protected]

I would not expect this to generate any further messages given the potential for confusion and mail loops.

Actual behavior

Sympa appears to fish a sending email address from the out of office message and complains:

This is an automatic response sent by Sympa Mailing Lists Manager.
Impossible to distribute your message for list [email protected] for the
following reason:

No command found in message.

I think that Sympa should test for the empty return path here before it attempts to process the message as a list of commands.

Feb 24 10:09:08 lists-1 sympa_msg[32474]: notice Sympa::Spindle::ProcessIncoming::_twist() Processing Sympa::Message [<[email protected]>](mailto:[email protected]); envelope_sender=; [[email protected]](mailto:[email protected]); [[email protected]](mailto:[email protected])
Feb 24 10:09:08 lists-1 sympa_msg[32474]: notice Sympa::Request::Message::__parse() Parsing: Automatic reply: Message distribution
Feb 24 10:09:08 lists-1 sympa_msg[32474]: notice Sympa::Request::Message::__parse() Parsing: <...the text of the out of office message...>
Feb 24 10:09:08 lists-1 sympa_msg[32474]: info Sympa::Spindle::DoCommand::_twist() No command found in message Sympa::Message [<[email protected]/shelved:arc_cv=pass>](mailto:[email protected]/shelved:arc_cv=pass)

The text was updated successfully, but these errors were encountered:

ikedas · 2022-03-02T21:58:00Z

Hi @dpc22 ,

Could you please show us the content (at least the subject and the body) of that " out of office message" ?

dpc22 · 2022-03-03T06:40:46Z

I don't have a copy of the subject line. The out of office was generated by Exchange Online, so it will probably be of the form:

Automatic reply: $original_subject

so:

Automatic reply: Message distribution

The message content is recorded in the Sympa logs:

I am working reduced hours whilst I recover from being unwell.
Therefore there may be a delay in responding to emails. Any urgent
Fitzwilliam College IT and AV related problems or requests should be
sent to [email protected]) or filed as a helpdesk ticket at
https://help.fitz.cam.ac.uk/

ikedas · 2022-04-15T22:32:37Z

As a workaround, you may disable sympa@mydomain, i.e. removing this address from the setting of MTA.

dpc22 · 2022-04-16T07:43:29Z

Don't people have to email sympa@mydomain in order to send email commands to the list? For example:

From: David Carter [email protected]
To: SYMPA [email protected]
Subject: help

responses:

             SYMPA -- Systeme de Multi-Postage Automatique
                       (Automatic Mailing System)

                                User's Guide


SYMPA is an electronic mailing-list manager that automates list management
functions such as subscriptions, moderation, and archive management.
[...]

ikedas · 2022-04-16T07:55:24Z

Don't people have to email sympa@mydomain in order to send email commands to the list? For example:

Exactly. The definition for this email address is included in setting of MTA as an alias, and may be removed if none uses mail commands.

ikedas · 2022-10-30T08:11:09Z

Hi @dpc22 ,

It turned out that, as of Sympa 6.2.13 or later, the response "No command found in message." was sent as DSN by Sympa (cf. the commit). Sympa won't send DSN as a response to the incoming message with null envelope sender.

Additionally, Sympa won't process the message with null envelope sender: Instead it will be ignored with this log:

Oct 30 16:09:50 hostname sympa_msg[2017]: notice Sympa::Spindle::DoCommand::_twist() Sympa::Message <[email protected]>: Ignoring message which would cause a loop; message appears to be DSN report

Thus, there seem no problem.

ikedas · 2022-10-30T09:20:58Z

Anyways it is worth sending the first response "Your message to the list 'XXX' has been forwarded to the moderator(s)" as an DSN: Legitimate autoresponders won't respond to DSNs. Please check the PR above.

dpc22 · 2022-10-31T08:12:14Z

I'm puzzled by:

Sympa won't send DSN as a response to the incoming message with null envelope sender

as Sympa 6.2.66 was definitely generating these "No command found in message" responses to out of office messages with null envelope sender earlier in the year.

Does (#1508) change the behaviour in some way? It isn't immediately clear to me what that does. Thanks.

ikedas · 2022-10-31T08:24:44Z

I'm puzzled by:

Sympa won't send DSN as a response to the incoming message with null envelope sender

as Sympa 6.2.66 was definitely generating these "No command found in message" responses to out of office messages with null envelope sender earlier in the year.

Sympa 6.2.66 rejects such messages:

sympa/src/lib/Sympa/Spindle/DoCommand.pm

Lines 72 to 81 in b17e3bb

    
           # Preventing loops based on Sympa processes sending DSN to each other. 
        
           if ($message->{envelope_sender} and $message->{envelope_sender} eq '<>' 
        
               or grep {/multipart\/report/i} $message->get_header('Content-type')) { 
        
               $log->syslog( 
        
                   'notice', 
        
                   '%s: Ignoring message which would cause a loop; message appears to be DSN report', 
        
                   $message 
        
               ); 
        
               return undef; 
        
           }

What is possible is that that out of office message did not have the null envelope sender.

Does (#1508) change the behaviour in some way? It isn't immediately clear to me what that does. Thanks.

No. This PR changes the first message described in above to the DSN. Thus auto-responder will not respond to the messages, i.e. out of office message will not be sent.

dpc22 · 2022-10-31T08:37:26Z

The log entry that I quoted when I opened the ticket indicated a null envelope sender (the <= <> bit below):

2022-02-24 10:09:06 +0000 1nNB3a-0008az-Hh
        <= <>
        H=ppsw-30.csi.cam.ac.uk [131.111.8.130]:50470
        I=[131.111.8.15]:25 P=esmtp S=12985 
        for [email protected]

I guess that it is possible that something else is going on here. I will do some testing.

dpc22 · 2022-10-31T08:51:25Z

The following SMTP dialogue generates a "No command found in message" response to [email protected] with Sympa 6.2.68:

$ telnet mx.cam.ac.uk smtp
Trying 131.111.8.147...
Connected to mx.CAM.AC.UK.
Escape character is '^]'.
220 ppsw-31.srv.uis.cam.ac.uk (mx.cam.ac.uk [131.111.8.147]:25) ESMTP Exim 4.96-0 Mon, 31 Oct 2022 08:46:23 +0000

HELO me
250 ppsw-31.srv.uis.cam.ac.uk Hello user-5.uis.private.cam.ac.uk [172.16.56.5]

MAIL FROM:<>
250 OK

RCPT TO:<[email protected]>
250 Accepted

DATA
354 Enter message, ending with "." on a line by itself
From: [email protected]
To: [email protected]
Subject: Test
Message-ID: 12345@localhost

Test

.

You should be able to replicate that. I don't think that I have made any local changes which would affect this behaviour.

ikedas · 2022-10-31T08:55:56Z

Please show the log of Sympa from when the maili server accepted the message to when Sympa responded (or didn't respond) to that message.

dpc22 · 2022-10-31T09:01:00Z

2022-10-31 08:47:15 +0000 1opQRh-0006XN-0v
        <= <>
        H=ppsw-31.srv.uis.cam.ac.uk [131.111.8.131]:48150
        I=[131.111.8.15]:25 P=esmtp S=1481
        id=12345@localhost
        for [email protected]
2022-10-31 08:47:16 +0000 1opQRh-0006XN-0v
        => sympa <[email protected]>
        F=<> P=<>
        R=sympa_command T=sympa_command S=1526 QT=1s DT=1s

Oct 31 08:47:18 lists-1 sympa_msg[2923]: notice Sympa::Spindle::ProcessIncoming::_twist() Processing Sympa::Message <[email protected]>; envelope_sender=; message_id=12345@localhost; [email protected]
Oct 31 08:47:18 lists-1 sympa_msg[2923]: notice Sympa::Request::Message::__parse() Parsing: Test
Oct 31 08:47:18 lists-1 sympa_msg[2923]: notice Sympa::Request::Message::__parse() Parsing: Test
Oct 31 08:47:18 lists-1 sympa_msg[2923]: info Sympa::Spindle::DoCommand::_twist() No command found in message Sympa::Message <[email protected]/shelved:arc_cv=none>
Oct 31 08:47:18 lists-1 sympa_msg[2923]: notice Sympa::Spindle::ProcessTemplate::_twist() Processing Sympa::Message::Template <[email protected],359/shelved:dkim_sign>; envelope_sender=<>; [email protected]; [email protected]; [email protected]; template=delivery_status_notification; action=failed; status=5.6.1
Oct 31 08:47:18 lists-1 sympa_msg[2923]: notice Sympa::Spool::Outgoing::store() Message Sympa::Message::Template <[email protected],359/shelved:dkim_sign> is stored into bulk spool as <[email protected]_s,2923,8065>
Oct 31 08:47:18 lists-1 bulk[7545]: notice Sympa::Spindle::ProcessOutgoing::_twist() Start sending message Sympa::Message <[email protected]_s,2923,8065/s/shelved:dkim_sign> to lists.cam.ac.uk (priority 1) (starting 0 seconds after scheduled expedition date)

2022-10-31 08:47:19 +0000 1opQRy-0006Xc-31
        <= <> U=sympa P=local S=3724
        [email protected]
        for [email protected]
2022-10-31 08:47:19 +0000 1opQRy-0006Xc-31
        => [email protected]
        F=<> P=<>
        R=remote T=smtp S=3823
        H=ppsw.cam.ac.uk [131.111.8.137]
        I=[131.111.8.13]
        C="250 OK
        id=1opQRz-0007Ad-X3" QT=0s DT=0s```

ikedas · 2022-10-31T09:06:55Z

Oct 31 08:47:18 lists-1 sympa_msg[2923]: notice Sympa::Spindle::ProcessIncoming::_twist() Processing Sympa::Message <[email protected]>; envelope_sender=; message_id=12345@localhost; sender=[email protected]

Envelope sender isn't passed from Postfix to Sympa. Expected log line is such as:

Oct 31 17:59:47 hostname sympa_msg[2017]: notice Sympa::Spindle::ProcessIncoming::_twist() Processing Sympa::Message <[email protected]>; envelope_sender=<>; message_id=12345@localhost; sender=[email protected]

You probably set up Postfix with "virtual domain setting" described in the manual. If you did, review the setting in master.cf

dpc22 · 2022-10-31T09:09:33Z

I am running Exim.

Wouldn't "envelope_sender=" indicate an empty envelope sender address? "<>" is just quoting around the actual value.

That certainly appears to be true of other email addresses which are quoted in the Sympa logs. There are never any "<" or ">" after the "envelope_sender=" clauses in my logs. Example: envelope_sender=[email protected]

ikedas · 2022-10-31T09:13:48Z

Oh sorry.

On Exim, pipe transport(s) for Sympa should have return_path_add. Check the manual about Exim 4.

dpc22 · 2022-10-31T09:17:01Z

Aha. If it is looking for a "Return-Path: " header in the message body that would explain things.

dpc22 · 2022-10-31T12:26:26Z

Okay, I added return_path_add to my Exim configuration. That didn't fix the problem, although I agree that it should.

After a lot of head scratching, I think that this is just because the header that Exim adds to the message is "Return-path" rather than "Return-Path" (note the case of "path"). The Exim source contains:

src/transport.c:    uschar * s = string_sprintf("Return-path: <%.*s>\n%n",

If I log "$serialized" in Sympa/Message.pm I see:

X-Sympa-To: [email protected]
Return-path: <>
Received: from [127.0.0.1] (port=57784 helo=me)
        by lists-2.srv.uis.cam.ac.uk (lists.cam.ac.uk [127.0.0.1]:25)
        with smtp id 1opTQc-0000n4-34 (Exim 4.96) for [email protected]
        (return-path <>);
        Mon, 31 Oct 2022 11:58:18 +0000
From: [email protected]
To: [email protected]
Subject: Test
Message-ID: 3@localhost

Test

That doesn't match the following in Sympa/Message.pm:

   if ($serialized =~ /\GReturn-Path: (.*?)\n(?![ \t])/cgs
        and not exists $self->{'envelope_sender'}) {

I confess that I don't know what "\G" does in Perl regular expressions, but "Return-Path" rather than "Return-path" seems to be significant. Presumably we can use just add an "i" modifier to the end of "/cgs"

…mmunity#1354)

ikedas · 2022-11-01T00:53:47Z

@dpc22, thank you for investigation. The PR above will fix the problem.

Exim: Failed to get envelope sender in "Return-path:" field (#1354)

ikedas added the bug label Mar 3, 2022

ikedas self-assigned this Mar 3, 2022

racke added this to the 6.2.70 milestone Jun 8, 2022

ikedas modified the milestones: 6.2.70, 6.2.72 Jun 10, 2022

ikedas mentioned this issue Oct 30, 2022

Use DSN for the message to notify moderation #1508

Merged

ikedas added a commit to ikedas/sympa that referenced this issue Nov 1, 2022

Exim: Failed to get envelope sender in "Return-path:" field (sympa-co…

e2ea4f7

…mmunity#1354)

ikedas mentioned this issue Nov 1, 2022

Exim: Failed to get envelope sender in "Return-path:" field (#1354) #1509

Merged

ikedas added the ready A PR is waiting to be merged. Close to be solved label Nov 1, 2022

ikedas closed this as completed in #1509 Nov 16, 2022

ikedas added a commit that referenced this issue Nov 16, 2022

Merge pull request #1509 from ikedas/issue-1354bis by ikedas

e924125

Exim: Failed to get envelope sender in "Return-path:" field (#1354)

ikedas changed the title ~~Sympa attempts to process out of office auto-responses sent to "sympa@mydomain" as commands~~ Exim: Failed to get envelope sender in "Return-path:" field Nov 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exim: Failed to get envelope sender in "Return-path:" field #1354

Exim: Failed to get envelope sender in "Return-path:" field #1354

dpc22 commented Mar 2, 2022

ikedas commented Mar 2, 2022

dpc22 commented Mar 3, 2022

ikedas commented Apr 15, 2022

dpc22 commented Apr 16, 2022

ikedas commented Apr 16, 2022 •

edited

Loading

ikedas commented Oct 30, 2022 •

edited

Loading

ikedas commented Oct 30, 2022

dpc22 commented Oct 31, 2022

ikedas commented Oct 31, 2022 •

edited

Loading

dpc22 commented Oct 31, 2022

dpc22 commented Oct 31, 2022

ikedas commented Oct 31, 2022

dpc22 commented Oct 31, 2022 •

edited

Loading

ikedas commented Oct 31, 2022 •

edited

Loading

dpc22 commented Oct 31, 2022 •

edited

Loading

ikedas commented Oct 31, 2022

dpc22 commented Oct 31, 2022 •

edited

Loading

dpc22 commented Oct 31, 2022

ikedas commented Nov 1, 2022

Exim: Failed to get envelope sender in "Return-path:" field #1354

Exim: Failed to get envelope sender in "Return-path:" field #1354

Comments

dpc22 commented Mar 2, 2022

Version

Installation method

Expected behavior

Actual behavior

ikedas commented Mar 2, 2022

dpc22 commented Mar 3, 2022

ikedas commented Apr 15, 2022

dpc22 commented Apr 16, 2022

ikedas commented Apr 16, 2022 • edited Loading

ikedas commented Oct 30, 2022 • edited Loading

ikedas commented Oct 30, 2022

dpc22 commented Oct 31, 2022

ikedas commented Oct 31, 2022 • edited Loading

dpc22 commented Oct 31, 2022

dpc22 commented Oct 31, 2022

ikedas commented Oct 31, 2022

dpc22 commented Oct 31, 2022 • edited Loading

ikedas commented Oct 31, 2022 • edited Loading

dpc22 commented Oct 31, 2022 • edited Loading

ikedas commented Oct 31, 2022

dpc22 commented Oct 31, 2022 • edited Loading

dpc22 commented Oct 31, 2022

ikedas commented Nov 1, 2022

ikedas commented Apr 16, 2022 •

edited

Loading

ikedas commented Oct 30, 2022 •

edited

Loading

ikedas commented Oct 31, 2022 •

edited

Loading

dpc22 commented Oct 31, 2022 •

edited

Loading

ikedas commented Oct 31, 2022 •

edited

Loading

dpc22 commented Oct 31, 2022 •

edited

Loading

dpc22 commented Oct 31, 2022 •

edited

Loading