Frequently merging mailing list

[wgseligman]

I'm in process of converting our mailing-list management from mailman 2 to sympa on a Linux system.

A key aspect of our mailman management is to merge mailing lists frequently, once every 15 minutes. To keep things simple, assume we have list-a and list-b. We want to merge list-a and list-b to list-c, via a shell script, running periodically.

In mailman, this can be done with (just a sketch):

list_members list-a > /tmp/list-a
list_members list-b > /tmp/list-b
sort -u /tmp/list-a /tmp/list-b > /tmp/list-c
sync_members --file /tmp/list-c list-c

How can one do this in sympa? Commands like sympa review or sympa dump can export lists of users. But the only ways I can find to update the contents of list-c is one of:

• close list-c, do some replacement of its dump files, then open it again with a revised dump file
• use sympa review on list-c, send that list to sympa del, and use sympa add with /tmp/list-c as input.

The difficulties with the first approach is that it's not clear how to merge dump files, and this appears to take list-c "down" for longer than mailman's sync_members command did.

When I tested the second approach, everyone in list-c received an unsubscribe email, then a resubscribe email. I don't see an option in sympa to turn off these emails (this is an option in mailman).

Have I missed something?

[racke]

Please try "include_sympa_list" (https://www.sympa.community/manual/customize/data-sources.html). The default frequency is 1 hour and can be change through the ttl parameter.

[wgseligman]

I may need some help here, since I didn't find many examples of this via web searches. I'm using the Linux RPMs distributed via EPEL; for me, $SYSCONFDIR is /etc/sympa and $EXPLDIR is /var/lib/sympa/list_data/${domain}. To continue my above example (list-a + list-b = list-c), does this look correct?

In /etc/sympa/data_sources/include_list_members.incl:

include_sympa_list
name include_list_members
listname [% param.0 %]

Edit /var/lib/sympa/list_data/domain.example.com/list-c/config and add at the bottom:

member_include
source include_list_members
source_parameters list-a

member_include
source include_list_members
source_parameters list-b

In which file do I define the ttl parameter? Are its units in seconds?

[wgseligman]

I think I have the answer to my own questions:

• I can define these merged lists solely through the web interface. It's a bit tricky to find. In case someone else has the same difficulty:

  • Click on a name of a list;
  • Click on List Home in the left-hand panel (by default Admin will be selected);
  • Data Sources Configuration is listed on the bottom of the page.
  • If you've created a .incl file, you can select it from the the data source (source) menu.
  • But that's not necessary. If you keep scrolling down the page, you'll see fields for all the different available data sources. In my case, that was List inclusion (include_sympa_list).
  • Define the data sources one at a time, scrolling to the bottom of the page and clicking Apply modifications each time. In my example, I defined the inclusion of list-a, clicked that button, then saw the page now had an additional set of fields in which I could add list-b.

• The ttl option can be defined in $SYSCONFDIR/sympa.conf. Since I wanted all my merged lists to be updated once every 15 minutes (instead of the default of 3600 seconds), I put:

ttl 900

Thanks, racke. Without your help I would not have found the answer.

[racke]

You are welcome 😁 And I think that is a more mature method for the inclusion compared to your solution with mailman.