Last modified: Thursday, July 25, 2024

GNU Mailman3 Installation Memo (FreeBSD)

Mailman3, FreeBSD, sendmail, apache, postorius

FreeBSD Daemon is buried under a lot of mail. This image is generated by DALL-E.

Image generated by DALL-E

Introduction

I have been using GNU Mailman2 to manage a small mailing list for a very narrow group of people, but I reluctantly migrated to mailman3 because both python2 and mailman2 are EoL. I thought I would be able to use it without much trouble since it is a well-known mailing list server, but it was very very troublesome, so I'll leave it as a memo before I forget.

To avoid misunderstanding, I think it can probably be installed without much trouble in a Linux+Postfix environment. You can often find reference pages on Google, and I think it is possible to install easily using packages. However, in the FreeBSD environment, the number of reference materials was drastically reduced, and in addition, when it comes to sendmail, I could hardly find any. I thought about giving up and switching to sympa, but it seemed like postfix was a prerequisite for sympa too (in ports), so I decided against it.

Installed Environments

Installation and Initial Configuration of mailman3

The required packages (ports) are assumed to be already installed.

Of the several search hits, the following two pages were ultimately the most helpful for me.

I set it up as a kind of merging of the contents of both.

Added the following to the MAILER_DEFINITIONS setting in the mc file used. 


Mmm3lmtp,       P=[IPC], F=PSXmnz9, S=EnvFromSMTP/HdrFromSMTP,
                R=EnvToMM3, E=\r\n, L=1024,
                A=TCP $h 8024

The following was also added to the LOCAL_RULESETS setting in the mc file. (Use tabs to separate R$+ and $: on the second line and $* and $: on the third line) 


SEnvToMM3
R$+     $: $>EnvToSMTP $1
R$+ < @ example . com . private . > $*       $: $1 < @ example . com . > $2

The following was added to /etc/mail/mailertable 


example.com.private        mm3lmtp:[localhost]

Created /usr/local/lib/python3.9/site-packages/mailman/mta/sendmail.py with the following contents. 


# Copyright (C) 2001-2022 by the Free Software Foundation, Inc.
#
# This file is part of GNU Mailman.
#
# GNU Mailman is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation, either version 3 of the License, or (at your option)
# any later version.
#
# GNU Mailman is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
# more details.
#
# You should have received a copy of the GNU General Public License along with
# GNU Mailman.  If not, see .

"""Creation/deletion hooks for the Sendmail MTA."""

import os

from collections import defaultdict
from contextlib import contextmanager
from flufl.lock import Lock
from mailman.config import config
from mailman.config.config import external_configuration
from mailman.interfaces.domain import IDomainManager
from mailman.interfaces.listmanager import IListManager
from mailman.interfaces.mta import (
    IMailTransportAgentAliases, IMailTransportAgentLifecycle)
from mailman.utilities.datetime import now
from operator import attrgetter
from public import public
from zope.component import getUtility
from zope.interface import implementer


ALIASTMPL = '{0}	%1%3@{2}.private'



@contextmanager
def atomic(path):
    # Write a new file and then atomically rename it.
    new_path = path + '.new'
    try:
        with open(new_path, 'w', encoding='utf-8') as fp:
            yield fp
    except:                                      # noqa: E722 pragma: nocover
        os.remove(new_path)
        raise
    else:
        os.rename(new_path, path)
        os.chmod(path, 0o664)


def _get_alias_domain(domain):
    domain_manager = getUtility(IDomainManager)
    d = domain_manager.get(domain)
    if d is not None and d.alias_domain:
        return d.alias_domain
    return domain


class _FakeList:
    """Duck-typed list for the `IMailTransportAgentAliases` interface."""

    def __init__(self, list_name, mail_host):
        self.list_name = list_name
        self.true_mail_host = mail_host
        self.mail_host = _get_alias_domain(mail_host)
        self.posting_address = '{}@{}'.format(list_name, self.mail_host)


@public
@implementer(IMailTransportAgentLifecycle)
class LMTP:
    """Connect Mailman to Sendmail via LMTP."""

    def __init__(self):
        # Locate and read the Postfix specific configuration file.
        mta_config = external_configuration(config.mta.configuration)

    def create(self, mlist):
        """See `IMailTransportAgentLifecycle`."""
        # We can ignore the mlist argument because for LMTP delivery, we just
        # generate the entire file every time.
        self.regenerate()

    delete = create

    def regenerate(self, directory=None):
        """See `IMailTransportAgentLifecycle`."""
        # Acquire a lock file to prevent other processes from racing us here.
        if directory is None:
            directory = config.DATA_DIR
        lock_file = os.path.join(config.LOCK_DIR, 'mta')
        with Lock(lock_file):
            lmtp_path = os.path.join(directory, 'sendmail_virtusertable')
            with atomic(lmtp_path) as fp:
                self._generate_lmtp_file(fp)

    def _generate_lmtp_file(self, fp):
        # The format for Postfix's LMTP transport map is defined here:
        # http://www.postfix.org/transport.5.html
        #
        # Sort all existing mailing list names first by domain, then by
        # local part.  For Postfix we need a dummy entry for the domain.
        list_manager = getUtility(IListManager)
        utility = getUtility(IMailTransportAgentAliases)
        by_domain = {}
        sort_key = attrgetter('list_name')
        for list_name, mail_host in list_manager.name_components:
            mlist = _FakeList(list_name, mail_host)
            by_domain.setdefault(mlist.mail_host, []).append(mlist)
        print("""\
# AUTOMATICALLY GENERATED BY MAILMAN ON {}
#
# This file is generated by Mailman.  YOU SHOULD NOT MANUALLY EDIT THIS
# FILE unless you know what you're doing, and can keep the two files properly
# in sync.  If you screw it up, you're on your own.
""".format(now().replace(microsecond=0)), file=fp)
        for domain in sorted(by_domain):
            print("""\
# Aliases which are visible only in the @{} domain.""".format(domain),
                  file=fp)
            for mlist in sorted(by_domain[domain], key=sort_key):
                aliases = list(utility.aliases(mlist))
                # width = max(len(alias) for alias in aliases) + \
                #    aliases[0].count('.') + 10
                print(ALIASTMPL.format(aliases.pop(0),
                                       config, domain), file=fp)
                for alias in aliases:
                    print(ALIASTMPL.format(alias, config, domain), file=fp)
                print(file=fp)
            #print(f"@{domain}	error:5.1.1:550 User unknown", file=fp)

The following file was created as /usr/local/lib/python3.9/site-packages/mailman/config/sendmail.cfg 


[sendmail]
# PLB started with exim4.cfg
# Additional configuration variables for sendmail

# sendmail doesn't need any additional configuration yet.

Added the following contents to /usr/local/mailman/etc/mailman.cfg 


[mta]
incoming: mailman.mta.sendmail.LMTP
outgoing: mailman.mta.deliver.deliver
lmtp_host: 127.0.0.1
lmtp_port: 8024
smtp_host: localhost
smtp_port: 25
configuration: python:mailman.config.sendmail

This will create a file as /usr/local/mailman/data/sendmail_virtusertable with the following contents. Refer to this file as /etc/mail/virtusertable 


mlname@example.com	%1%3@example.com.private
mlname-bounces@example.com	%1%3@example.com.private
mlname-confirm@example.com	%1%3@example.com.private
mlname-join@example.com	%1%3@example.com.private
mlname-leave@example.com	%1%3@example.com.private
mlname-owner@example.com	%1%3@example.com.private
mlname-request@example.com	%1%3@example.com.private
mlname-subscribe@example.com	%1%3@example.com.private
mlname-unsubscribe@example.com	%1%3@example.com.private

The above is the general outline of the sendmail support. There will probably be many errors outside of what is written here, but we will fix them according to the error messages (sorry, but there are too many to remember).

Installation and Initial Configuration of Postorius

The work here was more of a nightmare for me. The following information was most helpful to me.

Basically, you just follow this procedure (rewriting it as needed to suit your site), but even here there are many errors with missing modules, etc., so follow the instructions given in the message. The language and time zone settings may need to be modified.

Now it's time to load the apache configuration file. LoadModule mod_wsgi module and place the following settings under /usr/local/etc/apache24/Includes/. 


Alias /static /usr/local/mailman/static
<Directory "/usr/local/mailman/static/">
    Require ip Appropriate IP address range
</Directory>

WSGIScriptAlias /mailman3 /usr/local/mailman/wsgi.py process-group=mailman-web
WSGIScriptAlias /postorius /usr/local/mailman/wsgi.py process-group=mailman-web

WSGIPythonHome /usr/local
WSGIPythonPath /usr/local/mailman
WSGIDaemonProcess mailman-web display-name=mailman-web maximum-requests=1000 umask=0002 user=mailman group=mailman python-path=/usr/local/mailman home=/usr/local

<Directory "/usr/local/mailman">
	<Files wsgi.py>
		Require ip Appropriate IP address range
	</Files>
	WSGIProcessGroup mailman-web
</Directory>

Each time an error occurs, I modify the configuration file, but sometimes the changes do not take effect until I cpmpletely restart services such as uwsgi, apache, mailman, etc. (i.e., restart is not sufficient, stop & start is required).

Conclusion

This is the information for running Mailman3+Postorius in a FreeBSD+Sendmail+Apache environment. I'm not confident I can install it the same way again next time. Anyway, there are too many package and module dependencies, and it is clearly over-specified even though I am not asking for much. A long time ago majordomo was very easy to understand, how did mailman3 become like this? I may switch to mlmmj next.

Update (July 12, 2024)

The default version of python provided by FreeBSD ports went up from 3.9 to 3.11, so I followed the usual procedure to upgrade it to 3.11, and as expected, Postorius was no longer available. Without thinking, I tried pip install django-q while looking at the apache error log, and also tried /usr/local/lib/python3.9/site-packages/mailman/config/sendmail.cfg and /usr/local/lib/python3.9/site-packages/mailman/mta/sendmail.py to an equivalent location under /usr/local/lib/python3.11/ respectively, or /usr/local/etc/rc.d/apache24 and /usr/local/etc/rc.d/uwsgi restarting, or stop & start, and finally got back to being able to log in to Postorius.

Atsushi NUNOME (nunome@kit.ac.jp)