I wanted to document my process for setting up Mail-In-A-Box on AWS to potentially help future people with regards to their setup.
I utilized the GitHub aws-samples/aws-opensource-mailserver/tree/main?tab=readme-ov-file for the template file which I did need to modify to support dependencies discovered after troubleshooting.
First let me provide the role with trust profile utilized for the stack template setup. (This is good for bringing up and deleting the resources the stack has created)
Trust Policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "cloudformation.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Policy to assign to the role:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ssm:*",
"lambda:*",
"ec2:*",
"logs:*",
"ses:*"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"s3:CreateBucket",
"s3:PutBucketPolicy",
"s3:PutObject",
"s3:GetBucketLocation",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::${BackupS3Bucket}",
"arn:aws:s3:::${BackupS3Bucket}/*",
"arn:aws:s3:::${NextCloudS3Bucket}",
"arn:aws:s3:::${NextCloudS3Bucket}/*"
]
},
{
"Effect": "Allow",
"Action": [
"iam:CreateRole",
"iam:CreateInstanceProfile",
"iam:PutRolePolicy",
"iam:PassRole",
"iam:AttachRolePolicy",
"iam:AddRoleToInstanceProfile",
"iam:DeleteGroup",
"iam:GetRole",
"iam:DeleteRolePolicy",
"iam:GetGroup",
"iam:CreateGroup",
"iam:DeleteRole",
"iam:RemoveRoleFromInstanceProfile",
"iam:DeleteInstanceProfile",
"iam:GetInstanceProfile",
"iam:DeleteGroupPolicy",
"iam:RemoveUserFromGroup",
"iam:PutGroupPolicy",
"iam:GetUser",
"iam:CreateUser",
"iam:DeleteUser",
"iam:AddUserToGroup",
"iam:ListAccessKeys",
"iam:CreateAccessKey",
"iam:DeleteAccessKey",
"iam:TagRole",
"iam:GetRolePolicy"
],
"Resource": [
"arn:aws:iam::{Account_id}:role/MailInABoxInstanceRole",
"arn:aws:iam::{Account_id}:instance-profile/MailInABoxInstanceProfile",
"arn:aws:iam::{Account_id}:policy/*MailInABox*",
"arn:aws:iam::{Account_id}:group/SMTPUserGroup-{stack_name}",
"arn:aws:iam::{Account_id}:role/SMTPLambdaExecutionRole-{stack_name}",
"arn:aws:iam::{Account_id}:role/MailInABoxInstanceRole",
"arn:aws:iam::{Account_id}:instance-profile/MailInABoxInstanceProfile",
"arn:aws:iam::{Account_id}:user/SMTPUser-{stack_name}"
]
},
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::{Account_id}:role/MailInABoxInstanceRole"
}
]
}
replace {Account_id} with your account number and {stack_name} with the name of your mail in a box stack.
The modified template is at the bottom of the post
Changes made to the original template EC2 Execution Script troubleshooting process:
During Deployment I had to add logs to isolate an errors that came up:
bash -x setup/start.sh 2>&1 | tee /tmp/mailinabox_debug.log
after adding this I started addressing errors 1 at a time.
the first was a dialog error Notewhich later in the troubleshooting process since we run in non-interactive mode can probably be skipped but I still resolved it to move forward****
setup/functions.sh: line 168: dialog: command not found
To resolve this I needed to add the following to the start of the EC2 Section of the template:
# Install missing package (dialog) first
echo "Installing missing package: dialog..." | tee -a $LOGFILE | logger -t mailinabox_setup
apt-get install -y dialog
The next issue after resolving the dialog dependency was:
+ cd /opt/mailinabox/
+ setup/start.sh
cannot open tty-output
cannot open tty-output. After some research I found the following:
TTY (Teletypewriter) output refers to the text-based interface that a program sends to a terminal (or console). When you run a command in a terminal, the output you see on the screen is TTY output.
When is TTY Output Important?
- Some scripts behave differently when run interactively (in a terminal) vs. non-interactively (e.g., via
cronor a script). - If a command expects a TTY (interactive session) but runs in a background job or a script, it may fail or require a workaround.
The Logs from the error were following:
+ echo 'Running Mail-in-a-Box setup script...'
+ tee -a /var/log/mailinabox_setup.log
...
cannot open tty-output
++ result=
++ result_code=255
++ set -e
++ '[' -z '' ']'
++ exit
After some work with AI it led me to adding environment variables to ensure I had mail in a box options set and non interactive needed to be set to 1 instead of true to proceed without TTY
# Configure variables
echo "Configuring environment variables..." | tee -a $LOGFILE | logger -t mailinabox_setup
# export NONINTERACTIVE=true
export NONINTERACTIVE=1
# added to remove tty-output error.
export DEBIAN_FRONTEND=noninteractive
export TERM=xterm # Ensures dialog has a terminal interface
export SKIP_NETWORK_CHECKS=true
export STORAGE_ROOT=/home/user-data
export STORAGE_USER=user-data
export PRIVATE_IP=$(ec2metadata --local-ipv4)
export PUBLIC_IP=$(ec2metadata --public-ipv4)
export PRIMARY_HOSTNAME=${InstanceDns}.${MailInABoxDomain}
export DEFAULT_PRIMARY_HOSTNAME=${InstanceDns}.${MailInABoxDomain}
export DEFAULT_PUBLIC_IP=$(ec2metadata --public-ipv4)
# Setup Admin Account
echo "Setting up admin account..." | tee -a $LOGFILE | logger -t mailinabox_setup
export EMAIL_ADDR=admin@${MailInABoxDomain}
if [[ -z "${MailInABoxAdminPassword}" ]]; then
export EMAIL_PW=$(tr -dc A-Za-z0-9 </dev/urandom | head -c 16 ; echo '')
if [[ -z "${RestorePrefix}" ]]; then
aws ssm put-parameter \
--overwrite \
--name "/MailInABoxAdminPassword" \
--type SecureString \
--value "$EMAIL_PW"
fi
else
export EMAIL_PW=${MailInABoxAdminPassword}
fi
That issue was now resolved.
After another run we get to backup.py running into an error:
Template File Code being executed:
# Create Initial Backup
echo "Creating initial backup..." | tee -a $LOGFILE | logger -t mailinabox_setup
/opt/mailinabox/management/backup.py
Log File
+ echo 'Creating initial backup...'
Creating initial backup...
+ tee -a /var/log/mailinabox_setup.log
+ logger -t mailinabox_setup
+ /opt/mailinabox/management/backup.py
Traceback (most recent call last):
File "/usr/bin/duplicity", line 5, in <module>
from duplicity.__main__ import dup_run
ModuleNotFoundError: No module named 'duplicity.__main__'
Something is wrong with the backup:
In the template file we are installing:
pip3 install duplicity==1.0.1
Based on the forum: solved-the-latest-release-of-duplicity-is-missing-the-duplicity-binary/11439/2
It seems we need to install via snap the latest duplicity to ensure main is included.
Based on the discussion I changed the set of the script to be the following:
# Remove existing duplicity installation if any
echo "Removing existing duplicity installation..." | tee -a $LOGFILE | logger -t mailinabox_setup
apt-get remove -y duplicity || true
rm -rf /etc/apt/sources.list.d/duplicity-team-ubuntu-duplicity-release-git-jammy.list || true
apt-get update
# Install duplicity via Snap
echo "Installing duplicity via Snap..." | tee -a $LOGFILE | logger -t mailinabox_setup
snap install duplicity --classic
which duplicity
# Create a symlink to /usr/bin/duplicity
echo "Creating symlink for duplicity..." | tee -a $LOGFILE | logger -t mailinabox_setup
ln -sf /snap/bin/duplicity /usr/bin/duplicity
which duplicity
# Block duplicity from being installed via apt
echo "Blocking duplicity from being installed via apt..." | tee -a $LOGFILE | logger -t mailinabox_setup
echo -e "# Duplicity is installed via snap\nPackage: duplicity\nPin: release *\Pin-Priority: -1" > /etc/apt/preferences.d/duplicity
# Verify duplicity installation
echo "Verifying duplicity installation..." | tee -a $LOGFILE | logger -t mailinabox_setup
duplicity --version
I added this in the pre-configuration before start.sh executes which caused an error when duplicity was trying to be added later in the setup script when setup/system.sh executed:
...
Verifying duplicity installation...
+ duplicity --version
duplicity 3.0.3.2 November 25, 2024
When + source setup/system.sh executes we have a section whereby duplicity is trying to be added:
++ hide_output add-apt-repository -y ppa:duplicity-team/duplicity-release-git
+++ mktemp
++ OUTPUT=/tmp/tmp.qMA7FLWtsY
++ set +e
++ add-apt-repository -y ppa:duplicity-team/duplicity-release-git
Then when the system tries to execute apt get it returns:
FAILED: apt-get -y -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confnew upgrade
++ echo -----------------------------------------
I needed to remove duplicity and change the install once we finish the installation.
After moving the duplicity fix further down the script to after mail-in-a-box installs we were able to continue with the mail in a box successful installation:
there were some log segments during the next cloud setup I can investigate later but did not interrupt the installation of mail-in-a-box:
++ cd /usr/local/lib/owncloud
++ sudo -u www-data php8.0 /usr/local/lib/owncloud/index.php
sudo: unable to resolve host box.{removed-sensitive-info}: Name or service not known
...
++ sudo -u www-data php8.0 /usr/local/lib/owncloud/occ upgrade
sudo: unable to resolve host box.{removed-sensitive-info}: Name or service not known
Nextcloud is already latest version
++ '[' '(' 0 -ne 0 ')' -a '(' 0 -ne 3 ')' ']'
++ sudo -u www-data php8.0 /usr/local/lib/owncloud/occ app:disable photos dashboard activity
++ grep -v 'No such app enabled'
Afterwards we are now running Mail-In-A-Box:
+ echo Your Mail-in-a-Box is running.
Your Mail-in-a-Box is running.
+ echo
Backup.py executed confirmed in the logs below:
Creating initial backup...
+ /opt/mailinabox/management/backup.py
Clearing logs of sensitive data...
Now some sensitive files cannot be removed for they are not found:
+ rm '/var/lib/cloud/instances//scripts/part-00*' '/var/lib/cloud/instances//user-data.txt*' /var/lib/cloud/instances//obj.pkl
rm: cannot remove '/var/lib/cloud/instances//scripts/part-00*': No such file or directory
rm: cannot remove '/var/lib/cloud/instances//user-data.txt*': No such file or directory
rm: cannot remove '/var/lib/cloud/instances//obj.pkl': No such file or directory
Now we need to get the instance id and ensure these files are deleted or there is graceful failing:
# Fetch the EC2 Instance ID
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
echo "Instance ID: $INSTANCE_ID"
# Clear logs for security
echo "Clearing logs of sensitive data..." | tee -a $LOGFILE | logger -t mailinabox_setup
for file in /var/lib/cloud/instances/$INSTANCE_ID/scripts/part-00* \
/var/lib/cloud/instances/$INSTANCE_ID/user-data.txt* \
/var/lib/cloud/instances/$INSTANCE_ID/obj.pkl; do
if [ -e "$file" ]; then
rm -f "$file"
echo "Deleted: $file"
else
echo "File not found: $file"
fi
done
And that is it we not have our stack online:
Timestamp
Logical ID
Status
Detailed status
Status reason
Timestamp
Logical ID
Status
Detailed status
Status reason
2025-02-05 01:48:57 UTC-0500
{removed-sensitive-info}
CREATE_COMPLETE
The full modified ec2 section of the template file is below:
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0
...
EC2Instance:
Type: AWS::EC2::Instance
DependsOn: SmtpCredentialsWaitCondition
CreationPolicy:
ResourceSignal:
Timeout: PT30M
Count: 1
Properties:
InstanceType: !Ref InstanceType
KeyName: !Ref KeyName
ImageId: !Ref InstanceAMI
IamInstanceProfile: !Ref InstanceProfile
SecurityGroups:
- !Ref InstanceSecurityGroup
BlockDeviceMappings:
- DeviceName: /dev/sda1
Ebs:
VolumeType: gp2
VolumeSize: 8
DeleteOnTermination: true
Encrypted: true
Tags:
- Key: Name
Value: !Sub MailInABoxInstance-${AWS::StackName}
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
LOGFILE="/var/log/mailinabox_setup.log"
echo "Starting Mail-in-a-Box setup..." | tee -a $LOGFILE | logger -t mailinabox_setup
exec > >(tee -a $LOGFILE | logger -t mailinabox_setup) 2>&1
logger "Starting Mail-in-a-Box setup."
# Configure needrestart to auto-handle restarts
echo "Updating package lists..." | tee -a $LOGFILE | logger -t mailinabox_setup
apt-get update
echo "Upgrading installed packages..." | tee -a $LOGFILE | logger -t mailinabox_setup
apt-get upgrade -o DPkg::Lock::Timeout=120 -y
# Install missing package (dialog) first
echo "Installing missing package: dialog..." | tee -a $LOGFILE | logger -t mailinabox_setup
apt-get install -y dialog
# Pre-Install
echo "Installing dependencies..." | tee -a $LOGFILE | logger -t mailinabox_setup
apt-get install -o DPkg::Lock::Timeout=120 -y \
librsync-dev \
python3-setuptools \
python3-pip \
python3-boto3 \
unzip \
intltool \
python-is-python3
# pip3 install duplicity==1.0.1
# Install awscli and CloudFormation helper scripts
echo "Installing AWS CLI..." | tee -a $LOGFILE | logger -t mailinabox_setup
cd /tmp
curl "https://awscli.amazonaws.com/awscli-exe-linux-$(uname -m).zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install
pip3 install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz
# Configure variables
echo "Configuring environment variables..." | tee -a $LOGFILE | logger -t mailinabox_setup
# export NONINTERACTIVE=true
export NONINTERACTIVE=1
# added to remove tty-output error.
export DEBIAN_FRONTEND=noninteractive
export TERM=xterm # Ensures dialog has a terminal interface
export SKIP_NETWORK_CHECKS=true
export STORAGE_ROOT=/home/user-data
export STORAGE_USER=user-data
export PRIVATE_IP=$(ec2metadata --local-ipv4)
export PUBLIC_IP=$(ec2metadata --public-ipv4)
export PRIMARY_HOSTNAME=${InstanceDns}.${MailInABoxDomain}
export DEFAULT_PRIMARY_HOSTNAME=${InstanceDns}.${MailInABoxDomain}
export DEFAULT_PUBLIC_IP=$(ec2metadata --public-ipv4)
# Setup Admin Account
echo "Setting up admin account..." | tee -a $LOGFILE | logger -t mailinabox_setup
export EMAIL_ADDR=admin@${MailInABoxDomain}
if [[ -z "${MailInABoxAdminPassword}" ]]; then
export EMAIL_PW=$(tr -dc A-Za-z0-9 </dev/urandom | head -c 16 ; echo '')
if [[ -z "${RestorePrefix}" ]]; then
aws ssm put-parameter \
--overwrite \
--name "/MailInABoxAdminPassword" \
--type SecureString \
--value "$EMAIL_PW"
fi
else
export EMAIL_PW=${MailInABoxAdminPassword}
fi
# Pre-installation steps
echo "Creating user and setting up directories..." | tee -a $LOGFILE | logger -t mailinabox_setup
useradd -m $STORAGE_USER
mkdir -p $STORAGE_ROOT
git clone ${MailInABoxCloneUrl} /opt/mailinabox
export TAG=${MailInABoxVersion}
cd /opt/mailinabox && git checkout $TAG
# Restore if applicable
if [[ -n "${RestorePrefix}" ]]; then
echo "Restoring from S3 backup..." | tee -a $LOGFILE | logger -t mailinabox_setup
duplicity restore --force "s3://${BackupS3Bucket}/${RestorePrefix}" $STORAGE_ROOT
mkdir -p $STORAGE_ROOT/backup
fi
# Install Mail-in-a-Box
echo "Running Mail-in-a-Box setup script..." | tee -a $LOGFILE | logger -t mailinabox_setup
cd /opt/mailinabox/
## && setup/start.sh &&
# Run the setup script with debug logging
bash -x setup/start.sh 2>&1 | tee /tmp/mailinabox_debug.log
# Post-installation steps
echo "Configuring DNS settings..." | tee -a $LOGFILE | logger -t mailinabox_setup
INTERFACE=$(ip route list | grep default | grep -E 'dev (\w+)' -o | awk '{print $2}')
cat <<EOT > /etc/netplan/99-custom-dns.yaml
network:
version: 2
ethernets:
$INTERFACE:
nameservers:
addresses: [127.0.0.1]
dhcp4-overrides:
use-dns: false
EOT
netplan apply
# Remove existing duplicity installation if any
echo "Removing existing duplicity installation..." | tee -a $LOGFILE | logger -t mailinabox_setup
apt-get remove -y duplicity || true
rm -rf /etc/apt/sources.list.d/duplicity-team-ubuntu-duplicity-release-git-jammy.list || true
apt-get update
# Install duplicity via Snap
echo "Installing duplicity via Snap..." | tee -a $LOGFILE | logger -t mailinabox_setup
snap install duplicity --classic
which duplicity
# Create a symlink to /usr/bin/duplicity
echo "Creating symlink for duplicity..." | tee -a $LOGFILE | logger -t mailinabox_setup
ln -sf /snap/bin/duplicity /usr/bin/duplicity
which duplicity
# Block duplicity from being installed via apt
echo "Blocking duplicity from being installed via apt..." | tee -a $LOGFILE | logger -t mailinabox_setup
echo -e "# Duplicity is installed via snap\nPackage: duplicity\nPin: release *\Pin-Priority: -1" > /etc/apt/preferences.d/duplicity
# Verify duplicity installation
echo "Verifying duplicity installation..." | tee -a $LOGFILE | logger -t mailinabox_setup
duplicity --version
# Create Initial Backup
echo "Creating initial backup..." | tee -a $LOGFILE | logger -t mailinabox_setup
/opt/mailinabox/management/backup.py
# Fetch the EC2 Instance ID
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
echo "Instance ID: $INSTANCE_ID"
# Clear logs for security
echo "Clearing logs of sensitive data..." | tee -a $LOGFILE | logger -t mailinabox_setup
for file in /var/lib/cloud/instances/$INSTANCE_ID/scripts/part-00* \
/var/lib/cloud/instances/$INSTANCE_ID/user-data.txt* \
/var/lib/cloud/instances/$INSTANCE_ID/obj.pkl; do
if [ -e "$file" ]; then
rm -f "$file"
echo "Deleted: $file"
else
echo "File not found: $file"
fi
done
# Signal success to CloudFormation
echo "Signaling CloudFormation completion..." | tee -a $LOGFILE | logger -t mailinabox_setup
/usr/local/bin/cfn-signal --success true --stack ${AWS::StackId} --resource EC2Instance --region ${AWS::Region}
# Reboot
echo "Rebooting system..." | tee -a $LOGFILE | logger -t mailinabox_setup
reboot
Now since there is debugging in this file there is still sensitive information you may wish to remove or uncomment ## && setup/start.sh && and comment out bash -x setup/start.sh 2>&1 | tee /tmp/mailinabox_debug.log and remove all echo statements to the log file.
To be Continued…
Testing Sending and Receiving emails
