3.2.1 Inbound TLS support

MetaDefender Email Gateway Security's SMTP service can be configured to use TLS encryption for increased security. Refer to the sections below how to import/generate a TLS certificate for inbound SMTP traffic.

Prerequisite

  • OpenSSL 32-bit

Incoming TLS support

Follow the instructions below if you want to enable TLS encryption for incoming emails.

I have a pkcs#12 certificate:

For these steps you will need a pkcs#12 certificate file (.pfx).

If you don't have a certificate .pfx file, refer to 148381589 for instructions how to export a certificate to a .pfx file.

  1. Ensure that OpenSSL is installed. If you do not have OpenSSL installed, it can be downloaded from here: https://slproweb.com/products/Win32OpenSSL.html (unofficial distribution)
    Note: MetaDefender Email Gateway Security requires that the OPENSSL_CONF system environment variable is set (normally this is done automatically when installing OpenSSL)

  2. Open an administrator command prompt and navigate to the MetaDefender Email Gateway Security mailagenthost folder (default: C:\Program Files\OPSWAT\Metadefender Email Security\mailagenthost)

  3. Type the following command:

    tlsutil.exe -b "<path to .pfx>" -j "<certificate password>"

    (Replace <path to .pfx> with the path to your .pfx certificate file. -j parameter can be omitted if the certificate is not password protected)

  4. The certificate is imported and a tls.pem file is created.

    C:\Program Files\OPSWAT\Metadefender Email Security\mailagenthost>tlsutil.exe -b "<path to .pfx>" -j "<certificate password>"
    Private key & certificate written to C:\Program Files\OPSWAT\Metadefender Email Security\mailagenthost\tls.pem
  5. Go to Settings > Global Settings and enable TLS selecting STARTTLS optional or STARTTLS required as ENCRYPTION.

I have a pkcs#8 certificate:

For these steps you will need a certificate file and a private key file (pkcs#8).

  1. Ensure that OpenSSL is installed. If you do not have OpenSSL installed, it can be downloaded from here: https://slproweb.com/products/Win32OpenSSL.html (unofficial distribution)
    Note: MetaDefender Email Gateway Security requires that the OPENSSL_CONF system environment variable is set (normally this is done automatically when installing OpenSSL)

  2. Open an administrator command prompt and navigate to the MetaDefender Email Gateway Security mailagenthost folder (default: C:\Program Files\OPSWAT\Metadefender Email Security\mailagenthost)

  3. Type the following command:

    tlsutil.exe -y "<path to certificate file>" -z "<path to private key file>"

    (Replace <path to certificate file> with the path to your certificate file and <path to private key file> with with the path to your private key file)

  4. The certificate is imported and a tls.pem file is created.

    C:\Program Files\OPSWAT\Metadefender Email Security\mailagenthost>tlsutil.exe -y "<path to certificate file>" -z "<path to private key file>"
    Private key & certificate written to C:\Program Files\OPSWAT\Metadefender Email Security\mailagenthost\tls.pem
  5. Go to Settings > Global Settings and enable TLS selecting STARTTLS optional or STARTTLS required as ENCRYPTION.

I want to use a self-signed certificate:

Follow these steps if you wish to use a self-signed certificate.

  1. Ensure that OpenSSL is installed. If you do not have OpenSSL installed, it can be downloaded from here: https://slproweb.com/products/Win32OpenSSL.html (unofficial distribution)
    Note: MetaDefender Email Gateway Security requires that the OPENSSL_CONF system environment variable is set (normally this is done automatically when installing OpenSSL)

  2. Open an administrator command prompt and navigate to the MetaDefender Email Gateway Security mailagenthost folder (default: C:\Program Files\OPSWAT\Metadefender Email Security\mailagenthost)

  3. Type the following command:

    tlsutil.exe -e -d 365 -c US -s "California" -l "San Francisco" -o "Company"

    Replace any of the following parameters with desired values:
    -d = Number of days the certificate is valid
    -c = Country code (2 letter ISO)
    -s = State/District
    -l = City
    -o = Company name

  4. The certificate is imported and a tls.pem file is created.

    C:\Program Files\OPSWAT\Metadefender Email Security\mailagenthost>tlsutil.exe -e -d 365
    -c US -s "California" -l "San Francisco" -o "Company"
    Generating a 4096 bit RSA private key
    ............++
    ..............................................................................++
     
    writing new private key to 'tls_key.pem'
    -----
    Private key & certificate written to C:\Program Files\OPSWAT\Metadefender Email Security\mailagenthost\tls.pem
  5. Go to Settings > Global Settings and enable TLS selecting STARTTLS optional or STARTTLS required as ENCRYPTION.

TLS settings for incoming connections

Starting with v.4.3.0 Email Gateway Security uses Window's Schannel for incoming secure connections by default. This makes possible to configure the accepted SSL/TLS versions, ciphers, cipher suites etc. via Windows registry settings. You can find information about how to configure specific settings here: https://docs.microsoft.com/en-us/windows-server/security/tls/tls-registry-settings

Example: How to disable SSL3 support

Let's check first if SSL 3.0 is supported. For that you should run this command:

Check SSL 3.0 support
openssl.exe s_client -ssl3 -connect localhost:10025 -starttls smtp

If SSL 3.0 is supported you will have an output like this:

images/download/attachments/35749861/ssl3_supported.png

Let's disable it following these steps:

  1. Open the registry editor (regedit)

  2. Go to HKLM SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols

  3. Add a new key called "SSL 3.0"

  4. Add a new subkey to "SSL 3.0" called "Server"

  5. Create an Enabled entry under the "Server" key as a DWORD and set it's value to 0

  6. Restart MetaDefender Email Gateway Security service

The registry should look like this:

images/download/attachments/35749861/ssl3_disable.png

Run the command again to check the SSL 3.0 support now.

images/download/attachments/35749861/ssl3_not_supported.png

In the image above it can be seen that the SSL 3.0 is not supported after the changes.