Introduction

This Deployment Guide provides step-by-step instructions for deploying MetaDefender version 4.9.0 on Amazon Web Services infrastructure.

Organizations interested in protecting their solutions deployed in AWS can leverage MetaDefender to analyze and sanitize files residing, or transitioning, their AWS deployment. MetaDefender can scan and either sanitize or check for known vulnerabilities, depending on the type of traffic it's seeing. The ideal use case would be an organization that allows files to be uploaded to AWS through an external facing web portal. Analyzing files before they are made accessible to the end-users is critical to ensure that no malicious content is allowed and distributed through the web application.

Advanced attacks are concealing the malicious payload and are relying on productivity files (documents, pdfs, images) as a distribution mechanism. Productivity files allow active content to be leveraged, but these features are frequently exploited to execute the malicious behavior.

This guide is for IT infrastructure architects, administrators and DevOps professionals who are seeking to prevent potential malicious traffic being allowed in their AWS Cloud deployment. Threat Prevention is ensured for both productivity files that might be uploaded and for known vulnerabilities that can be identified for all running services/applications deployed in AWS. The vulnerability scanning is checking known vulnerabilities for unpatched OS and running applications.

MetaDefender is provided as an AMI through the AWS Marketplace (MetaDefender Windows and MetaDefender Linux offerings) or as a packaged installer available for download through the OPSWAT Portal.

For installing our solution and deploy it without taking leverage of the predefined AMIs and CloudFormation scripts, please review the guideline listed below:

• 9.3.1.1. Install MetaDefender in AWS EC2

• 9.3.1.2. AMI - Single MetaDefender Deployment

• 9.3.1.3. AMI - Distributed MetaDefender Deployment

For deploying the available AMIs, we provide the following deployment guidelines and as an example CloudFormation scripts:

• Single deployment of MetaDefender in a public subnet

• Distributed deployment - load balanced MetaDefender instances in 2 private subnets, maintained by OPSWAT Central Management deployed in a public subnet

By using predefined CloudFormation scripts the deployment time will be roughly 3 minutes (for single MetaDefender deployment) or 12 minutes (for distributed deployment)

Architecture diagrams

MetaDefender AWS (2).xml

Security

Depending on the selected deployment model, we provide guidance on which are the IAM roles and services that MetaDefender needs to interact with.

All the details for each service and their role in the deployment architecture are detailed in the guidelines below:

• 9.3.1.1. Install MetaDefender in AWS EC2

• 9.3.1.2. AMI - Single MetaDefender Deployment

• 9.3.1.3. AMI - Distributed MetaDefender Deployment

Costs

MetaDefender is made available as an annual subscription. Contact our sales team via our Contact form, available here: https://www.opswat.com/contact

Considering that are over 60 diferent options to license MetaDefender, is highly coupled to the use case and the advanced features that you are considering deploying. Note that the more functionality you are adding to MetaDefender, the more need will be of CPU from the EC2 instance.

Our recommendation would be a minimum 8 vCPU for our lower tiers. The list of recommended EC2 instances are below:

• m5.xlarge

• m5.2xlarge

• m5.4xlarge

• c5.2xlarge

• c5.4xlarge

• c5.8xlarge

We recommend to go with Reserved Instances, considering that you are committing to an annual subscription for MetaDefender.

For pricing per instance, please refer to the official AWS pricelist: https://aws.amazon.com/ec2/pricing/

Sizing

MetaDefender needs an EC2 instance with minimum 8 vCPU, in order to have an optimal response rate to submitted files for analysis. However, depending on the use case and expected throughput (analysis SLA), higher tier instances are recommended.

The system requirements (hardware and supported operating system) are defined here: https://onlinehelp.opswat.com/corev4/2.1._Recommended_System_Requirements.html

Deployment Assets

An ideal deployment in AWS will follow this process:

• Store the files in an S3 bucket before they are submitted to MetaDefender

  • Ideally MetaDefender would be deployed in the same region as the S3 bucket, to avoid additional traffic.

  • However, in the high unlikely event the entire region will fail, S3 provides a Cross-Region Replication, which will insure that the files are still saved and ready to be processed once MetaDefender services are back online.

• Define a pool of jobs for your webapp, which are in a pending state while MetaDefender analyses the files

  • Best option would be to use the SQS service and your application to send the files from the SQS to MetaDefender

    • Every time the files are being uploaded to a temporary S3 bucket, there is a new job added to the SQS

  • If MetaDefender fails, the job is stilled defined in SQS

  • If AZ or even the region fails, if the S3 bucket is synced in multiple regions, the SQS queue should be easily be able to be reconstructed for the remaining files in the bucket

    • On initializing stage, check the files in the S3 bucket and if there is any file left, add them to the SQS

• Based on the result from MetaDefender, the file should be moved to the final location or it's sanitized copy should. Either way, the original file should be deleted from the temporary S3 bucket

• When the job is being removed from SQS, remove also the file from the S3 bucket

Regarding MetaDefender deployment in AWS and it's needed assets/services, we provide 3 deployment guides:

• 9.3.1.1. Install MetaDefender in AWS EC2

• 9.3.1.2. AMI - Single MetaDefender Deployment

• 9.3.1.3. AMI - Distributed MetaDefender Deployment

Health Check

It is important to set a health check of the MetaDefender instance. There are multiple factors that can result in failure of the system, some of them are AWS related (AZ fault, hardware fault) or application fault. Depending on the use case, each customer defines differently what application failed is.

For generic hardware or AZ fault, we recommend always a distributed environment as defined in 9.3.1.3. AMI - Distributed MetaDefender Deployment. By deploying MetaDefender in different Availability Zones with a load balancer in front of them, you will always be sure that no hardware fault will result in service interruption.

1. If you have an ELB in front of MetaDefender instances, configure it to do the health checks using the engines status.

   • How to setup ELB to do health checks: https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-healthchecks.html

   • Use Engines status as API endpoint for health check: https://onlinehelp.opswat.com/corev4/Fetching_Engine_Database_Versions.html

2. In case you are not using an ELB or you would like to have a more advanced health check, our recommendation would be to setup a Lambda function to check on each MetaDefender instance. In case of the health check monitored via Lambda, you will have the flexibility to actually adapt based on the REST API response provided by MetaDefender. One of the following should apply:

   • If all engines are healthy, the instance is considered healthy

   • If some of the engines are not up to date or failing, based on the internal policy you might invalidate this MetaDefender instance.

     • Recovery of MetaDefender will be detailed in the next section

   • If the REST API response is an error, the instance should be considered down

3. An even more advanced check can be considered actually submitting a file to be analyzed using a Lambda function. Note that depending on the file size and complexity, workflow configurations and the number of files in the queue, it might result in timing out the Lambda execution.

   • Submit always the same file to MetaDefender through the REST API

   • Compare the response with a baseline

Backup and Recovery

MetaDefender service in general is not storing any data that needs to be recovered or backed-up. Regardless if it's an hardware or service failure, MetaDefender will not recover the files submitted in the queue. In order to have a sustainable service, not affected by any hardware or software failures, the recommendation would be to:

• Store the files in an S3 bucket before they are submitted to MetaDefender

  • In order to avoid any AZ or Region failure, would be recommended to use S3's Cross-Region Replication

• Define a pool of jobs for your webapp, which are in a pending state while MetaDefender analyses the files

  • Best option would be to use the SQS service and your application to send the files from the SQS to MetaDefender

    • Every time the files are being uploaded to a temporary S3 bucket, there is a new job added to the SQS

  • If MetaDefender fails, the job is stilled defined in SQS

  • If AZ or even the region fails, if the S3 bucket is synced in multiple regions, the SQS queue should be easily be able to be reconstructed for the remaining files in the bucket

    • On initializing stage, check the files in the S3 bucket and if there is any file left, add them to the SQS

• Based on the result from MetaDefender, the file should be moved to the final location or it's sanitized copy should. Either way, the original file should be deleted from the temporary S3 bucket

• When the job is being removed from SQS, remove also the file from the S3 bucket

In case of failure of MetaDefender service, follow the instructions defined in the Troubleshooting section: https://onlinehelp.opswat.com/corev4/9._Troubleshooting_Metadefender_Core.html

Routine Maintenance

Always update the MetaDefender deployment with the new versions published either on:

• portal.opswat.com > Products section

• AWS Marketplace

Release Notes are available here.

MetaDefender provides a direct update mechanism for the licensed analysis engines (all the engines listed in Inventory > Technology). The application logic will require a product update, however the licensed engines are automatically updated (in online deployments). Both the engine and the signature updates for Anti-malware engines will be automatically downloaded and deployed on a daily basis. We recommend to configure the product to check for updates at least every 4h (for more details see 3.4. Update settings).

In general MetaDefender instance has Internet connection. If an offline deployment is considered, make sure you are using either the Central Management or the Update Downloader and that you are uploading the engines' signatures updates minimum once a day.

Emergency Maintenance

In case of any availability failures of an AWS Service, recommended actions would be to pass the load to another MetaDefender instances, preferably deployed in a different AZ than the one affected. Any analysis in progress submitted to MetaDefender will be considered lost. However, is highly recommended to build a resilient system that won't rely on MetaDefender to recover the pending jobs, but to be managed by a queueing mechanism (e.g. AWS SQS). See Backup and Recovery suggested deployment scenarios.

Support

Support policies, costs, levels and SLA's are described on our website, at the Support section: https://www.opswat.com/support

Reference Materials

All MetaDefender documentation is available on onlinehelp.opswat.com

Localization

MetaDefender products and documentation are available exclusively in English for now.