[object Object] Icon

Encoding
Learn how to create, start, manage and modify Encodings

[object Object] Icon

Player
Learn how to create, start, manage and modify Players

[object Object] Icon

Analytics
Learn how to create, start, manage and modify Analyticss

Docs Home
User shortcuts for search
Focus by pressing f
Hide results by pressing Esc
Navigate via   keys

Tue Oct 20 2020

Using Bitmovin Cloud Connect with AWS

Bitmovin Cloud Connect with AWS - TutorialLink Icon

This document explains how to set up Bitmovin Encoding on AWS infrastructure so that the Bitmovin platform can run encoders using the AWS EC2 API.

The instructions in this document for the Bitmovin Encoding Service apply to live encoding and file-based encoding. For a complete list of formats and input types, see the Bitmovin website.

PrerequisitesLink Icon

This feature requires a commercial agreement and needs to be specifically activated for a Bitmovin Account. It is not available by default. You will not be able to complete the configuration below without this activation.

  • A Bitmovin account, enabled for use of the Cloud Connect feature
  • An AWS Account

AWS ConfigurationLink Icon

Create an IAM User

This user is used by the Bitmovin platform to get access to your AWS account and create compute resources to run parts of the the encoding service within it, including starting and stopping EC2 instances.

  1. Add a User, and enable programmatic access to AWS

  2. Grant it the necessary permissions. You can either use the managed AmazonEC2FullAccess policy or create a custom one with the details in the Appendix below.

  3. Collect the credentials (Access Key and Secret Key) that AWS has generated for that user, for example by downloading the CSV file

Set up a VPC

We recommend that you use the default VPC for the respective region. The Default VPC is the only VPC in which you can avoid specifying a subnet, which gives you the advantage that if one availability zone has reached its capacity, your instance will automatically be launched in another (free) zone. If you select a specific Subnet on instance launch (which you are required to do if a non-Default VPC is used), the instance can only be launched in the Availability Zone of the selected Subnet, and if there are no more resources available in that zone, then no instance can be launched.

Your AWS account should automatically come with a default VPC and default Subnet provisioned.

Note: if you have deleted the default VPC in your AWS account, you can re-create it easily.

If you prefer, you can create a custom VPC, with at least one subnet.

Note: the ability to use custom VPC is supported from Encoder version 2.51.0 and above.


Create a Security Group

The security group defines firewall rules that enable network access into your VPC, and on the EC2 instances in particular.

  1. Create a security group and associate it with the appropriate VPC.
  2. For each of the tables below, add inbound rules that allow specific traffic:
TypeAll traffic
DescriptionFor communication between the session manager VM instance and its instance manager VM instances
Port rangeAll
ProtocolAll
Source(select the security group id)


TypeCustom TCP Rule
DescriptionFor communication with the service that manages the encoding
Port range9999
ProtocolTCP
Source104.199.97.13/32
35.205.157.162/32


TypeSSH
DescriptionFor incoming commands (i.e. pulling and starting docker containers)
Port range22
ProtocolTCP
Source104.199.97.13/32
35.205.157.162/32
NoteYou can configure a different SSH port, but will need to remember to set the Bitmovin Infrastructure object accordingly (see below)

Additional inbound rules are required if you are encoding live streams transported over SRT, Zixi or RTMP.


Firewall rules necessary for RTMP live streams

TypeCustom TCP
Descriptionor RTMP live streams
Protocols and Portstcp:1935
SourceAnywhere (or the specific set of addresses where streams will originate from)


Firewall rules necessary for SRT live streams

TypeCustom TCP and Custom UDP (multiple)
DescriptionFor Zixi live streams
Protocols and Portstcp:2088
udp:2088
udp:2089
udp:2090
udp:2091
SourceAnywhere (or the specific set of addresses where streams will originate from)


Firewall rules necessary for Zixi live streams

TypeCustom TCP
DescriptionFor Zixi live streams
Protocols and Portstcp:4444
SourceAnywhere (or the specific set of addresses where streams will originate from)


Bitmovin ConfigurationLink Icon

Before you continue, make sure you have collected the following information:

From your AWS account

  • account_number

From the CSV file with the AWS credentials:

  • access_key
  • secret_key

In case you use a custom VPC:

  • security_group_id
  • subnet_id

Create Infrastructure

To enable your Bitmovin account to run encodings in your AWS account, you need to create an Infrastructure object.

With the Dashboard

  1. In the Bitmovin dashboard, go to the Infrastructure section.
  2. Click the Add new Infrastructure button and select the AWS icon
  3. Fill in the details, with the account_number, access_key and secret_key details collected previously.
  4. Select the infrastructure just created with Show details
  5. For each AWS region in which you want to run encodings, add new Region Settings
    • Fill in the details for that region, using the relevant security_group_id
    • If not using the default VPC, also set the “Subnet” to the relevant subnet_id
    • it is recommended to start with a low number of “Max parallel Encodings”
    • and to leave the “Allowed Machine Types” empty

Please note that the inbound rules are checked when you make an “Add AWS Region Setting” request to Bitmovin API. If there is any permission or configuration issue, the region will not be created and you will be returned to the previous screen showing the details of the infrastructure.


With the Bitmovin APIs

Using the Add AWS Account endpoint, submit the following JSON and replace the respective accessKey, secretKey, and accountNumber values with the appropriate values collected in the previous steps:

1 {
2 "name": "AWS Connect - <aws_account_number>",
3 "description": "<Something meaningful for the dashboard>",
4 "accessKey": "<user_access_key>",
5 "secretKey": "<user_secret_key>",
6 "accountNumber": "<aws_account_number>"
7 }

For each region in which you want to run encodings, you also need to create AWS Account Region Settings. Use the Add AWS Region Setting endpoint to do so, with the following payload:

1 {
2 "securityGroupId": "<aws_security_group_id>",
3 "subnetId": "<aws_subnet_id>"
4 }

If you are using the default VPC in your AWS account, you still need to supply the subnetId property, but it should be set to an empty string.

You can also set the sshPort in this payload if you are not using the default value of 22 in your security group inbound rules.

For example, if your Security Group has ID “sg-12345” and applies to the default VPC and you want to allow any subnet, and you want to run encodings in eu-west11, you will need to submit the following payload to this endpoint:

https://api.bitmovin.com/v1/encoding/infrastructure/gce/<infastructure-id>/regions/EU_WEST_1

1 {
2 "securityGroupId": "sg-12345",
3 "subnetId": ""
4 }

Request access to AMIs

The Bitmovin platform uses Amazon Machine Images (AMIs) from which to create encoder instances in AWS.

Ask your Bitmovin technical contact to:

  • Whitelist access to the AMIs for your specific AWS service account.

You will need to provide us with your AWS account_number to do so.


Run encoding jobs in AWSLink Icon

After configuration has been completed, you will be able to run encoding jobs in your own AWS account. To do so, use the Bitmovin API client SDKs to submit encoding jobs, in the same way as you would do for encodings running in the Bitmovin Managed Cloud service. The only difference is that you need to specify the new infrastructure instead of public cloud regions.

Here is a Python snippet demonstrating how to link your encoding to your infrastructure.

1 # ID of the Infrastructure object created in step 5.
2 infra_id = ‘<infrastructure_id>’
3
4 # AWS region of the AWS-connect setup
5 infra_region = CloudRegion.AWS_EU_WEST_1
6
7 infrastructure = InfrastructureSettings(infrastructure_id=infra_id,
8 cloud_region=infra_region)
9 encoding = Encoding(name='aws connect encoding',
10 cloud_region=CloudRegion.EXTERNAL,
11 infrastructure=infrastructure,
12 encoder_version='STABLE')

Resource QuotasLink Icon

If you want to run several encodings in parallel, the default limits may not be sufficient. In that case, you will have to request limit increases for the following resource in your Region(s), through the Service Quotas page:

Quota nameLimit to request
Running On-Demand All Standard (A, C, D, H, I, M, R, T, Z) instances(maximum number of encodings) (maximum number of instances per encoding) 8 vCPUs
Spot Instance requests(maximum number of encodings) * (maximum number of instances per encoding)
Network interfaces(maximum number of instances)
General Purpose (SSD) volume storage (TiB)(maximum number of encodings) 0.5 + ((number of instances) (number of encodings)) * 0.05

The values above assume 8-core instances. If you believe that your use case requires instances with a different number of cores, this number may need to be increased after discussion with your Bitmovin team.

The maximum number of instances needed depends on the maximum number of parallel encodings running multiplied by the maximum number of instances needed for one encoding. The number of instances used by one encoding varies depending on the input file size and the number and data rate of the encoder representations.

Generally, it is a good idea to multiply the expected limit calculated for your current situation by 2, to have some margin in case you need to ramp up.

Note: there is no form for Spot Instance Request Limit increases, so you have to request this in the Use Case description of the Instance Limit increase form.


AppendixLink Icon

AWS IAM User policy.

The following JSON snippet contain the set of permissions required to perform encoding on EC2 instance. You can assign it to the IAM User, if you prefer not to use the AmazonEC2FullAccess policy.

1 {
2     "Version": "2012-10-17",
3     "Statement": [
4         {
5             "Effect": "Allow",
6             "Action": [
7                 "ec2:TerminateInstances",
8                 "ec2:StartInstances",
9                 "ec2:CreateTags",
10                 "ec2:RunInstances",
11                 "ec2:StopInstances"
12             ],
13             "Resource": [
14                 "arn:aws:ec2:*:*:subnet/*",
15                 "arn:aws:ec2:*:*:instance/*",
16                 "arn:aws:ec2:*:*:volume/*",
17                 "arn:aws:ec2:*:*:security-group/*",
18                 "arn:aws:ec2:*:*:network-interface/*",
19                 "arn:aws:ec2:*::image/*"
20             ]
21         },
22         {
23             "Effect": "Allow",
24             "Action": [
25                 "ec2:DescribeInstances",
26                 "ec2:RequestSpotInstances",
27                 "ec2:DescribeTags",
28                 "ec2:DescribeVpnConnections",
29                 "ec2:DescribeVolumesModifications",
30                 "ec2:DescribeSpotInstanceRequests",
31                 "ec2:DescribeSecurityGroups",
32                 "ec2:GetConsoleOutput",
33                 "ec2:DescribeSpotPriceHistory",
34                 "ec2:CancelSpotInstanceRequests",
35                 "ec2:GetPasswordData",
36                 "ec2:GetLaunchTemplateData",
37                 "ec2:DescribeScheduledInstances",
38                 "ec2:DescribeVpcs",
39                 "ec2:DescribeScheduledInstanceAvailability",
40                 "ec2:DescribeElasticGpus",
41                 "ec2:DescribeInstanceStatus"
42             ],
43             "Resource": "*"
44         }
45     ]
46 }

Give us feedback