[object Object] Icon

Encoding
Learn how to create, start, manage and modify Encodings

[object Object] Icon

Player
Learn how to create, start, manage and modify Players

[object Object] Icon

Analytics
Learn how to create, start, manage and modify Analyticss

Docs Home
User shortcuts for search
Focus by pressing f
Hide results by pressing Esc
Navigate via   keys

Tue Oct 20 2020

Using Bitmovin Cloud Connect with GCP

Bitmovin Cloud Connect with GCP - TutorialLink Icon

This document explains how to set up Bitmovin Encoding on Google Cloud infrastructure so that the Bitmovin platform can run encoders using the Compute Engine API.

The instructions in this document for the Bitmovin Encoding Service apply to live encoding and file-based encoding. For a complete list of formats and input types, see the Bitmovin website.

PrerequisitesLink Icon

This feature requires a commercial agreement and needs to be specifically activated for a Bitmovin Account. It is not available by default. You will not be able to complete the configuration below without this activation.

Google Cloud ConfigurationLink Icon

Create a service account

The service account is used by the Bitmovin platform to get access to your GCP account and create compute resources to run parts of the the encoding service within it, including starting and stopping GCE instances.

  1. Create a service account
  2. Grant Compute Instance Admin (v1) permissions to it.
  3. Create a service account key and download the JSON file containing the private key.

Set up a VPC

By default, the Bitmovin platform assumes that you are running instances in an auto mode VPC named “default”.

If you prefer creating a specific custom mode VPC in your GCP account, make a note of the VPC name and subnet(s) that you create.

Note: the ability to use custom VPC is supported from Encoder version 2.51.0 and above.

Set up firewall rules

These are the basic firewall rules, without which starting an encoding will fail.

  1. Go to the Firewall page in the Cloud Console
  2. For each of the tables below, create a new Ingress firewall rule that allows specific traffic.
  • Ensure that you select the appropriate network, in particular if you have created a custom VPC.
  • Target all instances in the network.
  • Unless specified, leave other fields at their default value.

Namedefault-allow-internal
DescriptionFor communication between the session manager VM instance and its instance manager VM instances
Source filters10.0.0.0/8 (or add all VPC Subnets)
Protocols and Portstcp:0-65535
udp:0-65535
icmp



Nameencoder-service
DescriptionFor communication with the service that manages the encoding
Source filters104.199.97.13/32
35.205.157.162/32
Protocols and Portstcp:9999



Namedefault-allow-ssh
DescriptionFor incoming commands (i.e. pulling and starting docker containers)
Source filters104.199.97.13/32
35.205.157.162/32
Protocols and Portstcp:22

Additional firewall rules are required if you are encoding live streams transported over SRT, Zixi or RTMP.

Firewall rules necessary for RTMP live streams


Namertmp-listener
DescriptionFor RTMP live streams
Source filters0.0.0.0/0 (or the specific set of addresses where streams will originate from)
Protocols and Portstcp:1935


Firewall rules necessary for SRT live streams

Namesrt-listener
DescriptionFor SRT live streams
Source filters0.0.0.0/0 (or the specific set of addresses where streams will originate from)
Protocols and Portstcp:2088
udp:2088
udp:2089
udp:2090
udp:2091


Firewall rules necessary for Zixi live streams

Namezixi-listener
DescriptionFor Zixi live streams
Source filters0.0.0.0/0 (or the specific set of addresses where streams will originate from)
Protocols and Portstcp:4444



Bitmovin ConfigurationLink Icon

Before you continue, make sure you have collected the following information:

From your GCP account:

  • project_id

From the JSON service account key:

  • private_key
  • client_email

In case you use a custom VPC:

  • network_id
  • subnet_id

Create infrastructure

To enable your Bitmovin account to run encodings in your GCP project, you need to create an Infrastructure object, using the Bitmovin Infrastructure API.

Using the Add GCE Account endpoint, submit the following JSON and replace the respective serviceAccountEmail, privateKey, and projectId values with the appropriate values collected in the previous steps:

1{
2 "name": "GCP Connect - <gcp_project_id>",
3 "serviceAccountEmail": "<service_account_client_email>",
4 "privateKey": "<service_account_private_key>",
5 "projectId": "<gcp_project_id>"
6 }

If you have created a custom VPC, you will then need to create GCE Account Region Settings for each region that you want to run encodings in. Use the Add Google Cloud Region Setting endpoint to do so, with the following payload:

1 {
2 "network": "/global/networks/<network_id>",
3 "subnetId": "/regions/<gcp_region>/subnetworks/<subnet_id>"
4 }

For example, if your VPC is named “custom-vpc” and has a subnet called “custom-vpc-subnet1”, and you want to run encodings in europe-west1, you will need to submit the following payload to this endpoint:

https://api.bitmovin.com/v1/encoding/infrastructure/gce/<infastructure-id>/regions/EUROPE_WEST_1

1 {
2 "network": "/global/networks/custom-vpc",
3 "subnetId": "/regions/europe-west1/subnetworks/custom-vpc-subnet1"
4 }

Request access to machine images

The Bitmovin platform uses private machine images (MIs) from which to create encoder instances in GCE.

Ask your Bitmovin technical contact to:

  • Whitelist access to the MIs for your specific GCP service account.

You will need to provide the service account’s client_email as well as your Bitmovin Account’s Organisation ID (or email address).


Run encoding jobs in GCPLink Icon

After configuration has been completed, you will be able to run encoding jobs in your own GCP account. To do so, use the Bitmovin API client SDKs to submit encoding jobs, in the same way as you would do for encodings running in the Bitmovin Managed Cloud service. The only difference is that you need to specify the new infrastructure instead of public cloud regions.

Here is a Python snippet demonstrating how to link your encoding to your infrastructure.

1 # ID of the Infrastructure object created in step 5.
2 infra_id = ‘<infrastructure_id>’
3
4 # GCP region of the GPP-connect setup
5 infra_region = CloudRegion.GOOGLE_EUROPE_WEST_1
6
7 infrastructure = InfrastructureSettings(infrastructure_id=infra_id,
8 cloud_region=infra_region)
9 encoding = Encoding(name='gcp connect encoding',
10 cloud_region=CloudRegion.EXTERNAL,
11 infrastructure=infrastructure,
12 encoder_version='STABLE')

Resource QuotasLink Icon

If you want to run several encodings in parallel, then the default quota limits may not be sufficient. In that case, you will have to request limit increases for the following quotas in your regions, through the Quotas page in the Cloud Console.


Quota nameLimit to request
In-use IP addresses(maximum number of encodings) * (maximum number of instances per encoding)
CPUs(maximum number of encodings) * 8
Preemptible CPUs(maximum number of encodings) (maximum number of instances per encoding) 8
Persistent Disk SSD (TB)(maximum number of encodings) 0.5 + ((number of instances) (number of encodings)) * 0.05

The values above assume 8-core instances. If you believe that your use case requires instances with a different number of cores, this number may need to be increased after discussion with your Bitmovin team.

The maximum number of instances needed depends on the maximum number of parallel encodings multiplied by the maximum number of instances for one encoding. The number of instances used by one encoding varies depending on the input file size and the number and data rate of the encoder representations.

Generally, it is a good idea to multiply the expected limit calculated for your current situation by 2, to have some margin in case you need to ramp up.

Give us feedback