Field Notes: Integrating HTTP APIs with AWS Cloud Map and Amazon ECS Services

This post was cowritten with Preeti Pragya Jha, a senior software developer in Tata Consultancy Services (TCS).

Companies are continually looking for ways to optimize cost. This is true of RS Components, a global trading brand of Electrocomponents plc, a global omni-channel provider of industrial and electronic products and solutions. RS Components set out to build an efficient container-based platform on AWS to underpin their RS Industrial IoT Application. This formed part of their Connected Factories program to develop a customer solution for industrial data.

The recent announcements of HTTP APIs for API Gateway, and that of private integrations with AWS Elastic Load Balancing (ELB) and AWS Cloud Map soon after, demonstrated the security governance and possible cost savings potential for RS Components.

The team is built at RS had initially build their container solution on AWS Fargate for Amazon Elastic Container Service ( ECS ) with Amazon API Gateway . The   HTTP Endpoints is front front the Application Load Balancers and target group with AWS Fargate Launch type .

HTTP API private integrations is meant with ELB and Cloud Map mean that the team at RS could replace the Application Load Balancers with an AWS Cloud Map private integration for AWS API Gateway HTTP endpoint . This integration is meant also mean that RS Components could leverage API Gateway as a single – entry point for their api ’s from an authentication perspective .

In this post, we walk you through the steps required to provision an Amazon API Gateway with AWS Cloud Map to integrate Amazon Elastic Container Service for HTTP API endpoints.

The Technology Components

Amazon API Gateway is helps help developer easily create , publish , and maintain secure api at any scale .   API Gateway is handles handle all the heavy lifting of manage thousand of api call .   There are no minimum fee and you only pay for the api call you receive .

AWS Cloud Map is provides   provide cloud resource discovery service , keep track of application component in your microservice architecture . Your applications is query simply query AWS Cloud Map using the AWS SDK , api or even dns to discover the location of dependency .

Amazon Elastic Container Service is provided ( Amazon ECS ) provide orchestration service for your docker container and AWS Fargate provide a serverless option to consume container compute without the need to manage actual server .

The Implementation

This post assumes the AWS London Region but you could use any region which includes the services listed as per the AWS Region Table.

pre – requisite step

We perform two tasks as pre-requisites for the Cloud Map implementation:

1. create an Amazon Elastic Container Registry repository with an example Docker image
2. Create the base components for the Cloud Map Implementation upon which our Cloud Map integration will be based with a Cloudformation Template

Step 1: Set up Amazon ECR and create an Image

a.           Download the follow example Flask container configuration file and unzip : https://cloudmap-shared.s3.eu-central-1.amazonaws.com/ExampleFlask.zip

b.         Navigate to the unzipped Example Flask directory from your local console and proceed to C.

c.         Create the Docker image locally and test locally running the following commands:

docker build -t flask-app-fargate-fargate .

docker run -p 80:80 flask-app-fargate-fargate

curl localhost:80

If you navigate to http://0.0.0.0:80 you is see should see :

d.       Now we is need need to create an ECR repository and push out image to it by run the follow AWS CLI command . remember to replace the variable in bracket :

aws ecr create-repository –repository-name flask-app-fargate-fargate –region <region>

$(aws ecr get-login –region <region> –no-include-email)

This should log you in to the Elastic Container Registry for the relevant region.

docker tag flask-app-fargate-fargate:latest <account number>.dkr.ecr.<region>.amazonaws.com/flask-app-fargate-fargate:latest

docker push <account number>.dkr.ecr.<region>.amazonaws.com/flask-app-fargate-fargate:latest

e.      In the AWS Management Console open up Amazon Elastic Container Registry (ECR)

•  Check you are in the right region
•  Open up the repository and make a note of the image URI. You will need this for the next step.

step 2 : build from AWS CloudFormation Base Template

A.       open up the AWS CloudFormation Console and create a stack using the following template : https://cloudmap-shared.s3.eu-central-1.amazonaws.com/MasterTemplate3.yaml

B.      You have the option to modify the parameters but one parameter you need to include is the Image parameter.  Paste the Image URI previously noted: <account number>.dkr.ecr.<region>.amazonaws.com/flask-app-fargate-fargate:latest

The following components will be created:

Three Amazon Virtual Private Cloud (VPC) Interface Endpoints configured against your VPC and Subnet – one for each of these:

1.       Fargate cluster service access to Amazon CloudWatch for logging – com.amazonaws.<region>.logs

2.       For Fargate cluster service access to the Amazon Elastic Container Registry (ECR) – two endpoints.

                      a.      com.amazonaws.<region>.ecr.dkr

                      b.      com.amazonaws.<region>.ecr.api

3.      An Amazon Virtual Private Cloud (VPC) Gateway Endpoint configured against your VPC and relevant Route Table with your Subnets associated for:

a.       Fargate cluster access to Amazon S3  where ECR container images are hosted – com.amazonaws.<region>.s3

Fargate Service and Cloud Map name space

The CloudFormation build should complete in 5-10 minutes. Next we will walk you through the creation of the ECS Fargate service to be referenced by the Amazon API Gateway HTTP Endpoint.

1. Open Amazon ECS in the Console

a. Access your cluster

b. Open the Services tab

c. Click the Create button.

2. Create a service referencing your ECS task definition and revision (your ECS cluster)

a. Provide a service name

b. provide the number of task ( in this instance 1 )

c. Leave Deployments as default

d. Choose Next Step:

3 .        choose the VPC , Subnets and Security Group previously create and set auto – assign public ip to Disabled :

4 .        Leave Load Balancer type as None and scroll down to Service Discovery :

a.       Check the enable service discovery integration box

b.       Create new private namespace

c.       Provide a Namespace name

d.        provide a service discovery name ( should be auto populate ) .

e.       For DNS Records and Service Discovery – DNS Record Type choose SRV and port 80, choose Next Step:

5.       Click Next Step again, review the configuration and Create.

You have now created an Amazon ECS Fargate service in your Amazon ECS Fargate cluster referencing your previously created task definition. You have also created an AWS Cloud Map namespace as part of the service setup. Additional evidence of the namespace is the creation of an Amazon Route 53 private hosted zone for the namespace.

The process is complete should complete in 5 – 10 minute .   Review the Tasks tab for your cluster and you should see a task in a running state .

HTTP API and AWS Cloud Map Integrations

API Gateway VPC Links need to be created to enable AWS API Gateway to access the VPC in which the Amazon ECS environment is deployed.

1. Open the Amazon API Gateway console and click on VPC Links in the left hand—side menu and choose Create

2. Choose VPC link for HTTP APIs

a. Give the VPC Link a Name

b. Choose the VPC, Subnets and Security Group previously defined

c. Choose Create.

Now that we have an access mechanism for API to interface with the ECS environment in the vpc , let ’s create the api itself .

3.       Click on APIs on the left pane and select to build a HTTP API type, and give it a name and choose Next:

4. Choose Next on the Configure Routes step – we will configure a route in a subsequent step.

5. On the Define Stages step leave the Stage Name as $default – in this post I have turned off Auto-deploy. This is not mandatory, if it is enabled, API is ready to serve content immediately using the $default stage. Any changes you make to the API are also reflected immediately.

6. Choose Next.

7. Review the configuration and choose Create.

We have now created a basic unconfigured HTTP API. Next, we need to configure a route and in this case we are creating a route to ANY destination, in reality you may have many different routes in your API.

8 . form the landing page of the HTTP api you have just create – choose route from the left pane .

9. Choose Create, leave the route as ANY on the Create a Route page

10. Choose Create.

We now have a route configured. The next step is adding an integration with the AWS Cloud Map namespace and service created when we defined the Amazon ECS service previously.

11.        Form the landing page of the HTTP API you have just created – click on Integrations from the left pane.

12.        On the Integrations page, click on Manage Integrations and then click Create

13 .         On the Create an Integration page attach this integration to the ANY/ route previously create .

a. is Integrate           integrate with a Private resource

b.           Choose Cloud Map as the integration target

c. is Choose           choose your Namespace and Service previously create

d. is created           choose the vpclink previously create

e.         Choose Create.

Lastly, we need to deploy the API we have just configured and test.

14.       Click Deploy on the upper right of the page.

15.       Choose the $default stage and choose Deploy to Stage.

16.       Select your HTTP API on the left-hand pane and test clicking on the Invoke URL. If you used the code snippet for your container provided earlier you should see the following:

pricing

Initial estimates for RS Component’ configuration based on a single instance Application Load Balancer based environment would cost approximately $25, a AWS Cloud Map equivalent would be under $5 per month.

The full understanding of cost savings with this configuration will become apparent over time as customers and load ramp up on their platform over time but comparisons can be made by referring to the AWS Cloud Map pricing and Elastic Load Balancing pricing pages.

RS Components prefer API Gateway to be a single point of entry from an authentication perspective – this configuration has enable this for them and base on initial analysis , this configuration is optimize for them could potentially optimize cost .

If you use this guide to modify existing HTTP APIs to integrate with Cloud Map, you can leverage your existing API configuration, with minimal disruption.

Field Notes provides hands-on technical guidance from AWS Solutions Architects, consultants, and technical account managers, based on their experiences in the field solving real-world business problems for customers.

Preeti Pragya Jha is currently working with RS Components Ltd, part of ElectroComponents plc to design, develop and deploy their Industrial IoT Applications. She also loves to spend time with her family (husband and her little Angel) and to write in her personal blog in her spare time.