AWS-ECS

-

Elastic Container Service

Virtual Machine vs Containerization

Advantages

  • Environment Encapsulation
    • In the real enterprise development environment, configuring a local development environment spends too much time and sometimes needs to spend a few days to complete that. Too many unpredictable factors are able to affect the environment configuration.
  • Faster boot time

Difference

  • Virtual Machine
    • Infrastructure 
      • contains different host hardware, like CPU, RAM, Disk, Network Interface Card, etc.
    • Host Operating System 
      • the operating system running in the host
    • Hypervisor 
      • running above the host operating system also can directly communicate with the physical host, some of the hypervisors need to run in the host operating system
    • Virtual Machine
      • like an individual machine, each VM has its own operating system and the apps running above the VM operating system
      • spend too much time to launch a new EC2 instance (installing an operating system and execute the scripts, etc.) which cause the low efficiency of elastic scaling, especially under the extremely scenarios 
  • Containerization
    • Infrastructure
      • [same]
    • Host Operating System
      • [same]
    • Docker
      • Communicate with the host operating system, consider as a package which is included in the host operating system
      • As a Docker Engine, it has already know how to run the applications
      • A good optimizing for the application can do launching a new container instantly.
      • Because docker can remove many features from the operating system for making this environment is good and only good for the specific application/container

Docker

Docker is a containerization platform, the most common use is the docker engine. Simply, the architecture is like below:
Users use CLI to run the instructions which install the necessary docker image and packages from the registry to local docker containers.

Docker Image

  • It is kinds of like a template that defines how to launch a new container. Similar to the AMI which defines how to launch a new EC2. 
  • The docker image is generated by "dockerfile", and no matter the personal docker image or the public docker image, they all could be stored in the Docker Hub (the docker official public repository)
  • The "dockerfile" could contain the dockerfile of another existing docker image, commands, the environment variable, entry point, etc.. Through the command "docker build" to generate the docker image.

Docker Network

It defines how to establish a network connection between a container and a service.

Docker Container

It is a instance created based on docker image.

Docker Volume

Similar to the EBS, it is a network-based volume that is able to be mounted to a container for storing persistent data. (If the container is terminated, then the data stored in the container will be lost, that's why docker needs a mountable volume to store persistent data)

Docker Service

Similar to the Auto-Scaling Group, it is a logical group can scale-out/scale-in. Generally, we use it to organize multiple containers that are deployed in different hosts to run the same application as one service through a host cluster.

Docker Swarm (but the most popular product is Kubernetes (k8s), not docker swarm)

It is a cluster orchestrations service to handle the cluster related things, like scale-out/scale-in,  data sync, etc.

Elastic Container Service

ECS is an AWS Docker Orchestration service, it is a wrapped/simplified orchestration service.
  • AWS ECS uses the docker image to launch the task which will be running in an ECS cluster.
  • Each task can host multiple containers
  • Launch Type
    • Fargate: the user doesn't need to maintain the cluster infrastructure, but AWS will do that (serverless)
    • EC2: the user is responsible to maintain the cluster infrastructure, and AWS will do the preparation.
  • Region-specific, but is able to cross multi-AZs
  • The cluster could be assigned to a VPC

ECS Container Agent

  • The ECS container agent can be installed into the normal EC2 instance to make the EC2 instance to become a ECS provisioned EC2 instance, which can send data to ECS cluster in real-time and it is able to execute the instructions (like start/stop a task) from the cluster. In short, it is like a ECS cluster interface for EC2.
  • ECS-optimized AMI has wrapped the ECS agent, therefore, the instance launched by ECS-optimised AMI is the ECS provisioned EC2 instance, and do not need to manually install the agent for it.

Task

  • The task is a definition that Amazon raised. 
  • Type
    • Fargate, entirely controlled by AWS, and the user cannot view or control the EC2 instances, similar to the lambda which is the serverless service too.
  • Network Mode
    • Bridge
      • By default, the docker container is isolated by host
      • Bridge network defines a link-layer which forward the stream to the host or the container
    • Host (not recommend)
      • Do not isolate to host, and use the network of the host
    • AWS-VPC
      • Establish an ENI for a task, and the containers from this task could communicate to outside through this ENI
      • Fargate task instance only supports aws-vpc mode
  • Roles
    • Task role
      • During the task running, the role which used
      • For example
        • Interact with AWS resources, S3, RDB, etc.
    • Task execution role
      • During the initialization of a task, the role which used. 
      • For example
        • Pull image (Elastic Container Register)
        • Publish ClouldWatch log
    • Task Size
      • The physical specifics, like memory, CPU, etc.
    • Task Volume
      • The persistent data storage
      • If mount the task volume, the task type must be EC2, the Fargate tasks only support non-persistent storage volume
    • Task Containers
      • Each task is able to run multiple containers which could be launched by different docker images
  • Volume
    • If the user set the volume to a Task, then the volume will be attached to all containers which inside of this task

Container


The exact place which the application is running on

  • Container Image
    • The container image is the image to launch a new docker container, the image actually is the docker image
    • The image could be pulled from any public repository, like docker hub, like ECR
  • Memory limit
    • Soft limit: memory-reservation
    • Hard limit: memory, when the needed memory exceeds the hard memory limit, the container will be killed
  • Port mappings
    • Mapping the port between the host and the containers
  • Health check
    • Running commands from the inside of the container to check the container healthy
  • Environment
    • The inside physical environment
      • CPU unites
      • GPU unites
      • Essential, the operators after the task are killed
      • Entry point
      • Command
      • Working directory
      • Environment variable

Elastic Beanstalk (EB)

  • For being a convenience for the business which doesn't have experience on AWS, and make it easy to use the most of AWS services and infrastructures.
  • The user only needs to upload its application, all the infrastructures will be configured by EB. 
  • EB will automatically generate instance, load balancer, auto-scaling group, database, etc. for the user.
  • EB also has its own CLI for helping the user to interact with EB
  • Similar to the HEROKU

Summary

ECS cluster

  • The cluster which running multiple tasks

Launch Types

  • Fargate type
    • The EC2 instances are managed by AWS, the user cannot see and manage 
  • EC2 type
    • The EC2 instances are managed by the user 

ECS Agent

  • How to make an EC2 to become an ECS provisioned EC2 instance
ECS Task

  • What is the definition of ECS task
  • How to create the ECS container

Comments

Post a Comment

Popular posts from this blog

Union Find 模板

-
Union Find Union Find Union Find 动态的计算联通块问题 结构 global variables constructor() find(int x) union(int a, int b) 模板 (Map) global variables private Map<Integer, Integer> f; constructor() UnionFind(int n) { this.f = new HashMap<Integer, Integer>(); for (int i = 0; i < n; i++) { f.put(i, i); } } find(int x) int find(int x) { int root = x; // 找到“根” while (f.get(root) != root) { root = f.get(root); } // 拆“整”为“零”，把每个元素直接接到“根”上 while (root != x) { int nx = f.get(x); f.put(x, root); x = nx; } return root; } union(int a, int b) void union(int a, int b) { int rA = find(a); int rB = find(b); if (rA != rB) { f.put(rB, rA); } } 例题 Friend Circles 根据模板添加额外的全局变量，比如此题需要count，在每次合并的时候 count-- ，在初始化的时候或者写一个 setter 和 getter 去给 count 赋值和取值 public class Solution { class UnionFind { private int count; private Map...
Read more

Tree的经典题

-
Tree的经典题 Tree的经典题 找第k小 Kth Smallest Element in a BST 在树上就不需要使用Heap了，因为inorder traversal本身就是一个从小到大的序列，这里我们有几种方法 1. Recursive traversal 创建全局变量，记录traverse到哪儿了，当走到k的时候把值赋给result即可 时间复杂度 O(n) public class Solution { //设置全局变量 private int count = 0; private int result = 0; public int kthSmallest(TreeNode root, int k) { traversal(root, k); return result; } private void traversal(TreeNode root, int k) { if (root == null) { return; } traversal(root.left, k); //如果count等于k的时候才把结果赋给result //注意count从0开始，所以count先++之后再判断 count++; if (count == k) { result = root.val; } traversal(root.right, k); } } 2. Quick Select on Tree 首先要traverse计算每个点到它的时候子树总共有多少个点，然后用quick select，如果大于k，直接去左边找，如果小于k，判断当左边的个数加1正好等于k，那么这个k就落在root上，如果加1之后还小于k那么就要去向右边找了，因为此时左边没有第k个点。 时间复杂度 O(n) 最好最坏都是 public class Solution { public int kthSmal...
Read more

Tree DFS

-
Tree DFS Tree DFS Tree上的DFS分为两种（Divide Conquer + Traversal和纯Divide Conquer） 建立当前/用全局变量（根据不同的方式） root 为 空 （ 返回 什么/ 更新 指针） 左 节点和 右 节点 都 为 空 ，用 root 构建 当前 解（ 返回 什么/ 更新 指针） 左 节点和 右 节点 不 为 空 ，用 左右 构建 当前 解 返回 当前 /选择 左 ， 右 ， 中 Divide Conquer + Traversal 不同 一般 没有 全局变量， 创建当前 变量作为当前最终解 通过 左 节点的最终解和 右 节点的最终解， 计算 并更新 当前 变量 判断 左 节点的最终解， 右 节点的最终解， 当前 变量，在三个解中 选择 一个 作为当前层的最终解 返回 更新传入的参数指针 （ 有时会单独建立 class Result 类 ） 流程 建立 当前 变量（不一定第一步） root 为 空 （ 返回 什么） left 节点和 right 节点 都 为 空 ，用 root 构建 当前 解（ 返回 什么/ 更新 指针） left 节点和 right 节点 不 为 空 ，用 左右 解 leftResult , rightResult 计算 当前 解 选择 符合的解 比较 （ leftResult ， rightResult ， result ） + 选出 合适的解 直接使用 ：当前解 使用 选中的解 返回 更新 传入的指针 例题（Minimum Subtree） public class Solution { class Result { TreeNode minSubtree; int minSum; int sum; Result(TreeNode minSubtree, int minSum, int sum) { this.minSubtree = minSubtree; this...
Read more