Anil Paul

Deploy your code from bitbucket to EC2 instance(s) on the AWS using AWS CodeDeploy is easy to set up.

1. Create IAM Role with the relevant policies
2. create new EC2 Instance by selecting the newly created IAM Role
3. create new s3 bucket to push revision history
4. Create CodeDeploy Application
5. Create CodeDeply Group
6. Add Environment Variables on the Bitbucket Repository
7. Create bitbucket pipeline and code deploy script
8. Create hook scripts to install dependencies and manage artifacts

1. Create IAM Role with the relevant policies

   go to IAM -> policies -> create policy
   Create a policy with name ‘CodeDeploy-EC2-Permissions’ with the following json . you can prefix your company name for instance
   WebdaweCodeDeploy-EC2-Permissions if you want.

   { "Version": "2012-10-17", "Statement": [ { "Action": [ "s3:Get*", "s3:List*" ], "Effect": "Allow", "Resource": "*" } ] }

   1 2 3 4 5 6 7 8 9 10 11 12 13

   {     "Version": "2012-10-17",     "Statement": [         {             "Action": [                 "s3:Get*",                 "s3:List*"             ],             "Effect": "Allow",             "Resource": "*"         }     ] }

   

   Name the Role  CodeDeployRole or prefix your company name like WebdaweCodeDeployRole
   select  AWSCodeDeployRole , AmazonS3FullAccess and the policy you just created (CodeDeploy-EC2-Permissions).

   And Trust Relationship should be the following json

   Trust Relationship

   { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "codedeploy.amazonaws.com", "ec2.amazonaws.com", "codedeploy.ap-southeast-2.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

   {   "Version": "2012-10-17",   "Statement": [     {       "Effect": "Allow",       "Principal": {         "Service": [           "codedeploy.amazonaws.com",           "ec2.amazonaws.com",           "codedeploy.ap-southeast-2.amazonaws.com"         ]       },       "Action": "sts:AssumeRole"     }   ] }

   you can add the zone according to the zone which you want to use.

2. create new EC2 Instance by selecting the newly created IAM Role

   you have to create the instance(s) to which the code is to be deployed with the relevant security groups and vpc ( which you may already set up)
   you need to make sure the IAM role is the one which you have created just now.
   

3. create new S3 Bucket to push revision history

   you just want to create / you can reuse the bucket which you created already. It will work because we have given the full access permission in the IAM role for accessing S3.
   

4. Create CodeDeploy Application

   You have to create a new CodeDeploy Application to deploy to EC2.
   
   you can name it ‘CodeDeployApplication’ or prefix it with your company name

5. Create CodeDeply Group

   Create New Code Deploy Group inside the CodeDeployApplication and name it CodeDeployGroup. while we do this for real project better its the same name as the code branch ( example master / staging/ testing).
   
   And Select the Service Role which we have created CodeDeployRole
   Deployment Type should be In place
   
   Environment Configuration , choose Amazon EC2 instance and  add the tag(s) for instance(s) to which the code is to be deployed.
   Deployment settings should be oneAtATime.
   

6. Add Environment Variables on the Bitbucket Repository

   you can set the following variables either as account variables or the repository variables according to the way your repository is been set up . if there is only one aws account you can set this in account level.
   
   you have to add repository variable which will be the CodeDeployApplication Name as APPLICATION_NAME . in our case it will be CodeDeployApplication

7. Create bitbucket pipeline and code deploy script

   create the codedeploy_deploy.py script which is the edited version of
   this python script.
   what I have done here is to provided an option to pass the deployment group to which

   codedeploy_deploy.py

   """ A BitBucket Builds template for deploying an application revision to AWS CodeDeploy deployment branch should be passed as argument example call - python codedeploy_deploy.py master """ from __future__ import print_function import os import sys from time import strftime, sleep import boto3 from botocore.exceptions import ClientError DEPLOYMENT_BRANCH = sys.argv[1] VERSION_LABEL = strftime("%Y%m%d%H%M%S") BUCKET_KEY = os.getenv('APPLICATION_NAME') + '/' + VERSION_LABEL + \ '-app-bitbucket_builds.zip' def upload_to_s3(artifact): """ Uploads an artifact to Amazon S3 """ print("Deployment Group:" + str(DEPLOYMENT_BRANCH)) try: client = boto3.client('s3') except ClientError as err: print("Failed to create boto3 client.\n" + str(err)) return False try: client.put_object( Body=open(artifact, 'rb'), Bucket=os.getenv('S3_BUCKET'), Key=BUCKET_KEY ) except ClientError as err: print("Failed to upload artifact to S3.\n" + str(err)) return False except IOError as err: print("Failed to access artifact.zip in this directory.\n" + str(err)) return False return True def deploy_new_revision(): """ Deploy a new application revision to AWS CodeDeploy Deployment Group """ try: client = boto3.client('codedeploy') except ClientError as err: print("Failed to create boto3 client.\n" + str(err)) return False try: response = client.create_deployment( applicationName=str(os.getenv('APPLICATION_NAME')), deploymentGroupName=str(DEPLOYMENT_BRANCH), revision={ 'revisionType': 'S3', 's3Location': { 'bucket': os.getenv('S3_BUCKET'), 'key': BUCKET_KEY, 'bundleType': 'zip' } }, deploymentConfigName=str(os.getenv('DEPLOYMENT_CONFIG')), description='New deployment from BitBucket', ignoreApplicationStopFailures=True ) except ClientError as err: print("Failed to deploy application revision.\n" + str(err)) return False """ Wait for deployment to complete """ while 1: try: deploymentResponse = client.get_deployment( deploymentId=str(response['deploymentId']) ) deploymentStatus=deploymentResponse['deploymentInfo']['status'] if deploymentStatus == 'Succeeded': print ("Deployment Succeeded") return True elif (deploymentStatus == 'Failed') or (deploymentStatus == 'Stopped') : print (deploymentStatus) print ("Deployment Failed") return False elif (deploymentStatus == 'InProgress') or (deploymentStatus == 'Queued') or (deploymentStatus == 'Created'): continue except ClientError as err: print("Failed to deploy application revision.\n" + str(err)) return False return True def main(): if not upload_to_s3('/tmp/artifact.zip'): sys.exit(1) if not deploy_new_revision(): sys.exit(1) if __name__ == "__main__": main()

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103

   """ A BitBucket Builds template for deploying an application revision to AWS CodeDeploy deployment branch should be passed as argument example call - python codedeploy_deploy.py master """ from __future__ import print_function import os import sys from time import strftime, sleep import boto3 from botocore.exceptions import ClientError   DEPLOYMENT_BRANCH = sys.argv[1] VERSION_LABEL = strftime("%Y%m%d%H%M%S") BUCKET_KEY = os.getenv('APPLICATION_NAME') + '/' + VERSION_LABEL + \     '-app-bitbucket_builds.zip'   def upload_to_s3(artifact):     """     Uploads an artifact to Amazon S3     """          print("Deployment Group:" + str(DEPLOYMENT_BRANCH))          try:         client = boto3.client('s3')     except ClientError as err:         print("Failed to create boto3 client.\n" + str(err))         return False     try:         client.put_object(             Body=open(artifact, 'rb'),             Bucket=os.getenv('S3_BUCKET'),             Key=BUCKET_KEY         )     except ClientError as err:         print("Failed to upload artifact to S3.\n" + str(err))         return False     except IOError as err:         print("Failed to access artifact.zip in this directory.\n" + str(err))         return False     return True   def deploy_new_revision():     """     Deploy a new application revision to AWS CodeDeploy Deployment Group     """     try:         client = boto3.client('codedeploy')     except ClientError as err:         print("Failed to create boto3 client.\n" + str(err))         return False       try:         response = client.create_deployment(             applicationName=str(os.getenv('APPLICATION_NAME')),             deploymentGroupName=str(DEPLOYMENT_BRANCH),             revision={                 'revisionType': 'S3',                 's3Location': {                     'bucket': os.getenv('S3_BUCKET'),                     'key': BUCKET_KEY,                     'bundleType': 'zip'                 }             },             deploymentConfigName=str(os.getenv('DEPLOYMENT_CONFIG')),             description='New deployment from BitBucket',             ignoreApplicationStopFailures=True         )     except ClientError as err:         print("Failed to deploy application revision.\n" + str(err))         return False       """     Wait for deployment to complete     """     while 1:         try:             deploymentResponse = client.get_deployment(                 deploymentId=str(response['deploymentId'])             )             deploymentStatus=deploymentResponse['deploymentInfo']['status']             if deploymentStatus == 'Succeeded':                 print ("Deployment Succeeded")                 return True             elif (deploymentStatus == 'Failed') or (deploymentStatus == 'Stopped') :                 print (deploymentStatus)                 print ("Deployment Failed")                 return False             elif (deploymentStatus == 'InProgress') or (deploymentStatus == 'Queued') or (deploymentStatus == 'Created'):                 continue         except ClientError as err:             print("Failed to deploy application revision.\n" + str(err))             return False     return True   def main():     if not upload_to_s3('/tmp/artifact.zip'):         sys.exit(1)     if not deploy_new_revision():         sys.exit(1)   if __name__ == "__main__":     main()

    

   Now we have to create bitbucket_pipeline.yml accodingly.

   YAML

   image: python:3.5.1 pipelines: branches: master: - step: name: Test App script: - echo "Testing" - step: name: Deploy To Production trigger: manual script: - apt-get update - apt-get install -y zip - echo "install prerequisites" - pip install boto3==1.3.0 - echo "Zip Artifacts.." - zip -r /tmp/artifact.zip * - python codedeploy_deploy.py CodeDeployGroup

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

   image: python:3.5.1   pipelines:   branches:     master:       - step:           name: Test App           script:             - echo "Testing"       - step:           name: Deploy To Production           trigger: manual           script:             - apt-get update             - apt-get install -y zip             - echo "install prerequisites"             - pip install boto3==1.3.0             - echo "Zip Artifacts.."             - zip -r /tmp/artifact.zip *             - python codedeploy_deploy.py CodeDeployGroup

   here what we are doing is

   adding branch wise logic for testing and deployment. so that we can set variables , files etc according to the branch and environment.
   for instance if you are deploying to staging the environment variables will be different.

   you can see the revision history inside the CodeDeploy Application in AWS Console.

8. Create hook scripts to install dependencies and manage artifacts

   official AWS documentation explained it like below
   ApplicationStop – This deployment lifecycle event occurs even before the application revision is downloaded. You can specify scripts for this event to gracefully stop the application or remove currently installed packages in preparation of a deployment. The AppSpec file and scripts used for this deployment lifecycle event are from the previous successfully deployed application revision.

   Note

   An AppSpec file does not exist on an instance before you deploy to it. For this reason, the ApplicationStop hook does not run the first time you deploy to the instance. You can use the ApplicationStop hook the second time you deploy to an instance.

   To determine the location of the last successfully deployed application revision, the AWS CodeDeploy agent looks up the location listed in the deployment-group-id_last_successful_install file. This file is located in:

   /opt/codedeploy-agent/deployment-root/deployment-instructions folder on Amazon Linux, Ubuntu Server, and RHEL Amazon EC2 instances.

   C:\ProgramData\Amazon\CodeDeploy\deployment-instructions folder on Windows Server Amazon EC2 instances.

   To troubleshoot a deployment that fails during the ApplicationStop deployment lifecycle event, see Troubleshooting failed ApplicationStop, BeforeBlockTraffic, and AfterBlockTraffic deployment lifecycle events.

   DownloadBundle – During this deployment lifecycle event, the AWS CodeDeploy agent copies the application revision files to a temporary location.This event is reserved for the AWS CodeDeploy agent and cannot be used to run scripts.To troubleshoot a deployment that fails during the DownloadBundle deployment lifecycle event, see Troubleshooting a failed DownloadBundle deployment lifecycle event with “UnknownError: not opened for reading”.
   BeforeInstall – You can use this deployment lifecycle event for preinstall tasks, such as decrypting files and creating a backup of the current version.
   Install – During this deployment lifecycle event, the AWS CodeDeploy agent copies the revision files from the temporary location to the final destination folder. This event is reserved for the AWS CodeDeploy agent and cannot be used to run scripts.
   AfterInstall – You can use this deployment lifecycle event for tasks such as configuring your application or changing file permissions.
   ApplicationStart – You typically use this deployment lifecycle event to restart services that were stopped during ApplicationStop.
   ValidateService – This is the last deployment lifecycle event. It is used to verify the deployment was completed successfully.
   BeforeBlockTraffic – You can use this deployment lifecycle event to run tasks on instances before they are deregistered from a load balancer.To troubleshoot a deployment that fails during the BeforeBlockTraffic deployment lifecycle event, see Troubleshooting failed ApplicationStop, BeforeBlockTraffic, and AfterBlockTraffic deployment lifecycle events.
   BlockTraffic – During this deployment lifecycle event, internet traffic is blocked from accessing instances that are currently serving traffic. This event is reserved for the AWS CodeDeploy agent and cannot be used to run scripts.
   AfterBlockTraffic – You can use this deployment lifecycle event to run tasks on instances after they are deregistered from a load balancer.To troubleshoot a deployment that fails during the AfterBlockTraffic deployment lifecycle event, see Troubleshooting failed ApplicationStop, BeforeBlockTraffic, and AfterBlockTraffic deployment lifecycle events.
   BeforeAllowTraffic – You can use this deployment lifecycle event to run tasks on instances before they are registered with a load balancer.
   AllowTraffic – During this deployment lifecycle event, internet traffic is allowed to access instances after a deployment. This event is reserved for the AWS CodeDeploy agent and cannot be used to run scripts.
   AfterAllowTraffic – You can use this deployment lifecycle event to run tasks on instances after they are registered with a load balancer.

   Read More on AWS.
   So now how we hook these life cycle events ?
   you have to create a file called appspec.yml in the repository which will be the inventory for these scripts.
   

   YAML

   version: 0.0 os: linux files: - source: / destination: /var/www/html/your-site-name owner: ec2-user mode: 777 hooks: BeforeInstall: - location: scripts/installDependencies.sh runas: ec2-user - location: scripts/preDeploy.sh runas: ec2-user AfterInstall: - location: scripts/postDeploy.sh runas: ec2-user

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

   version: 0.0 os: linux files:   - source: /     destination: /var/www/html/your-site-name     owner: ec2-user     mode: 777 hooks:    BeforeInstall:      - location: scripts/installDependencies.sh        runas: ec2-user      - location: scripts/preDeploy.sh        runas: ec2-user    AfterInstall:      - location: scripts/postDeploy.sh        runas: ec2-user

   
   so you have to create the relevant scripts in the folder scripts according to this example.
   files will copy the artifact to the destination folder which is /var/www/html/your-site-name according to the example appspec.