Deploy BitBucket Code to AWS EC2 Instances using AWS CodeDeploy
Deploy your code from bitbucket to EC2 instance(s) on the AWS using AWS CodeDeploy is easy to set up.
- Create IAM Role with the relevant policies
- create new EC2 Instance by selecting the newly created IAM Role
- create new s3 bucket to push revision history
- Create CodeDeploy Application
- Create CodeDeply Group
- Add Environment Variables on the Bitbucket Repository
- Create bitbucket pipeline and code deploy script
- Create hook scripts to install dependencies and manage artifacts
-
Create IAM Role with the relevant policies
go to IAM -> policies -> create policy
Create a policy with name 'CodeDeploy-EC2-Permissions' with the following json . you can prefix your company name for instance
WebdaweCodeDeploy-EC2-Permissions if you want.{ "Version": "2012-10-17", "Statement": [ { "Action": [ "s3:Get*", "s3:List*" ], "Effect": "Allow", "Resource": "*" } ] }
Name the Role CodeDeployRole or prefix your company name like WebdaweCodeDeployRole
select AWSCodeDeployRole , AmazonS3FullAccess and the policy you just created (CodeDeploy-EC2-Permissions).And Trust Relationship should be the following json
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "codedeploy.amazonaws.com", "ec2.amazonaws.com", "codedeploy.ap-southeast-2.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }
you can add the zone according to the zone which you want to use.
-
create new EC2 Instance by selecting the newly created IAM Role
you have to create the instance(s) to which the code is to be deployed with the relevant security groups and vpc ( which you may already set up)
you need to make sure the IAM role is the one which you have created just now.
-
create new S3 Bucket to push revision history
you just want to create / you can reuse the bucket which you created already. It will work because we have given the full access permission in the IAM role for accessing S3.
-
Create CodeDeploy Application
You have to create a new CodeDeploy Application to deploy to EC2.
you can name it 'CodeDeployApplication' or prefix it with your company name -
Create CodeDeply Group
Create New Code Deploy Group inside the CodeDeployApplication and name it CodeDeployGroup. while we do this for real project better its the same name as the code branch ( example master / staging/ testing).
And Select the Service Role which we have created CodeDeployRole
Deployment Type should be In place
Environment Configuration , choose Amazon EC2 instance and add the tag(s) for instance(s) to which the code is to be deployed.
Deployment settings should be oneAtATime.
-
Add Environment Variables on the Bitbucket Repository
you can set the following variables either as account variables or the repository variables according to the way your repository is been set up . if there is only one aws account you can set this in account level.
you have to add repository variable which will be the CodeDeployApplication Name as APPLICATION_NAME . in our case it will be CodeDeployApplication -
Create bitbucket pipeline and code deploy script
create the codedeploy_deploy.py script which is the edited version of
this python script.
what I have done here is to provided an option to pass the deployment group to which""" A BitBucket Builds template for deploying an application revision to AWS CodeDeploy deployment branch should be passed as argument example call - python codedeploy_deploy.py master """ from __future__ import print_function import os import sys from time import strftime, sleep import boto3 from botocore.exceptions import ClientError DEPLOYMENT_BRANCH = sys.argv[1] VERSION_LABEL = strftime("%Y%m%d%H%M%S") BUCKET_KEY = os.getenv('APPLICATION_NAME') + '/' + VERSION_LABEL + \ '-app-bitbucket_builds.zip' def upload_to_s3(artifact): """ Uploads an artifact to Amazon S3 """ print("Deployment Group:" + str(DEPLOYMENT_BRANCH)) try: client = boto3.client('s3') except ClientError as err: print("Failed to create boto3 client.\n" + str(err)) return False try: client.put_object( Body=open(artifact, 'rb'), Bucket=os.getenv('S3_BUCKET'), Key=BUCKET_KEY ) except ClientError as err: print("Failed to upload artifact to S3.\n" + str(err)) return False except IOError as err: print("Failed to access artifact.zip in this directory.\n" + str(err)) return False return True def deploy_new_revision(): """ Deploy a new application revision to AWS CodeDeploy Deployment Group """ try: client = boto3.client('codedeploy') except ClientError as err: print("Failed to create boto3 client.\n" + str(err)) return False try: response = client.create_deployment( applicationName=str(os.getenv('APPLICATION_NAME')), deploymentGroupName=str(DEPLOYMENT_BRANCH), revision={ 'revisionType': 'S3', 's3Location': { 'bucket': os.getenv('S3_BUCKET'), 'key': BUCKET_KEY, 'bundleType': 'zip' } }, deploymentConfigName=str(os.getenv('DEPLOYMENT_CONFIG')), description='New deployment from BitBucket', ignoreApplicationStopFailures=True ) except ClientError as err: print("Failed to deploy application revision.\n" + str(err)) return False """ Wait for deployment to complete """ while 1: try: deploymentResponse = client.get_deployment( deploymentId=str(response['deploymentId']) ) deploymentStatus=deploymentResponse['deploymentInfo']['status'] if deploymentStatus == 'Succeeded': print ("Deployment Succeeded") return True elif (deploymentStatus == 'Failed') or (deploymentStatus == 'Stopped') : print (deploymentStatus) print ("Deployment Failed") return False elif (deploymentStatus == 'InProgress') or (deploymentStatus == 'Queued') or (deploymentStatus == 'Created'): continue except ClientError as err: print("Failed to deploy application revision.\n" + str(err)) return False return True def main(): if not upload_to_s3('/tmp/artifact.zip'): sys.exit(1) if not deploy_new_revision(): sys.exit(1) if __name__ == "__main__": main()
Now we have to create bitbucket_pipeline.yml accodingly.
image: python:3.5.1 pipelines: branches: master: - step: name: Test App script: - echo "Testing" - step: name: Deploy To Production trigger: manual script: - apt-get update - apt-get install -y zip - echo "install prerequisites" - pip install boto3==1.3.0 - echo "Zip Artifacts.." - zip -r /tmp/artifact.zip * - python codedeploy_deploy.py CodeDeployGroup
here what we are doing is
adding branch wise logic for testing and deployment. so that we can set variables , files etc according to the branch and environment.
for instance if you are deploying to staging the environment variables will be different.you can see the revision history inside the CodeDeploy Application in AWS Console.
-
Create hook scripts to install dependencies and manage artifacts
official AWS documentation explained it like below
ApplicationStop – This deployment lifecycle event occurs even before the application revision is downloaded. You can specify scripts for this event to gracefully stop the application or remove currently installed packages in preparation of a deployment. The AppSpec file and scripts used for this deployment lifecycle event are from the previous successfully deployed application revision.Note
An AppSpec file does not exist on an instance before you deploy to it. For this reason, the ApplicationStop hook does not run the first time you deploy to the instance. You can use the ApplicationStop hook the second time you deploy to an instance.
To determine the location of the last successfully deployed application revision, the AWS CodeDeploy agent looks up the location listed in the
file. This file is located in:deployment-group-id
_last_successful_install/opt/codedeploy-agent/deployment-root/deployment-instructions
folder on Amazon Linux, Ubuntu Server, and RHEL Amazon EC2 instances.C:\ProgramData\Amazon\CodeDeploy\deployment-instructions
folder on Windows Server Amazon EC2 instances.To troubleshoot a deployment that fails during the ApplicationStop deployment lifecycle event, see Troubleshooting failed ApplicationStop, BeforeBlockTraffic, and AfterBlockTraffic deployment lifecycle events.
DownloadBundle – During this deployment lifecycle event, the AWS CodeDeploy agent copies the application revision files to a temporary location.This event is reserved for the AWS CodeDeploy agent and cannot be used to run scripts.To troubleshoot a deployment that fails during the DownloadBundle deployment lifecycle event, see Troubleshooting a failed DownloadBundle deployment lifecycle event with "UnknownError: not opened for reading".
BeforeInstall – You can use this deployment lifecycle event for preinstall tasks, such as decrypting files and creating a backup of the current version.
Install – During this deployment lifecycle event, the AWS CodeDeploy agent copies the revision files from the temporary location to the final destination folder. This event is reserved for the AWS CodeDeploy agent and cannot be used to run scripts.
AfterInstall – You can use this deployment lifecycle event for tasks such as configuring your application or changing file permissions.
ApplicationStart – You typically use this deployment lifecycle event to restart services that were stopped during ApplicationStop.
ValidateService – This is the last deployment lifecycle event. It is used to verify the deployment was completed successfully.
BeforeBlockTraffic – You can use this deployment lifecycle event to run tasks on instances before they are deregistered from a load balancer.To troubleshoot a deployment that fails during the BeforeBlockTraffic deployment lifecycle event, see Troubleshooting failed ApplicationStop, BeforeBlockTraffic, and AfterBlockTraffic deployment lifecycle events.
BlockTraffic – During this deployment lifecycle event, internet traffic is blocked from accessing instances that are currently serving traffic. This event is reserved for the AWS CodeDeploy agent and cannot be used to run scripts.
AfterBlockTraffic – You can use this deployment lifecycle event to run tasks on instances after they are deregistered from a load balancer.To troubleshoot a deployment that fails during the AfterBlockTraffic deployment lifecycle event, see Troubleshooting failed ApplicationStop, BeforeBlockTraffic, and AfterBlockTraffic deployment lifecycle events.
BeforeAllowTraffic – You can use this deployment lifecycle event to run tasks on instances before they are registered with a load balancer.
AllowTraffic – During this deployment lifecycle event, internet traffic is allowed to access instances after a deployment. This event is reserved for the AWS CodeDeploy agent and cannot be used to run scripts.
AfterAllowTraffic – You can use this deployment lifecycle event to run tasks on instances after they are registered with a load balancer.Read More on AWS.
So now how we hook these life cycle events ?
you have to create a file called appspec.yml in the repository which will be the inventory for these scripts.version: 0.0 os: linux files: - source: / destination: /var/www/html/your-site-name owner: ec2-user mode: 777 hooks: BeforeInstall: - location: scripts/installDependencies.sh runas: ec2-user - location: scripts/preDeploy.sh runas: ec2-user AfterInstall: - location: scripts/postDeploy.sh runas: ec2-user
so you have to create the relevant scripts in the folder scripts according to this example.
files will copy the artifact to the destination folder which is /var/www/html/your-site-name according to the example appspec.