How to increase productivity in AWS cloud operations by using shortcuts
In our last blog Join us for an exciting ride of DevOps @Titan , we mentioned that “We value speed and efficiency” and hence have been using an internal cli tool called as aws_shortcuts. And, as we promised, here are more details about AWS shortcuts that take our efficacy to the next level.
The idea came from some of our most pressing problems like getting the EC2 instance name, finding whether a ssm parameter or lambda exists etc. If you’ve worked with AWS, you may be aware that it’s nearly impossible to find if an ssm parameter/lambda exists, especially if you don’t remember the name of that parameter. At the same time, when finding the name of the EC2 instances (as we name all our instances with a private DNS), we often need the name of the machine to ssh.
Using AWS console (slowest) or even aws command line (slower) to discover this information required a lot of time. In the past, we would manage aliases to quickly gather information. This later got converted into a full-fledged tool, including local caching, thereby allowing us to quickly discover information for EC2 servers and Load balancers, IP addresses, SSM params, dns records, S3 buckets, Cloudfronts, Lambdas etc.
AWS cli is powerful when you want to achieve various powerful operations, but it is an overkill when we need to pull some basic information. This is because it is quite difficult to remember the long syntax and options. This is what motivated us to write an AWS shortcut tool. The AWS shortcut tool is a most effective, time-saving utility where you can easily fetch the details about various AWS resources.
Lets see below some use cases for this:
ali api: This command can simply help us list any EC2 host whose name includes apiall api: can help us list any lambda that has the name api alp param: can help us list any ssm param that has param in its nameallb <instance id> can list the load balancer along with its configuration agd api.example.com: can help us learn the name of load balancers, EC2 instances, ports config, etc. All of the above and many more such cases can simply consume too much time. Opening the AWS console itself can take 5-10 seconds wherein our team requires less than 5 seconds to find out any information required at hand.
This is a lifesaver for Titan’s DevOps to quickly find various resource information without spending too much time in AWS GUI
For authentication and authorization, the tool relies on boto3, which is recommended to have read access to all the AWS resources (eg: ReadonlyAccess) to utilize most of the features.
Please note that confidential information like ssm parameters are not being stored to handle security issues.
We have options to reconfigure the shortcut commands according to our convenience This help us customize the shortcuts with our preference.
The cli is a multi-threaded application written in Python. To install this(on Linux/mac), please ensure that you have an aws cli already configured with you and just run the below command.
curl -s https://raw.githubusercontent.com/talk-to/aws_shortcuts/master/awss.sh -o awss.sh && . ./awss.sh && rm awss.shThe installation will automatically create a cron in your local machine which will frequently fetch all the latest AWS configurations for popular resources in multiple threads and caches in the local machine. Local caching avoids repeated AWS api calls and improves the response time.
Few commands supported by the tool
>> awss
---------------------------------------------------------------------
List ec2 instances - ali
List s3 buckets - alb
List lambdas - all
List ssm_parameters - alp
Get ssm_parameters - agp
List route53 domains - ald
Get route53 domains - agd
List lb - allb
List cloudfront distributions - alcf
List commands - awss
Rename commands - awss configure
Fetch latest data from AWS - awss update
Update project to latest version - awss upgrade
---------------------------------------------------------------------To find out more examples and see the source code head over to Github https://github.com/talk-to/aws_shortcuts .
For the curious ones some details below for cron and caching directory
>> Crontab -l
0 */6 * * * /bin/bash -l /<user-home>/.aws_shortcuts/aws_shortcuts/scripts/cron.sh
## Local cache Directory:
## /<user-home>/.aws_shortcuts