peter
asked on
Run shutdown python script on AWS autoscale scale In event
Hi everyone,
I have a python script which removed a AWS Ec2 instance from Zabbix monitoring.
I need to have this script run prior to the instance terminating on a auto-scale scale in event.
I have tried placing the script in /etc/init.d and ln -s to /etc/rc0.d
I have also tried chkconfig 0 1 99 and
chkconfig --add script.py --level 0
None of the above methods cause the script to execute when entering runlevel 0
I have created a auto-scale life cycle hook which waits 90 seconds in order to give the script time to execute.
I then created the following bash script which is polled by cron every minute waiting for the Terminating:Wait signal
This also does not work. Nothing executes.
#!/bin/bash INSTANCE_ID=$(curl http://169.254.169.254/latest/meta-data/instance-id)
aws autoscaling describe-auto-scaling-inst ances --instance-ids $INSTANCE_ID | grep -q 'Terminating:Wait'
if [ $? -eq 0 ]; then
/usr/local/sbin/remove_zab bix.py
aws autoscaling complete-lifecycle-action --lifecycle-hook-name Zabbix-shutdown-wait --auto-scaling-group-name 5sight-prod --lifecycle-action-result CONTINUE --instance-id $INSTANCE_ID
fi
So my question is, HOW can I get a aws Linux instance to execute a python script on a auto_scale Termination event.
Thanks
I have a python script which removed a AWS Ec2 instance from Zabbix monitoring.
I need to have this script run prior to the instance terminating on a auto-scale scale in event.
I have tried placing the script in /etc/init.d and ln -s to /etc/rc0.d
I have also tried chkconfig 0 1 99 and
chkconfig --add script.py --level 0
None of the above methods cause the script to execute when entering runlevel 0
I have created a auto-scale life cycle hook which waits 90 seconds in order to give the script time to execute.
I then created the following bash script which is polled by cron every minute waiting for the Terminating:Wait signal
This also does not work. Nothing executes.
#!/bin/bash INSTANCE_ID=$(curl http://169.254.169.254/latest/meta-data/instance-id)
aws autoscaling describe-auto-scaling-inst
if [ $? -eq 0 ]; then
/usr/local/sbin/remove_zab
aws autoscaling complete-lifecycle-action --lifecycle-hook-name Zabbix-shutdown-wait --auto-scaling-group-name 5sight-prod --lifecycle-action-result CONTINUE --instance-id $INSTANCE_ID
fi
So my question is, HOW can I get a aws Linux instance to execute a python script on a auto_scale Termination event.
Thanks
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Again, here is the check.sh script which checks for the terminating wait signal of the auto scale group
#!/bin/bash
#checks the auto scale groups lifecycle for Terminating:Wait signal
#if recieved, executes whatever.sh
INSTANCE_ID=$(curl http://169.254.169.254/latest/meta-data/instance-id)
EC2_AVAIL_ZONE=`curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone`
export AWS_DEFAULT_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\([0-9][0-9]*\)[a-z]*\$ :\\1:'`"
aws autoscaling describe-auto-scaling-inst ances --instance-ids $INSTANCE_ID | grep -q 'Terminating:Wait'
if [ $? -eq 0 ]; then
#add your commands below
/etc/init.s/httpd stop
python /home/ec2-user/whatever.sh
aws autoscaling complete-lifecycle-action --lifecycle-hook-name zabbix-terminate-delay --auto-scaling-group-name 5sight-test --lifecycle-action-result CONTINUE --instance-id $INSTANCE_ID
sleep 5
sudo init 0
fi
#!/bin/bash
#checks the auto scale groups lifecycle for Terminating:Wait signal
#if recieved, executes whatever.sh
INSTANCE_ID=$(curl http://169.254.169.254/latest/meta-data/instance-id)
EC2_AVAIL_ZONE=`curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone`
export AWS_DEFAULT_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\([0-9][0-9]*\)[a-z]*\$
aws autoscaling describe-auto-scaling-inst
if [ $? -eq 0 ]; then
#add your commands below
/etc/init.s/httpd stop
python /home/ec2-user/whatever.sh
aws autoscaling complete-lifecycle-action --lifecycle-hook-name zabbix-terminate-delay --auto-scaling-group-name 5sight-test --lifecycle-action-result CONTINUE --instance-id $INSTANCE_ID
sleep 5
sudo init 0
fi
ASKER
Here is the life cycle hook
aws autoscaling put-lifecycle-hook \
--lifecycle-hook-name shutdown-delay \
--auto-scaling-group-name myserver-test \
--lifecycle-transition autoscaling:EC2_INSTANCE_T ERMINATING \
--role-arn arn:aws:iam::123456789:rol e/xxx-app- server \
--notification-target-arn arn:aws:sns:us-east-1:1234 56789:inst ance_shutd owns \
--heartbeat-timeout 300 \
--default-result ABANDON
aws autoscaling put-lifecycle-hook \
--lifecycle-hook-name shutdown-delay \
--auto-scaling-group-name myserver-test \
--lifecycle-transition autoscaling:EC2_INSTANCE_T
--role-arn arn:aws:iam::123456789:rol
--notification-target-arn arn:aws:sns:us-east-1:1234
--heartbeat-timeout 300 \
--default-result ABANDON
ASKER
This was a special case that I evolved laws shut down processes which h no o e replied to. I had to resolve myself. But I provided all documentation for anyone e else who may run I to this.
ls -s /usr/local/sbin/remove_zab
I feel like that *should* work, but I haven't done that in a while. Obviously getting it to work that way would be the ideal solution. An uglier solution would be maybe using autoscaling lifecycle hooks to invoke a lambda function that runs the script.
One other thing to keep in mind is that scripts at init level 0 lose a lot of the usual environment variables, so the scripts can end up breaking. For example, if the PATH variable is lost, you have to make sure to specify the full path to any commands.