performing health checks to application VPC internal applications

Hello AWS experts,

We have EC2 instances running applications in docker containers for which the auto-scaling part is managed bij ECS. It's all running internally in the VPC. So these EC2 instances or docker containers have no public internet-routeable IP addresses.

What we want to do is use Route 53 health checks to check certain TCP ports and send HTTP requests to the applications coming from various locations. The problem is of course that these probes on the various locations cannot get to services running internally on the VPC. What's the proper and easiest way to go to accomplish this?

We want to avoid setting up a monitoring service inside the VPC
We want it to be setup in a way that when we autoscale we don't have to change anything.

J ZSysAdminAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Shalom CarmelCTOCommented:
Well, you nailed it.
Either you open your VPC to external monitoring, or set up a monitoring service inside the VPC.
There is however a sort of "middle ground".
Catchpoint have 2 things to help you going.
1. They have OnPrem agents, that communicate with the Catchpoint service only via outbound traffic.
2. When using their public nodes, you can get a specific list of IPs to whitelist, so you don't have to open your VPC to the entire world.

Look out for similar capabilities with other monitoring tools.

Catchpoint Is Expensive.
Some integrators (such as the company I work at) can give you a slice of the enterprise package at sane costs with low commitments.
If you come to a point when you decide that you need it, contact me.
J ZSysAdminAuthor Commented:
When using catchpoint with option 1 that could make sense.
Bu when using it with option 2 (opening up for the whitelist) then I can just as well open up the services and whitelist the AWS Route 53 health check probes, no?

The tricky part is it also has to continue working when autoscaling will start to be used.

Any ideas on how to accomplish that?
Shalom CarmelCTOCommented:
Yes, you can whitelist the route53 health check probes too.
You have to get the AWS IP addresses and find the addresses under "service": "ROUTE53_HEALTHCHECKS"

When autoscaling starts to work, you will be using a proxy server or a routing mesh as a single front to your cluster/swarm.
Monitor that.
You don't care for individual nodes on an elastic cluster.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
J ZSysAdminAuthor Commented:
For various reasons I don't want to open it up even it has a ACL. So in the meantime I did a lot of reading and apparently a way to do it without opening the services to the internet is using an ABL (appl. load balancer). You can then use the health checks to monitor your services and be notified. Advantage is that it is aware of the concept of docker containers.

I set it up but for some reason it is reporting the services as Not healthy. Anyone has experience with AWS ALBs and is free to take a look together using a remote session?

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.