Link to home
Start Free TrialLog in
Avatar of sunhux
sunhux

asked on

Fast/easy way to isolate which hardenings caused app (mesosphere) issue

After running the CIS hardening script on our RHEL 7,
Mesosphere can't install at all : I don't have the error
message as app team & vendor working on it.

General questions:

Q1:
Besides logging a case with reseller/vendor (which often
disappoints, what's the fastest way to isolate/narrow
down which hardening item caused an issue?
Binary (ie harden half & then kept halving down) isolation
or google for the error?  

Q2:
Or are there free tools out there (in Tripwire we can quickly
tick/untick for remediation/auto-remediation) to ease this
isolation (esp for Linux & Windows)?
Avatar of noci
noci

There should be some error message somewhere..... (otherwise there is no starting point)

Unable to connect to ...
Unable to translate hostname ...
Unable to openfile ...
...??

Good starting point output on the screen
a log file produced wil running
system logging: /var/log/*...
Provide the actual Mesosphere installation steps (cut + paste as text) including all diagnostic output success/failures produced by all steps.
Avatar of sunhux

ASKER

Will ask the apps team on Mon/Tue when the vendor is back.
Avatar of sunhux

ASKER

Feedback from apps team:

After running the hardening script for RHEL7 from CIS, mesosphere could not start up.

Bootstrap messages:
================
2019/09/19 02:46:58 8#8: *1 open() "/usr/share/nginx/html/dcos_install.sh" failed (13: Permission denied), client: 10.121.0.46, server: localhost, request: "GET /dcos_install.sh HTTP/1.1", host: "10.121.0.43:1380"
10.121.0.46 - - [19/Sep/2019:02:46:58 +0000] "GET /dcos_install.sh HTTP/1.1" 403 153 "-" "curl/7.29.0" "-"

dcos_install.sh.log:
===============
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.15.8</center>
</body>
</html>


Further request from apps team:
please help to verify if there is any other services that we should also bypass in order for the mesosphere to work properly.
Avatar of sunhux

ASKER

Logged case with vendor & vendor only asks us to refer to 2 links
below, without telling us which specific hardening is to be loosened:


[1] https://docs.d2iq.com/mesosphere/dcos/1.12/installing/production/system-requirements/
[2] https://docs.d2iq.com/mesosphere/dcos/1.12/installing/production/deploying-dcos/installation/

So, not much of any help from the vendor
SOLUTION
Avatar of noci
noci

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of sunhux

ASKER

Testing out Noci & BTan's  suggestions.
will get back.

Meanwhile, attached is the CIS hardening script that we ran if it helps in further narrowing this down.
RHEL7hardening_Benchmarkscript_v2.1.0.sh
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Also I am thinking if you are into container and using Kubernetes, this scan can be useful to check against CIS benchmark. In other words, install the DC/OS first, run scan then harden accordingly.

https://github.com/mesosphere/kubernetes-security-benchmark/blob/master/README.md
Also I am thinking if you are into container and using Kubernetes, this scan can be useful to check against CIS benchmark. In other words, install the DC/OS first, run scan then harden accordingly.

https://github.com/mesosphere/kubernetes-security-benchmark/blob/master/README.md
Avatar of sunhux

ASKER

Thanks, we don't use Kubernetes (surprisingly) though with Containers, nginx, DCOS/Mesosphere.

This is what the apps team replied:
The problem could be due to the server being cloned form another VM and the docker network is still based on the old IP.
Some of the hardening part on the network portion was reverted to allow port forwarding.
The docker network also needed to be pruned and server needed to be rebooted.
Following that the mesosphere is able to start docker services & stabilized so far.