Solved

Need Perl or BASH/Expect scripts to login to network devices and inventory SSL certificates

Posted on 2014-02-14
13
926 Views
Last Modified: 2014-02-18
Hi team,   I just need assistance with some kind of "expect" script where I can automate logging in to our nearly 100 load balancers and collect names and expiration dates of all the installed SSL certificates.

Here is an example of a dialog that I do manually...

-bash-3.00$ ssh netadmin@10.1.8.9
Password: 
Last login: Thu Feb 13 15:39:33 2014 from 10.1.2.3
netadmin@(Load-Balancer-A)(cfg-sync Standalone)(Active)(/Common)(tmos)# run util bash
[netadmin@Load-Balancer-A:Active] ~ # cd /config/ssl/ssl.crt
[netadmin@Load-Balancer-A:Active] ssl.crt # ls -l
total 1568
-rw-r--r-- 1 root root 778900 Mar 11  2013 ca-bundle.crt
-rw-r--r-- 1 root root 778900 Mar 11  2013 ca-bundle.crt.preremoval
-rw-r--r-- 1 root root   1334 Mar 11  2013 default.crt
-rw-r--r-- 1 root root   2205 Nov  4 06:08 dtca-bundle.crt
-rw-r--r-- 1 root root   1359 Sep 21  2012 f5-irule.crt
[netadmin@Load-Balancer-A:Active] ssl.crt # openssl x509  -issuer -subject -startdate -enddate -in ca-bundle.crt -noout
issuer= /C=US/O=GTE Corporation/OU=GTE CyberTrust Solutions, Inc./CN=GTE CyberTrust Global Root
subject= /C=US/O=GTE Corporation/OU=GTE CyberTrust Solutions, Inc./CN=GTE CyberTrust Global Root
notBefore=Aug 13 00:29:00 1998 GMT
notAfter=Aug 13 23:59:00 2018 GMT
[netadmin@Load-Balancer-A:Active] ssl.crt # openssl x509  -issuer -subject -startdate -enddate -in default.crt -noout
issuer= /C=US/ST=WA/L=Seattle/O=MyCompany/OU=IT/CN=localhost.localdomain/emailAddress=root@localhost.localdomain
subject= /C=US/ST=WA/L=Seattle/O=MyCompany/OU=IT/CN=localhost.localdomain/emailAddress=root@localhost.localdomain
notBefore=Aug 17 02:25:27 2012 GMT
notAfter=Aug 15 02:25:27 2022 GMT
[netadmin@Load-Balancer-A:Active] ssl.crt # exit
exit
netadmin@(Load-Balancer-A)(cfg-sync Standalone)(Active)(/Common)(tmos)# quit
Connection to 10.1.8.9 closed.
-bash-3.00$ 

Open in new window


As the script iterates logging in to the load balancers, the final output I want to build is something like the format below.....and can potentially be about 7000 lines long.
==============================Expected output csv file==========
DEVICE_Name, CERT_NAME, CN, Issuer, Subject, Issued, Expiration
Load-Balancer-A, ca-bundle.crt,CN=GTE CyberTrust Global Root, CN=GTE CyberTrust Global Root,Aug 13 00:29:00 1998 GMT, Aug 13 23:59:00 2018 GMT
Load-Balancer-A,  default.crt,CN=localhost.localdomain/emailAddress=root@localhost.localdomain, CN=localhost.localdomain/emailAddress=root@localhost.localdomain,Aug 17 02:25:27 2012 GMT, Aug 15 02:25:27 2022 GMT

Open in new window


In the example above, I just printed out two of the certs in the directory, although in reality I need to loop the openssl command through  every one of the certs.

* I had a similar question   --> http://www.experts-exchange.com/Programming/Languages/Scripting/Shell/Q_28301894.html    but I was told by our security team not to run the bash script from within the devices themselves, but to login and do the commands manually, and just collate the output somehow.

THanks very much for all the help.
0
Comment
Question by:rleyba828
  • 7
  • 4
  • 2
13 Comments
 
LVL 76

Expert Comment

by:arnold
Comment Utility
If the certificate is presented during an https type session, you can use a simple perl script with HTTPS connection to each system and during the certificate exchange check its expiration.

This is effectively what you seem to be doing, going into each machine and locally parsing the certificate using openssl.  For Future, you should create a database with your certificates and their expiration dates.  This way as soon as you get one, you know when it will expire.  You could enhance it by adding on which system/s it is installed and their purpose, etc.
0
 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
An expect script will require to have passwords in clear. Are you OK with that?
Please post whether you are OK  with passwords - if so then I'll give you a proper answer.
This is my template for starting
#!/bin/sh
# tcl, tk, expect &c. action comment continuations, but sh doesn't.
# So you can start expect as long as it's somewhere along your path
# without knowing precisely where it is
# (as you whould have to for a #! line).
# Be warned though, that tcl et al insist on matching curly brackets ("{}"),
# even in commemnts.
# Ok, shell will obey the next line but expect won't \
exec expect $0

# just start a shell, it's so much more convenient than spawning individual apps
# then waiting for them to terminate...

spawn sh
expect "$ "

# While you're developing a script, a good line to end it with is something like
# interact \\\\ {puts {}; exit}
# Then you can try stuff out and add it to the script once you get it right.
# If something locks up, type "\\" and expect will exit
# (and kill whatever it was).

Open in new window

0
 

Author Comment

by:rleyba828
Comment Utility
Hi Duncan......, for the password, I will provide the password at runtime.  So the idea is I will run the script on-demand (like once a week) and I don't mind interactively providing the password then.   I only need to supply the password once, as all our devices use TACACS, so the single username/password will work for all.  
  The one that I foresee getting stuck in is how I can do an "ls -l *.crt", and then have the output of the list (could be a hundred or so certs) be temporarily stored in an array because I would need to iteratively do "openssl x509  -issuer -subject -startdate -enddate -in <certificate variable>.crt  -noout" and then collecting the output on a file on my local linux PC.   At some point, I will upload this certificate-inventory.csv to a spreadsheet or database somewhere for processing.

If you could provide some actual syntax above, then it would really be helpful.  

Thanks very much.

Hi Arnold,
   Regarding your suggestion, some of the certs are installed by other teams, and we have no control over which certs get installed, that's why we need to do a *physical* inventory from time to time.   Also we can't simply do an https connection because for the load balancers, some of the  applications, which use the certs are not accessible to us.  We just do the installation.   Thanks
0
 
LVL 76

Assisted Solution

by:arnold
arnold earned 30 total points
Comment Utility
Do you have an internal CA that issues these certificates? Or are these certificates are externally sources?

If you have an option to standardize the certificate procurement/install process such that the start/expiration and all the data you need is collected on the font end, it will make it much simpler including the option to maintain a backup set of certificates.
0
 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
Do you really need ls -l (ell)? I only see you using the name. ls -1 (one) will give you one name per line, so I'll use that. This script builds up an array called crt_files, listing all files with a .crt extension in the directory it cds to. The call to parray at the end demonstrates that it did indeed work
#!/bin/sh
# \
exec expect $0 "$@"
#exp_internal 1
spawn sh
expect -re {[$#] }
exp_send {cd /home/dunc/q/q};exp_send \r
expect -re {[$#] }
exp_send {ls -1 *.crt};exp_send \r

# Skip echo of the ls command
expect \r\n
set i 0
expect \
{
  -re {([^\r]*)\r\n} \
  {
    set crt_files($i) $expect_out(1,string)
    incr i
    exp_continue
  }
  -re {[$#] }
}
parray crt_files

Open in new window

Run it, with an ls of the target for comparison
18:51:45$ ls /home/dunc/q/q/*.crt
/home/dunc/q/q/AIX.termio5.crt     /home/dunc/q/q/edmast.crt
/home/dunc/q/q/HP-UX.termio5.crt   /home/dunc/q/q/hpux.termio5.crt
/home/dunc/q/q/alledit.crt         /home/dunc/q/q/linux.termio5.crt
/home/dunc/q/q/c1in.crt            /home/dunc/q/q/linuxlibc5.termio5.crt
/home/dunc/q/q/ckalloc.crt         /home/dunc/q/q/macros.crt
/home/dunc/q/q/cmndcmmn.crt        /home/dunc/q/q/scrnedit.crt
/home/dunc/q/q/cygwin.termio5.crt  /home/dunc/q/q/sun.termio5.crt
18:51:48$ ./tiny.sh 
spawn sh
sh-4.2$ cd /home/dunc/q/q
sh-4.2$ ls -1 *.crt
AIX.termio5.crt
HP-UX.termio5.crt
alledit.crt
c1in.crt
ckalloc.crt
cmndcmmn.crt
cygwin.termio5.crt
edmast.crt
hpux.termio5.crt
linux.termio5.crt
linuxlibc5.termio5.crt
macros.crt
scrnedit.crt
sun.termio5.crt
sh-4.2$ crt_files(0)  = AIX.termio5.crt
crt_files(1)  = HP-UX.termio5.crt
crt_files(10) = linuxlibc5.termio5.crt
crt_files(11) = macros.crt
crt_files(12) = scrnedit.crt
crt_files(13) = sun.termio5.crt
crt_files(2)  = alledit.crt
crt_files(3)  = c1in.crt
crt_files(4)  = ckalloc.crt
crt_files(5)  = cmndcmmn.crt
crt_files(6)  = cygwin.termio5.crt
crt_files(7)  = edmast.crt
crt_files(8)  = hpux.termio5.crt
crt_files(9)  = linux.termio5.crt
18:51:52$ 

Open in new window

0
 

Author Comment

by:rleyba828
Comment Utility
Hi duncan,

   I had something working partially before I saw your reply.  I so far had this thing working and it can login to one device, and then output the directory listing of all certs to a file, and then logout elegantly.  Good thinking about ls -1 instead of ls -l.  This is what I had which can login properly and do a listing of all the certs.


#!/usr/bin/expect -f

set env(TERM) dumb
set timeout -1
spawn ssh netadmin@10.1.8.9
expect -exact "Password: "
send -- "test@123\r"
expect  "bp>"
send  -- "!bash\r"
expect  "netadmin@Load-Balancer-A"
sleep 1
send -- "cd /config/ssl\r"
expect  "netadmin@Load-Balancer-A"
sleep 1

send -- "ls -1 *.crt\r"
expect  "netadmin@Load-Balancer-A"
puts [open extractcertoutput.txt w] $expect_out(buffer)
sleep 1
send -- "exit\r"
expect  "bp>"
send -- "exit\r"
expect "closed."
##send -- "\r\r\r"
expect eof

Open in new window


I tried to adapt what you had, but the script just gets stuck at the password prompt.  I tried google searching the correct syntax, and tried doing all sorts of permutations of *asswords*, "Password:", "*assword*", with quotes, without quotes,  with expect -exact and without the -exact option, but the script just sits there.  What could be wrong?

#!/bin/sh
# \
exec expect $0 "$@"
#exp_internal 1
spawn sh
expect -re {[$#] }
exp_send {ssh netadmin@10.1.8.9};exp_send \r
expect -exact {Password}
exp_send {test@123\r}
expect {bp>}
exp_send {!bash};exp_send \r

### Skip echo of the ls command
#expect \r\n
#set i 0
#expect \
#{
#  -re {([^\r]*)\r\n} \
#  {
#    set crt_files($i) $expect_out(1,string)
#    incr i
#    exp_continue
#  }
#  -re {[$#] }
#}
#parray crt_files


Thanks very much.

For Arnold,  unfortunately we have very little control about the issuance  and scheduling of the ssl Certs and another team manages all the CA signing and all, we just own the load balancers.  Thanks
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 34

Accepted Solution

by:
Duncan Roe earned 470 total points
Comment Utility
This line is your problem:
exp_send {test@123\r}
Inside curly braces, escape sequences are not expanded. That's why you need
exp_send {test@123};exp_send \r
You could use double-quotes and have one exp_send but then you can run into other troubles (like having to escape square brackets and dollar signs) so my editor macro does it that way.
By the way, if you run into a problem like you did, try inserting the line exp_internal 1. Expect will show you exactly what it sends and how it parses the response. You can see it commented out in my script.
A few observations re your posted script:
You don't need the sleep commands
Usually there's no space after the Password: prompt
I do not recommend set timeout -1
0
 

Author Comment

by:rleyba828
Comment Utility
Hi duncan, excellent!   The exp_internal 1 option is such a lifesaver.  And you're right...it pointed to exactly what was wrong in the *conversation* between the script and the remote host.

We're almost there.   The last thing I have to do is to run openssl on EACH of the certificates we are looping through and accumulate the output to a file on the local computer running the script (not write the file on the load balancer).   The command I have inserted below "puts [open duncanoutput.txt w] $expect_out(buffer)" somehow accumulates the cert list, all I just need is to extract the cert info.

I have sort of inserted where I think the commands should be in my code below, but I'm not sure of the syntax or how to express it in a command.

Here's the nearly finished script.

#!/bin/sh
# \
exec expect $0 "$@"
#exp_internal 1
spawn sh
#sleep 1
expect -re {[$#] }
exp_send {ssh netadmin@10.1.8.9};exp_send \r
#sleep 1
expect  {assword:}
exp_send {test@123};exp_send \r
expect {bp>}
exp_send {!bash};exp_send \r
expect  {~ # }
exp_send {cd /config/ssl/ssl.crt};exp_send \r
expect  {ssl.crt # }
exp_send {ls -1};exp_send \r
expect  {ssl.crt # }
puts [open duncanoutput.txt w] $expect_out(buffer)

### Skip echo of the ls command
#expect \r\n
#set i 0
#expect \
#{
#  -re {([^\r]*)\r\n} \
#  {
#    set crt_files($i) $expect_out(1,string)
#### exp_send {openssl x509  -issuer -subject -startdate -enddate -in crt_files($i) -noout};exp_send \r
####aggregate the output to a file somehow........
#    incr i
#    exp_continue
#  }
#  -re {[$#] }
#}
#parray crt_files
exp_send {exit};exp_send \r
expect  {bp}
exp_send {quit};exp_send \r

Open in new window


* a couple of questions though....what does exec expect $0 "$@" mean?   Also, somewhere in the middle of your code you had the construct expect \.   Does it mean that the continuation is on the next line?

Thanks very much.
0
 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
Your 2 questions:
1. expect $0 is as per http:#a39862029. The addition of "$@" passes to expect any and all command-line arguments given to the shell script. You can access them as the argv list - I thought you might use arguments later, e.g. to pass in host addresses.
2. Yes expect \ is a continuation, to the curly bracket { on the next line. I could have written expect { but I like braces on their own lines
0
 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
I assume you want to fix the line starting #### exp_send
This line is a case where you want to use double-quotes instead of braces. Braces are for constant strings. If you want parameter substitution you have to use double-quotes. (This is Tcl syntax: expect is a Tcl extension). You need
exp_send "openssl x509  -issuer -subject -startdate -enddate -in $crt_files($i) -noout\r"

Open in new window

There's no need for a separate exp_send for the carriage return since escape expansion and parameter substitution are both done inside double-quotes.
Note the array access is $crt_files($i) - the first dollar sign says this is a variable, the parentheses say it is an array element and the second dollar says that the array index is a variable. Tcl needs the first dollar sign when reading variables but not when writing to them (with the set command).
0
 

Author Comment

by:rleyba828
Comment Utility
Hi duncan,

  thanks for your tips above.  Finally, I've got it working.  See my final code below.  I must admit that TCL has so many idiosyncracies.  For example, I spent the better part of today, trying to figure out, why, in your example, when you print parrary crt_files, then the elements of the array appear intact, but when I now substitute the elements into a function $crt_files($j) below, the elements have all these weird characters ahead and after the element
(such as \u00070m , \u001b[00m,  0;32m , etc) and it took quite a lot of google searching to figure out a way to remove them.

  Also seems like basic things like addition   Set i = $i+1 don't really work.


#!/bin/sh
# \
exec expect $0 "$@"
#exp_internal 1
spawn sh
#sleep 1
expect -re {[$#] }
exp_send {ssh netadmin@10.1.8.9};exp_send \rls
#sleep 1
expect  {assword:}
exp_send {test@123};exp_send \r
expect {bp>}
exp_send {!bash};exp_send \r
expect  {~ # }
exp_send {cd /config/ssl/ssl.crt};exp_send \r
expect  {ssl.crt # }
exp_send {ls -1};exp_send \r
expect \r\n
#puts [open duncanoutput.txt w] $expect_out(buffer)
set i 0
set certcount 0
expect \
  { 
     -re {([^\r]*)\r\n} \
   {
    set crt_files($i) $expect_out(1,string)
    incr i
    set certcount i
    exp_continue
  }

    {ssl.crt # }
}
parray crt_files
#incr i -1
exp_send "####value of I is $i\r"
expect  {ssl.crt # }   


for {set j 0} {$j < $i} {incr j} { 
    set cleanedcert [string map {\u00070m "" \u001b[00m "" 0;32m ""} $crt_files($j)]
    exp_send "openssl x509  -issuer -subject -startdate -enddate -in $cleanedcert -noout\r"
    expect  {ssl.crt # }   
}

  exp_send {exit};exp_send \r
  expect  {bp}
  exp_send {quit};exp_send \r

Open in new window


At any rate, we'll stop here , thanks very much for all your help.  I have a long way to go in my simple SSL certificate collection project, but the best way is to break down my questions into individual tasks, and ask help for those.  This way, I don't burden you or other experts with follow up questions.

Thanks very much for all the help, and this questions deservers more than 500 points, I believe.
0
 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
Everything in Tcl is a string. You must use the expr function to do arithmetic:
set i [expr {$i+1}].
Tcl takes some getting used to. Post more Q's as you have them. Good luck in the meantime,
Cheers ... Duncan.
0
 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
Manual pages are in mann. I usually leave an xterm sitting in mann and issue man commands by e.g. man ./expr<Tab>
0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

Recently, an awarded photographer, Selina De Maeyer (http://www.selinademaeyer.com/), completed a photo shoot of a beautiful event (http://www.sintjacobantwerpen.be/verslag-en-fotoreportage-van-de-sacramentsprocessie-door-antwerpen#thumbnails) in An…
Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now