URL parser port script to work with macOS.

Hi richsark,

"looking for a way to suck in a text file or input file that reads a list of urls..."
How about just "looking for a way to suck in a text file that reads a list of urls...", since "suck in" implies "input" in this context.
But neither a text file nor an input file can read in a list of anything.  A program reads.  An input file is read by a program.

"date-time stamp" doesn't look like an example to me, and it doesn't show the format.  An example would include a real date and time.  Please provide an example of a real date-time stamp so whoever attempts this can clearly see the required format and get it right first time.

Thanks.
tel2

Fairly simple. Just takes a bit of time.

You'll likely use ping to guess whether site is up or curl to test if HTTP/HTTPS is running on the site.

Likely you'll use nmap first (against all ports of interest to you), then based on ports with listeners determine your next action.

hello tel2 and David.

I apologize for not being more clear. I value everyone's time and experience.

to adjust my request.
I wish to have a txt file with said list of various urls. the script would likely have a switch to reference the file.

the logic of the script would go through the input file referencesing on the script name, produce and output file of the findings in Csv format


header would be something like this
date    time     urlname    active   not-reached   port-numbers-opened



example--a.csv
11-22-2020,2:55pm,www.sark.com,yes-reachable,port 8080,443

example--b.csv

11-22-2020,3:55pm,www.sark.com,not-reachable.ports-open,8080,22.23.443

does this look more reasonable and better to understand what I am wishing of the script.

thanks so much
rich

Hi richsark,
You're getting there, but I have questions:
1. What sample input file produced those 2 output files?
2. Is there supposed to be 1 output file per URL in the input file, or what, because your sample looks as if there may be.

When providing sample data, it's important to provide input and output data which correspond with each other, and which correspond to your description of what you want  done.  If you do that, then experts have a better chance of getting the answer right first time.

Hello tel2. Thank you, I have clarified it better
So the input file would be just a txt file with a list of all urls. The file name could be named input-url.txt That contains the list

For example

www.test.com
www.foobar.com
https://sark.test.com

Select allOpen in new window

header would be something like this:  
date,time-AM/PM,url-name,site active Y/N,Nmap-Scan Completed Y/N,Port scan results


example--a.csv
11/23/2020,12:33pm,www.foobar.com,Y,Y,"80,22,8443,192,21"




example--b.csv
date,time-AM/PM,url-name,site active Y/N,Nmap-Scan Completed Y/N,Port scan results
11/23/2020,12:33pm,www.foobar.com,Y,Y,"80,22,8443,192,21"
11/23/2020,12:33pm,https://sark.test.com,N,Y,"22,128,80,8080,443,23"

Select allOpen in new window

​
the output file can be one. The trick is to have the header names there to fill in once the output is created in the csv file

please let me know if you need anything else. I would be more then happy to answer. I appreciate the help

Hi richsark,
I think you're getting closer to a clear specification which is consistent with itself, but not quite.

the output file can be one

So why do you still have 2 output files in the example in your last post?
And why do you have things like www.foo.com in your output example, which is not in your input example.

Is there some reason why you can't simply provide sample input and output files "which correspond with each other" (as I requested in my last post)?  That means the output file should be exactly what you would expect if you fed the input file to the script.

I'm probably going to give up on this now, as we've been down this track before where it took hours of my time for nothing, and I don't want to keep having to ask the same thing over and over.  If you won't put in the effort of making your specifications clear by doing things like providing sample input and output data which properly correspond to each other and to the described requirements, then I'm probably not going to be willing to put in the effort of trying to guess what you mean and provide a solution.  But someone less stubborn might be willing to help.

Hello tel2. I apologize for my mistake. I have a learning disability but I am trying my hardest not waste your time. I really appreciate you spending your valuable time on this.

I try my hardest to comprehend. If it’s ok I will fix my example to insure both I and yourself get to a accurate resolution
Again ,I thank you kindly. 

I have problems with learning too, richsark.
But all your many attempts in this question from 2018 failed, even when you had an intern check it, and I don't want to go down that path again.  So as mentioned, someone less stubborn might help.
Here's my tip for the day:
If you provided sample input data, and an expert wrote a script, and it produced the wrong output, you'd be able to test / work out where it's wrong, right?  So given a sample of input data that you provide, apply the same test to come up with the exact output you'd be happy with, and provide that, along with the input data and your description of what's required when you open a question at EE or wherever.  But before you post the question (or any updates to it), I'd suggest you get someone else who is good at spotting problems to check everything, including the description of what you want.

Hello Tel2 and everyone, I have streamlined my question and fixed the mistakes that caused confusion.

The input file to use as an example:

input-url.txt

www.test.com 
www.foobar.com 
https://sark.test.com

Select allOpen in new window

Header :
header would be the below.

date,time-AM/PM,url-name,site active Y/N,Nmap-Scan Completed Y/N,Port scan results

Select allOpen in new window

The output ( named, output-11-23-2020_1233pm.csv)  file based on input file: I selected an output name to match scan timestamp.

11/23/2020,12:33pm,www.foobar.com,Y,Y,"80,22,8443,192,21"
11/23/2020,12:33pm,https://sark.test.com,N,Y,"22,128,80,8080,443,23"

Select allOpen in new window

I hope this is much cleaner and would allow Tel2 or other colleagues to pitch in please. I really appreciate everyone's effort. Sorry again for delaying or wasting time.

Best regards,

Hello tel2. I actually took my time and attempted myself. Is something missing? Could I ask for your help with this please. I would appreciate it greatly 

Better to modify your script switching your up/down flag to be affixed to each port, as your current data representation fails to consider if one port is down + all others are up, so maybe something like this...

11/23/2020,12:33pm,https://sark.test.com,"22:Y,128:N,80:Y,8080:N,443:Y,23:N"

Select allOpen in new window

For example, port 23 should always be down/off, as no one in their right mind would ever run telnetd, as conversations are cleartext (rather than SSH encrypted text), so running telnetd or ftpd (port 21) will produce a hacked machine... sometimes in only a few minutes.

Same applies to port 80. All port 80 requests should immediately promote to port 443 (HTTPS) conversations, else any logins will be quickly scrapped + resold many times, producing hacked CMSes (like WordPress) which require a great deal of time + money to cleanse.

.