bash script to get geo data into values

The following command and it's returned values;
An example command using to get info about an IP along with the commands results.

# curl
  "ip": "",
  "hostname": "",
  "city": "Mountain View",
  "region": "California",
  "country": "US",
  "loc": "37.3860,-122.0838",
  "org": "AS15169 Google Inc.",
  "postal": "94040"

What I need to do is to get those results into values which I can use in a bash script.

into $IP
  "ip": "",
into $CITY
  "city": "Mountain View",
into $REGION
  "region": "California",
  "country": "US",
into $LAT and $LON respectively
  "loc": "37.3860,-122.0838",
into $ASN and $ORG respectively
  "org": "AS15169 Google Inc.",
into $POSTAL
  "postal": "94040"

I started with the following;


LAT=`echo $DATA | cut -d "," -f 1`
LON=`echo $DATA | cut -d "," -f 2`

                LATLON=$(curl -s "" | awk '/loc/ {print $2}' | sed 's/"//g')
                LAT=$(cut -d "," -f1 <<< $LATLON)
                LON=$(cut -d "," -f2 <<< $LATLON)

echo $LAT, $LON

I basically just want to echo each value as shown above. I'm not a programmer and have been struggling with this for days.

# curl -s "" | awk '{print $2}' | sed 's/"//g',,

I've no clue how NOT to pick up the second line nor how to get the first and second parts of LOC and org or worse, if the output changes and the company name has four words for example, etc etc.
Someone on here could do this in 10 minutes using sed and awk.

Finally, since I have the city in the results, how can I find it's time zone as well and stick that into a variable which I can also display?

Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jan SpringerCommented:
I don't understand what you want to do with the data once you retrieve, but this is an example of how you get each field assigned to a variable:



  RESULT=`echo $RESULT | sed 's/"//g'`
  RESULT=`echo $RESULT | sed 's/{//g'`
  RESULT=`echo $RESULT | sed 's/}//g'`

  IP=`echo $RESULT | cut -d "," -f1`
  IP=`echo $IP | cut -d ":" -f2`

  HOST=`echo $RESULT | cut -d "," -f2`
  HOST=`echo $HOST | cut -d ":" -f2`

  CITY=`echo $RESULT | cut -d "," -f3`
  CITY=`echo $CITY | cut -d ":" -f2`

  REGION=`echo $RESULT | cut -d "," -f4`
  REGION=`echo $REGION | cut -d ":" -f2`

  COUNTRY=`echo $RESULT | cut -d "," -f5`
  COUNTRY=`echo $COUNTRY | cut -d ":" -f2`

  LAT=`echo $RESULT | cut -d "," -f6`
  LAT=`echo $LAT | cut -d ":" -f2`

  LONG=`echo $RESULT | cut -d "," -f7`
  ORG=`echo $RESULT | cut -d "," -f8`
  ORG=`echo $ORG | cut -d ":" -f2`

  POST=`echo $RESULT | cut -d "," -f9`
  POST=`echo $POST | cut -d ":" -f2`

  echo $IP
  echo $HOST
  echo $CITY
  echo $REGION
  echo $COUNTRY
  echo $LAT
  echo $LONG
  echo $ORG
  echo $POST

You can also create an array an populate with each of the field data.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Daniel McAllisterPresident, IT4SOHO, LLCCommented:
So what you have is actually a well defined output -- perfect for GREP, AWK & SED.

So I think the first thing you are missing is the opportunity to let AWK use something other than whitespace as a delimiter. As I see the output of the curl response, there is a : that separates the first and the second part... so modify your awk statement to use that:

curl -s "" | awk -F: '{print $2}' | sed 's/"//g'

The output then has some pesky , characters in it, so lets modify your SED to capture them too:

curl -s "" | awk -F: '{print $2}' | sed 's/[",]//g'

That should get you started....

projectsAuthor Commented:
I need the variables so I can work with them as individual values in another script to display the results with other data.
I changed it slightly to output the following;
Mountain View
37.3860, -122.0838
AS15169 Google Inc.

This works perfectly but one small thing. I need the asn as a value and the string after it as another value.
So in the above case, $ASN would contain 'AS15169' and $ORG would contain 'Google Inc.'
The thing I'm not clear on is that some results would contain multiple words. For example, say instead of 'Google Inc.', the result was 'some longer company name, inc.'

Can a simple bash script be intelligent?

Consider the following two test results using random IPs;

# curl -s ""
  "ip": "",
  "hostname": "No Hostname",
  "city": "Tokyo",
  "region": "Tokyo",
  "country": "JP",
  "loc": "35.6850,139.7514",
  "postal": "100-0001"

# curl -s ""
  "ip": "",
  "hostname": "No Hostname",
  "bogon": true
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

Jan SpringerCommented:
Yes.  But we have to change the operation so that we understand that the records are variable length and use the labels provided to determine the field data.

Do you want to account for all fields and represent those fields as empty or only retrieve/store what you receive?
projectsAuthor Commented:
>Do you want to account for all fields and represent those fields as empty
>or only retrieve/store what you receive?

I don't need empty fields, only the values that I can get from the results.
So, in the case of my example;

This is what we get when running the command;

  "ip": "",
  "hostname": "",
  "city": "Mountain View",
  "region": "California",
  "country": "US",
  "loc": "37.3860,-122.0838",
  "org": "AS15169 Google Inc.",
  "postal": "94040"

These are the items I need to store as variables.

into $IP ""
into $CITY "Mountain View"
into $REGION "California"
into $COUNTRY "US"
into $LAT and $LON respectively "37.3860" and "-122.0838"
into $ASN "AS15169"
into $ORG "Google Inc."
into $POSTAL "94040"

If some results aren't returned by the query, no results, instead of an empty value we could insert a word such as 'unknown'. Our php code checks for values and doesn't like when something is received empty so putting something in as a value would prevent php from complaining.

Hope that makes sense.
Jan SpringerCommented:
I'm a perl/expect gal if scripting.  i'll see about getting this to work with bash.
projectsAuthor Commented:
Thank you very much.
I wish I could use another language but bash is what everything else is done in :)
projectsAuthor Commented:
Ping :)
Jan SpringerCommented:
I have not forgotten, just swamped!  I'll see if I can do it when I'm caught up today.
Daniel McAllisterPresident, IT4SOHO, LLCCommented:
I think that the "simple approach" of assuming all data will be there in the same fields is doomed.
I suggest the following approach:

You know that the fields are "printed" with a LABEL, a COMMA and a VALUE, so do your assignments with a BASH case statement:

# initialize (as empty) your target variables
IP="" HOST="" CITY="" REGION="" NATION="" LAT="" LON="" ASN=0 ORG="" POST=""
# input will come from below, but read each line separately
while read LINE ; do
  # the field part has whitespace -- but it won't affect string matching, so we ignore it
  # we DO however need to get rid of the " characters...
  FIELD=`echo $LINE | awk -F: '{ print $1}' | sed 's/"//g'`
  # NOTE: the value part has a whitespace at the front, a , at the end, and the "s that all get removed
  VALUE=`echo $LINE | awk -F: '{ print $2 }' | sed 's/"//g;s/,$//;s/^ *//'`
  # set your TARGET variables to the VALUE based on the FIELD identifier
  case $FIELD in
    ip) IP="$VALUE"
    hostname) HOST="$VALUE"
    org) ASN="`echo $VALUE | awk '{ print $1 }'`"
            ORG="`echo $VALUE | sed s/$ASN\ *//`"
done << END
`curl -s "$TARGET" `

echo $IP
echo $HOST
echo $ASN
echo $ORG

I'll leave it to you do fill in the rest... I could have been more elegant in my scripting, but this should demonstrate more clearly a way to do this...


NOTE: You cannot PIPE the output of the curl command into the while statement, as the pipe creates a subshell and in doing so, the shell variables become obscured. You could to that in KSH, but not BASH.... thus, the input redirect at the end of the while...
projectsAuthor Commented:
I'm not a programmer and I mentioned that at the start of my question. I'm looking for a working script :).
I usually just hire people from freelancer sites for things like this but when they are relatively simple, a bit of a challenge, or interesting to someone, I post here first to offer points.
Daniel McAllisterPresident, IT4SOHO, LLCCommented:
Really? The only parts missing in my "scriptlet" are:
1) inputting the IP address (to replace the hard-coded TARGET="" line), and
2) finishing the rest of the variable assignments -- I already did the "hard one" (the one with the ASN and Org Name together).

The output of your site is multiple lines of the general form:
   "field-name": "field-value",
My scriptlet reads each of these 2-part lines and creates 2 variables: FIELD and VALUE
As demonstrated, for each "FIELD" possible, you create a variable of your choosing and assign it the VALUE -- this gets repeated by the WHILE, and the processing is determined by the CASE. Quite literally, you need to add 2 lines for each remaining "field" you want to capture in the output, with the general form:
   field-name) VARIABLE="$VALUE"

So you should be able to see how the input line "ip": "", is processed:
 - set the variable FIELD to "ip" and the variable VALUE to "" (no quotes in the actual values).
 - use the CASE statement to match the "ip" string value of FIELD
 - set the variable IP to the value of VALUE

NOTE, you specifically asked that this be done in BASH, and that the values be placed into VARIABLES... however, any variables set in this script will disappear once the script exits. ... meaning this has to be the front end to doing SOMETHING with this data, or else this is a programming assignment for a class.

The most common way to export data out of a script is to output it to STD OUT -- but if that's the goal, why worry about assigning the values into variables?

All of that aside, I'm afraid we have conflicting goals... I don't need points to "maintain" my membership here - I get plenty of points each month...
I AM on here to help people -- to solve a problem at their job, or to learn about the technologies, either in school or on the job.
I AM NOT on here to DO people's jobs, or homework, for them.

I'm tapping out on this one... good luck!


PS: The one part of the original problem I did NOT solve is the determination of the timezone. It may be possible to determine that with the Country and Postal values combined... but that'll be more than simple scripting...
projectsAuthor Commented:

Yes, please, go away if you're going to get all huffy.
projectsAuthor Commented:
I've requested that this question be deleted for the following reason:

Close this question and I'll just hire someone to do it for me instead of getting lame replies like 'I'm not going to do your work for you'.
projectsAuthor Commented:
Thank you for giving it a try.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Shell Scripting

From novice to tech pro — start learning today.