Jason Yu
asked on
how to use curl to read file names from a column in excel file
Dear experts:
I got a task to script by using "curl" command to download hundreds of image files from a file server.
The command format is as following:
curl -o 77711-79137-p01p.png http://s7d3.scene7.com/is/image/companyname/77711-79137-p01p
where
— "http://s7d3.scene7.com/is/image/companyname/“ is common prefix
— 77711-79137-p01p is the image name
— “.png” is the postfix
The image names are from one column of an excel file ended with .xlsx, please see attachment. How could I program for this task?
Thanks.
file-name-input.xlsx
I got a task to script by using "curl" command to download hundreds of image files from a file server.
The command format is as following:
curl -o 77711-79137-p01p.png http://s7d3.scene7.com/is/image/companyname/77711-79137-p01p
where
— "http://s7d3.scene7.com/is/image/companyname/“ is common prefix
— 77711-79137-p01p is the image name
— “.png” is the postfix
The image names are from one column of an excel file ended with .xlsx, please see attachment. How could I program for this task?
Thanks.
file-name-input.xlsx
'xlsx' files are compressed and probably not readable by cURL. If you export the file names to a text file, you could use a batch file or script to read the names and fetch the files.
ASKER
HI, Dave:
Thanks for your quick respond. I have made a script like below, but got "curl: (3) Illegal characters found in URL" error, could you please kindly take a look?
Thanks for your quick respond. I have made a script like below, but got "curl: (3) Illegal characters found in URL" error, could you please kindly take a look?
#!/bin/bash
FILE="./shortname.txt"
OUT=$(awk '{ print $1 }' $FILE)
echo "*** File Name Are Here****"
for file_name in $OUT
do
curl -o $file_name.png http://s7d3.scene7.com/is/image/AveryDennison/$file_name
done
How about wget?
wget -i text_file_with_urls
wget -i text_file_with_urls
ASKER
how about this script:
#!/bin/bash
2 FILE="./shortname.txt"
3 OUT=$(awk '{ print $1 }' $FILE)
4 echo "*** File Name Are Here****"
5 for file_name in $OUT
6 do
7
8 # echo "$file_name"
9 wget -i http://s7d3.scene7.com/is/image/AveryDennison/$file_name
10 # curl -o echo "$file_name".png http://s7d3.scene7.com/is/image/AveryDennison/$file_name
11 done
#!/bin/bash
2 FILE="./shortname.txt"
3 OUT=$(awk '{ print $1 }' $FILE)
4 echo "*** File Name Are Here****"
5 for file_name in $OUT
6 do
7
8 # echo "$file_name"
9 wget -i http://s7d3.scene7.com/is/image/AveryDennison/$file_name
10 # curl -o echo "$file_name".png http://s7d3.scene7.com/is/image/AveryDennison/$file_name
11 done
ASKER
It looks like it doesn't support:
Warning: wildcards not supported in HTTP.
The name is too long, 245 chars total.
Trying to shorten...
New name is ▒%97▒▒}▒%1D▒b%1F'a%0C%078▒ %82z▒z▒+r% 95I▒%82%9D I9>▒▒▒▒mJP ▒w8S%82%8A ▒▒+%11YI▒. ▒▒▒%07%0D% 04%90B%99▒ F▒%19R▒%86 %9F5%1D▒▒▒ ^▒0%90%1Ec ▒▒▒!%05▒%1 D%80>*%04% 01%80▒%8A` ▒N<3▒%02▒c ▒}F%0F▒▒|U _▒▒▒%06$▒▒ N~▒WʵYz%88 ▒s8%17>▒_\ %88▒T▒▒%D▒ ▒%14n%96i% 1C%18▒▒.
The name is too long, 245 chars total.
Trying to shorten...
New name is ▒%97▒▒}▒%1D▒b%1F'a%0C%078▒ %82z▒z▒+r% 95I▒%82%9D I9>▒▒▒▒mJP ▒w8S%82%8A ▒▒+%11YI▒. ▒▒▒%07%0D% 04%90B%99▒ F▒%19R▒%86 %9F5%1D▒▒▒ ^▒0%90%1Ec ▒▒▒!%05▒%1 D%80>*%04% 01%80▒%8A` ▒N<3▒%02▒c ▒}F%0F▒▒|U _▒▒▒%06$▒▒ N~▒WʵYz%88 ▒s8%17>▒_\ %88▒T▒▒%D▒ ▒%14n%96i% 1C%18▒▒.
--2016-06-09 17:30:51-- http://s7d3.scene7.com/is/image/AveryDennison/%C3%97%C1%CB%7D%A4%1D%D3b%1F'a%0C%078%EB%82z%EEz%FD+r%95I%D5%82%9DI9%3E%B6%F2%FD%AC%E3mJP%A1w8S%82%8A%D1%E2+%11YI%BC.%AC%B3%EE%07%0D%04%90B%99%DDF%D8%19R%E0%86%9F5%1D%B6%C8%C0%5E%E70%90%1Ec%B9%DF%CC!%05%C4%1D%80%3E*%04%01%80%EE%BE%8A%60%E0N%3C3%B2%02%A5c%C0%7DF%0F%EA%F8%7CU_%AA%A3%D8%06$%A9%FEN~%D5W%CA%B5Yz%88%FDs8%17%3E%B6_%5C%88%A6T%D0%C1%25D%D1%C3%14n%96i%1C%18%C8%D8%D2%5C%E7%13%80
Reusing existing connection to s7d3.scene7.com:80.
HTTP request sent, awaiting response... 400 Bad Request
2016-06-09 17:30:51 ERROR 400: Bad Request.
Warning: wildcards not supported in HTTP.
The name is too long, 245 chars total.
Trying to shorten...
New name is ▒%97▒▒}▒%1D▒b%1F'a%0C%078▒
The name is too long, 245 chars total.
Trying to shorten...
New name is ▒%97▒▒}▒%1D▒b%1F'a%0C%078▒
--2016-06-09 17:30:51-- http://s7d3.scene7.com/is/image/AveryDennison/%C3%97%C1%CB%7D%A4%1D%D3b%1F'a%0C%078%EB%82z%EEz%FD+r%95I%D5%82%9DI9%3E%B6%F2%FD%AC%E3mJP%A1w8S%82%8A%D1%E2+%11YI%BC.%AC%B3%EE%07%0D%04%90B%99%DDF%D8%19R%E0%86%9F5%1D%B6%C8%C0%5E%E70%90%1Ec%B9%DF%CC!%05%C4%1D%80%3E*%04%01%80%EE%BE%8A%60%E0N%3C3%B2%02%A5c%C0%7DF%0F%EA%F8%7CU_%AA%A3%D8%06$%A9%FEN~%D5W%CA%B5Yz%88%FDs8%17%3E%B6_%5C%88%A6T%D0%C1%25D%D1%C3%14n%96i%1C%18%C8%D8%D2%5C%E7%13%80
Reusing existing connection to s7d3.scene7.com:80.
HTTP request sent, awaiting response... 400 Bad Request
2016-06-09 17:30:51 ERROR 400: Bad Request.
ASKER
How can i use the file name inside "curl" command grepped from the txt file?
curl -o $file_name.png http://s7d3.scene7.com/is/image/AveryDennison/$file_name
curl -o $file_name.png http://s7d3.scene7.com/is/image/AveryDennison/$file_name
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I figured it out myself