wget link autocompletion

Hello,

is there a way that wget complete links (or give a few often alternatives) when I press tab? Similarly as
bash shell, but it should complete links.
Could you show me how to achieve this?

thanks
LVL 6
xRalfAsked:
Who is Participating?
 
The--CaptainConnect With a Mentor Commented:
Why don't you just write a script that uses the URLs in your history file to create a local dummy directory structure that bash will parse (and autocomplete) just like any other part of the filesystem?  

Turn something like this:

http://www.wired.com/reviews/2010/11/review_bg_mobilephones/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+wired%2Findex+%28Wired%3A+Index+3+%28Top+Stories+2%29%29

into a local filesystem entry like this:

mkdir -p 'http://www.wired.com/reviews/2010/11/review_bg_mobilephones'

touch 'http://www.wired.com/reviews/2010/11/review_bg_mobilephones/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+wired%2Findex+%28Wired%3A+Index+3+%28Top+Stories+2%29%29'

then you can use wget to autocomplete by typing two characters (aside from TAB):

wget h[TAB]/[TAB]

(sorry, but I can't yet figure out how to get the second slash to autocomplete - you'll always have to type one of them if you use my idea).

Ugly, and by no means perfect, but it's a lot easier than coding ;-)  Also, do not forget the single quotes!

Cheers,
-Jon


0
 
lucius_theCommented:
I'm afiraid not so. Bash and other shell interpreters can complete thecommand because they know what possible commands are there on a system.

wget expects a link as a parameter. It's not aware of it's location or it's contents until you completely tell him the URL.
0
 
level9wizardCommented:
You would need to write your own bash extension that would dynamically and asynchronously fetch a sitemap with all the urls for that domain (i.e. http://example.com/sitemap.xml), parse it, and then load that into an array or existing class that supports auto-completion. It wouldn't be easy for me, maybe for others - Good luck!
0
Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
xRalfAuthor Commented:
Thank you for answer,

maybe somebody has written the bash extension already but we don't know it's name or how to find it.
0
 
lucius_theCommented:
I believe it's quite unlikely. Although you have similar things on wikipedia and goole, this is only because there is one backend that provides such data.

As wget can connect to ANY http or ftp address, so, expect some theoretical possibility, there is no practical way to achieve this. Another problem is you need the information back from the site instantly or near-instantly so... no way. Not yet at least.
0
 
xRalfAuthor Commented:
This is difficult question. I increased points and made the question easier as follows.

It's sufficient that the bash is able to autocomplete address that are already in firefox history.
0
 
level9wizardCommented:
Just to confirm, you are wanting to input in a Linux command line after you type 'wget ' and then see URL's auto-complete (matching against FireFox's history) by starting to type and press tab?

If I understand correctly, this is likely going to require some c. You may want to consider re-posting this question in a c and linux zones; the ones you are using presently are not going to address that low level functionality.
0
 
Duncan RoeSoftware DeveloperCommented:
The ncftp ftp client can do tab completion on remote file names. Also it bookmarks every site as you visit it for the first time, so you only have to input the address once ever. But it is an ftp client, not http (as wget is)
0
 
NovaDenizenCommented:
In order for wget to do command line tab autocompletion it would have to connect to the server (assuming that the server is completely entered) and somehow via http get a list of all links in the appropriate directory.  Unfortunately, there is no standard way to do a directory listing via http.  In fact in some cases it is actually impossible to return any kind of meaningful directory listing because the local URLs might not have a manageable correspondence with files on the disk, and each of an infinite set of filenames could point to unique automatically generated content.
0
 
xRalfAuthor Commented:
It's sufficient as I described in comment number 34222150
0
 
The--CaptainCommented:
sed -e 's/^\(http.*\/\)\(.*\)/mkdir -p '\''\1'\''; touch '\''\1\2'\''/'

turns that URL above into the two commands above.

Cheers,
-Jon

P.S.  There are no double quotes in the above - those are a bunch of single quotes.
0
 
xRalfAuthor Commented:
Thank you The--Captain
for very original and great solution.
Could you help with two things yet?
I can't find the location of firefox history file.
Here is described http://www.linuxforums.org/forum/newbie/127046-storing-history-file-mozila-firefox-fedora-7-a.html that it should be in history.dat which is not on my system (Xubuntu)
Where should be created the dummy directory structure and files? In the root directory?
Is it possible somewhere it doesn't get in the way?
0
 
The--CaptainCommented:
"Where should be created the dummy directory structure and files? In the root directory?"

Just create a new directory somewhere and just put all the dummy filesystem stuff in there.

Run a locate or find for bookmarks.html to find the file.  If you need help parsing it, you might want to open another question specifically dedicated to that.

Cheers,
-Jon
0
 
xRalfAuthor Commented:
Thank you "locate" found it. I will try to parse and if needed I will open another question.

I've got an idea to download with curl the whole page and extract more links from that, so it will be able to do more completions, but I should probably restrict it for domain, because it could be infinite...
0
 
xRalfAuthor Commented:
But those are bookmarks.html. I have problem with finding history file.
I tried
locate history | grep irefox
I returned only formhistory.sqlite
0
 
The--CaptainCommented:
You're right - I forgot that you wanted history, not bookmarks.  I think the file you're looking for is called places.sqlite , but I don't know off the top of my head the layout - you'd probably have to search online (or ask a new question right here at EE) to find a way to use sqlite off the command line to dump the URLs.

Cheers,
-Jon
0
 
xRalfAuthor Commented:
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.