crawling gofundme

There are
airline
hotel
websites that download and sort results from other websites

gofundme.com homepage only shows successful campaigns

I would like to see unsuccessful campaigns where donor only has $1
Maybe a php crawler or perl crawler
LVL 1
rgb192Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Mark BradyPrincipal Data EngineerCommented:
Is this your account you want to scrape or anyones account? I'm not sure they would appreciate or allow people to scrape their site, however if you had a login for your owns campaigns then it wouldn't be hard to scrape your account and extract the data you want.

I prefer doing scrapes in python myself and have done a lot of them. The first thing you will need to do is see how they authenticate logins.

For example, some sites might have little to no security requiring only a username and password to be submitted in order to login. Others such as google sites use OAuth2 for verification. This is very tough to simulate in a scrape and you will need to deal with cookies that the site sets in their responses.

If you are thinking of writing one for yourself then I suggest a great tool. Fiddler4 is a program that you can target to your browser (it records all requests and responses from that browser). You can see what information is sent to which URL and what the responses are. From there it is not difficult to write a scrape to mimic those requests.

I can not stress enough though you need to have permission to do this kind of thing or you could get in trouble so be very careful.
0
Scott Fell, EE MVEDeveloper & EE ModeratorCommented:
>There are airline hotel websites that download and sort results from other websites
This is done through API's and probably marketing agreements.  Not from scraping.

>gofundme.com homepage only shows successful campaigns
>I would like to see unsuccessful campaigns where donor only has $1

Please look for an api and documentation.  When you can find that, report back here with the information for us to help you.
1

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
rgb192Author Commented:
Is this your account you want to scrape or anyones account?

All of gofundme.com
not just one user


Please look for an api and documentation.
If I find and contact gofundme.com support
regular person will not know what 'api' is
0
rgb192Author Commented:
thanks for information about gathering website words
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
PHP

From novice to tech pro — start learning today.