Avatar of nepaluz
nepaluz
Flag for United Kingdom of Great Britain and Northern Ireland asked on

Regex and extract information from webpage (scraping)

Hi,
I am trying to scrape the ISIN Number from yahoo finance pages and need some help.

I have built the url that I need but can not figure out how to scrape this information. I have attached a text file (which essentially is the web page) and here is the link to the page.

http://uk.finance.yahoo.com/q?s=prty&m=L&d=
Visual Basic.NET

Avatar of undefined
Last Comment
smash_pants

8/22/2022 - Mon
ErezMor

download the html page, then
use something like ReadAllText function to read the html page's source, then
use a InStr function to find "ISIN" - the value your after is just infront of this location in the text string,
and is ended by a closing paretheses - which you may easily find too

hope this makes sense to you, quite some work is yet to be done...good luck
smash_pants

you could use xpath to access the text for the ISIN:

/html/body/div/div[2]/div[4]/div[3]/div/div/span

If you really need a regex for it then here you go:
^.*ISIN (.{12}) \).*$
nepaluz

ASKER
ErezMor thanks for the contrib. I will give it a bash andsee what I come up with.

smash_pants (what a name!)I really am interested in the regex but can not get it to work. Heres the code I am using (and don't laugh!)


Dim fxk = Regex.Matches(sr.ReadToEnd.ToString, "^.*ISIN (.{12}) \).*$")

Open in new window

Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy
nepaluz

ASKER
OK - I have been un-able to implement either way suggested here, and though they may be correct, I am un-able to award poits for them. I am thus closing this thread and will try another one.
ErezMor

since the last post from user was "i'll give it a try and get back to you...", i dont think his closing reason is justified.
i happen to have some actual experience in doing just what i suggested the user, so in terms of if it works or not, there's no doubt here.
unless i didnt understand his question, or his closing request reason...

Erez.
nepaluz

ASKER
OK then erez, I have been unable to use the instr as suggested (I think I have to repeat the obvious), could you post some code to this end?
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
smash_pants

I've tested this code and it works.
Sorry i haven't been around much... I'm moving house.

string result = null;
string url = "http://uk.finance.yahoo.com/q?s=prty&m=L&d=";
WebResponse response = null;
StreamReader reader = null;

try {
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    request.Method = "GET";
    response = request.GetResponse();
    reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8);
    result = reader.ReadToEnd();
}
catch (Exception ex) {
    // handle error
    Console.WriteLine(ex.Message);
}
finally {
    if (reader != null)
        reader.Close();
    if (response != null)
        response.Close();
}
foreach (Match txt in Regex.Matches(result, @"^.*ISIN (.{12}) \).*$", RegexOptions.Multiline)) {
    Console.WriteLine(txt.Groups[1]);
}

Open in new window

ASKER CERTIFIED SOLUTION
smash_pants

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
nepaluz

ASKER
Thanks a lot and happy moving! If I may ask for a tweak, is it possible to have the regex NOT limiting the result string to 12 characters as you have it, but pich up to the closing bracket?

The ISIN (could?) be longer than 12 characters long and it would thus return an incomplete numer (or I am wrong?)

I have awarded the marks anyhow.
smash_pants

in the regex:
^.*ISIN (.{12}) \).*$

just change the .{12} to what you need.

Capital letters an numbers at least 12 chars long:
[A-Z0-9]{12,}

any character ,any length:
.*


I started with Experts Exchange in 2004 and it's been a mainstay of my professional computing life since. It helped me launch a career as a programmer / Oracle data analyst
William Peck