[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now


using webrequest to get file list from website

Posted on 2014-08-18
Medium Priority
Last Modified: 2014-08-18
Hi all,

I have a dynamically generated site whereby I want to provide a list of fonts (.ttf files) that are present in a public folder on a common resource site i have setup.

for example i have user1.myurl.co.uk which will host the generated site. I then want to use AJAX to contact resources.myurl.co.uk/fonts (this will store all common resources for these generated sites).

now i have to following code;

WebRequest request = WebRequest.Create(fonts_path);
        WebResponse response = request.GetResponse();
        Regex regex = new Regex("<a href=\"?<url>.*\">(?<name>.*)</a>");
        List<KeyValuePair<String, String>> files = new List<KeyValuePair<String, String>>();

        using (var reader = new StreamReader(response.GetResponseStream()))
            string result = reader.ReadToEnd();

            MatchCollection matches = regex.Matches(result);
            if (matches.Count == 0)
                Console.WriteLine("parse failed.");
                return null;

            foreach (Match match in matches)
                if (!match.Success) { continue; }

                files.Add(new KeyValuePair<String, String>(match.Groups["url"].ToString(),match.Groups["name"].ToString()));

        return files;

Open in new window

now the url is giving a 403 forbidden error on the webrequest. I have tried adding the .htaccess file to the root directory.

Can anyone please tell me what the issue coudl be herE?
Question by:flynny
  • 4
  • 2
LVL 37

Accepted Solution

Kimputer earned 2000 total points
ID: 40267795
Are you sure this is a programming issue you need help with (see your topics you entered), or is this a web server thing you need help with?
If you are sure about the url, use the url in the browser. It's NOT possible you use exactly the same URL and it's visible in the browser and gives a 403 error in the application (other than a server side check on cookies, headers and what not).

Use Wireshark to analyse some of the traffic (if you're sure URLs are correct), try to match your code as close as possible (headers, useragent, cookies etc)
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 40267835
The only way to get a list of files available in a directory on a web server would be to parse the HTML file that is generated for directories where directory browsing is enabled. Directory browsing is typically not enabled as a security precaution. I would suggest writing a small web-service-like endpoint that will browse the directory (on the server) and return a list of file names.

Author Comment

ID: 40267850
Hi all,

sorry yes I am getting the 403 in the browser as well.

the server is running IIS 7. so yes, it appears its probably more web server issues than programming.
Prepare for your VMware VCP6-DCV exam.

Josh Coen and Jason Langer have prepared the latest edition of VCP study guide. Both authors have been working in the IT field for more than a decade, and both hold VMware certifications. This 163-page guide covers all 10 of the exam blueprint sections.


Author Comment

ID: 40267872
sorry guys solved the issue.

IIS 7 allows me to switch on directory browser in the direcotry browsing snap in.

thanks for the help.

Author Closing Comment

ID: 40267875
not a solution but helped point me in the right direction.
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 40267877
Hmmmm... I thought I was the one who mentioned directory browsing. Alas...

Author Comment

ID: 40267943
sorry kaufmed i didnt see your post can i give some point to you somehow?

Featured Post

[Webinar] Cloud and Mobile-First Strategy

Maybe you’ve fully adopted the cloud since the beginning. Or maybe you started with on-prem resources but are pursuing a “cloud and mobile first” strategy. Getting to that end state has its challenges. Discover how to build out a 100% cloud and mobile IT strategy in this webinar.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Problem Hi all,    While many today have fast Internet connection, there are many still who do not, or are connecting through devices with a slower connect, so light web pages and fast load times are still popular.    If your ASP.NET page …
This article is for Object-Oriented Programming (OOP) beginners. An Interface contains declarations of events, indexers, methods and/or properties. Any class which implements the Interface should provide the concrete implementation for each Inter…
Screencast - Getting to Know the Pipeline
With just a little bit of  SQL and VBA, many doors open to cool things like synchronize a list box to display data relevant to other information on a form.  If you have never written code or looked at an SQL statement before, no problem! ...  give i…
Suggested Courses
Course of the Month20 days, 7 hours left to enroll

868 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question