I need to access a complex website (Javascript/Frames) from Java

Posted on 2004-11-17
Last Modified: 2010-03-31
I'm writing a testing tool that needs to POST some data into a complex website. Then based on the data posted it will take care of the rest. Results will be delivered back out to me when processing is done (via an external servlet call).

The problem is that the website uses FRAMESET's and has a ton of javascript (none of which I care about). When I access the website with javascript disbled, I get nothing. Apparently the website interacts with each of these frames to pass information to/form the main servlet. At any rate, the website belongs to our partner and I don't have the source code or enough knowledge to bypass that stuff, so I'm assuming I'll have to interact with it through a full web browser like Mozilla, IE, etc instead of using Java's URL like I had planned on.

This tool is to be run on a Solaris server (with no graphics card) and so anything I do needs to be aware of that.

My thoughts so far:
1. Call Mozilla from the commandline with an html file which will post the data. Problems: I would need to do this hundreds of times (hopefully in parallel), I don't know how to reuse a browser that is currently available (from unix), it needs a DISPLAY (X-Windows) and I'm worried about the performance and reliability of this solution.
2. Maybe there's an API out there that can handle framesets and javascript that can be used similarily to URL? (this should be faster and easier to use)
3. Something else I can't think of.

I'd appreciate any API's, code snippets or ideas that would help with this problem.

Question by:forloop
    LVL 7

    Accepted Solution

    Ok, that is a huge task. Hopefully there is ample amounts of time planned for it.  There are many tools/libraries that will help you.  Take a look here:
    I would take the time to evaluate a couple of tools, create some test scripts. Evaluate on ease of use, functions covered and capabilities.  It is best to consider such an endevour as a project in itself.  

    Identify your specific test cases needed.  something like, When 'x' is posted I should receive 'y', and so on.   Some tools allow you to listen to the port connected to the server and will save the requests and replys.  It is hard to tell what you need and which tool would be best, which is another reason to take the time and evaluate a couple of tools to see which works best for you.  

    The javascript itself I would not worry about too much.  Simply concentrate on the messaging that occurs.  There are tools to work with javascript, but if you are not testing the functionality of the script then it is irrelevant.  

    That should get you started with your White-box testing of the web application.  
    LVL 35

    Assisted Solution

    I do not see what difference will make if you use Mozilla (or any other browser) and if you use the URL connection Java class. Essentially a browser is a GUI to an HTTP connection. The HTTP concept should be *exactly* the same regardless of where you initiate it, be it the browser, a command line GET request, a java HttpURL class request or somethign else.

    By loading up a browser and sending a request it is essentially the same as sending a request from Java. You do not care if there is complex JavaScript or several frames on the client, all you care about is to send the http request. What the web server will do on the other end it is not your concern.

    Author Comment

    girions: I don't care of about the javascript, but it directs the servlet to start processing the data. If I knew exactly what the javascript sends to servlet I could immitate that. But I don't and there's thousands of lines of javascript.

    Author Comment

    tomboshell: I believe you thought I needed to create or get an entire testing suite. The testing tool has already been written. It's hooked up inside this website. If I can get a tool that process a bit of javascript, I'm good.

    This is a tough problem mainly becuase I need something no one really cares about: a full web browser with no gui, preferable in the form of a Java api.

    LVL 35

    Expert Comment

    >  girions: I don't care of about the javascript, but it directs the servlet to start processing the data.

    Where? And does it also send parameters with the redirection? YOu can just fidnm the url it redirects to and the parameters it sends and then simulate the call.
    LVL 15

    Expert Comment

    Can yuou give us an url of relevant pages in that partnersite. Out of thoes "tons of Javascript" most likely only a few lines are relvant to you problem. We can find and explain those lines.

    PS: This appear to be a JavaScript question more than a Java question.

    regards JakobA

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    How to run any project with ease

    Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
    - Combine task lists, docs, spreadsheets, and chat in one
    - View and edit from mobile/offline
    - Cut down on emails

    Suggested Solutions

    Title # Comments Views Activity
    tenRun challenge 28 67
    bigHeights  challenge 13 43
    Receive file in Servlet 1 26
    countPairs challenge 7 36
    Go is an acronym of golang, is a programming language developed Google in 2007. Go is a new language that is mostly in the C family, with significant input from Pascal/Modula/Oberon family. Hence Go arisen as low-level language with fast compilation…
    Java functions are among the best things for programmers to work with as Java sites can be very easy to read and prepare. Java especially simplifies many processes in the coding industry as it helps integrate many forms of technology and different d…
    The viewer will learn how to implement Singleton Design Pattern in Java.
    This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.

    760 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    9 Experts available now in Live!

    Get 1:1 Help Now