Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Classpath problems with IBM App Srv: Jython & ORO conflicts

Posted on 2007-12-04
21
Medium Priority
?
472 Views
Last Modified: 2013-12-29
Hi Experts,

this seems to be a classic Java/IWebphere problem, considering the amount of Google hits I get about classpath issues with IBM Websphere and Jython. Main story that precisely describes the problem I have: http://www.ibm.com/developerworks/websphere/techjournal/0406_brown/0406_brown.html (the suggestions on IBM's website do not suffice, unfortunately.)

The issue: I need to use a newer ORO library from Apache Commons, namely Jakarta Oro 2.0.8. However, IBM comes bundled with a ton of libraries and preferably it loads these libraries first (PARENT_FIRST). This means that if you have any newer jar file on the lib path of your application (a WAR in my situation) you can be certain it will never be loaded.

According to the aforementioned link, Java will never "see" my newer version. To change that I should set the classloader behavior to "PARENT_LAST" (either on server, app or module level). Unfortunately this raises a ton of new issues which seem hard to solve (many incompatibilities).

I also tried adding it to the MANIFEST.MF, but that wouldn't work because IBM first loads its own classes and then the ones in the MANIFEST, which in turn must be part of the EAR.

When I renamed jython.jar to jython.old, my NoSuchMethodError went away. The reason that my development environment (Rational Rose XDE) does not complain is because it uses its own classpath for the application, hence the error only appears after deployment.

I do not have access to the deployment servers and thus I cannot disable jython.jar by hand. It has to be done using the server interface.

The question is simple, but possibly hard (?) to answer: how can I force the loading of the new ORO lib or how can I force Websphere to stop loading jython.jar. The means I have are:

 1. change the configuration of the WAR or EAR
 2. change the configuration of the server settings
 3. programming (I have full access to applicable sources)

Anybody any idea? Any help's appreciated,

Cheers,
-- Abel --
0
Comment
Question by:abel
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 11
  • 7
  • 3
21 Comments
 
LVL 15

Accepted Solution

by:
bpmurray earned 1000 total points
ID: 20414882
First, PARENT_LAST is important, but has to be both on the WAR *AND* the EAR. My guess is that you've only done this in one place.

Next, make sure that no spurious jars have been added to your project, i.e. that'll be sucked into your deployed app to override what you expect.

That should do the trick.
0
 
LVL 39

Author Comment

by:abel
ID: 20419430
Well, that is precisely what has happened, these spurious jars. They come from the ancient IBM WSAS 5.1. I've tried your suggestion and some other combinations too, but none seem to help. It's not like a couple of errors that I could readily solve, but a whole lot of them, from different libraries, seem to be unresolvable when I change it.

It's probably because of the way the system has been build. It doesn't really help either that the EAR hosts several projects that are interdependent (the legacy of indoor development...).

I did find one workaround, though it feels like cracking a walnut using a sledgehammer: using my own classloader that loops through the jars in the WEB-INF/lib directory. Because of the conflicts I cannot use the objects directly, so I added reflection and introspection (using java reflect api). And as it turns out, it works like a charm. Luckily I didn't need it in a performance sensitive part.

Cheers,
-- Abel --
0
 
LVL 39

Author Comment

by:abel
ID: 20419458
Hmm, when I think about it, with .NET or plain old (not so good) COM+ it was possible to use the version information in the dll to call the correct one (or when not using version information it called the latest version). Isn't there a mechanism somehow with Java that does something similar? That would give the possibility to mix and match older libs that are dependent on other older libs, together with more recent libs that you need to use yourself.

(example, again: Jython is dependent on an old ORO, and you want the new ORO *and* Jython).
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
LVL 41

Assisted Solution

by:HonorGod
HonorGod earned 1000 total points
ID: 20463645
Since you have iterated through the jar files in the WEB-INF/lib directory, you should now know which needs to be referenced.

Can you keep this information across invocations in order to remove the lookup penalty?

By the way, interesting solution to the problem.
0
 
LVL 39

Author Comment

by:abel
ID: 20463864
Well, yes, I know which specific library needs to be referenced, I just generalized the solution. The lookup is done once (singleton) and a reference to the needed classes is then kept in memory.

It feels a bit as a large way around to get a specific class from a specific jar file. Version conflicts can happen more often (in situations you are stuck with a default set of libraries and you cannot touch this default set, however, you need the new versions of these libs), why is it so hard to choose a specific lib, or, put differently, a lib with a specific version in the Manifest file. I mean the information is all there, it shouldn't be too hard should it?
0
 
LVL 41

Assisted Solution

by:HonorGod
HonorGod earned 1000 total points
ID: 20463879
That is certainly a reasonable question.
However, another question, from a different perspective might be:

Why do we have to have multiple versions of the "same" jar file?
Wouldn't it be better if newer versions of a jar file provided backward compatability?
0
 
LVL 39

Author Comment

by:abel
ID: 20464063
Yes, of course. And they do. This one too. However, I needed added functionality (new methods in the new version). And the classpath system of IBM WSAS was "bottom up", i.e., it first looks at Java, then at IBM's classpath and lastly at you own added libs. Where the default should be the other way around.

You don't add libs for them to not be used or found.

So, actually what we are talking of here is forward compatibility. IBM finds the old jar first and sees it as the "right" jar (however in my dev IDE, the normal way of "most specific first" was followed and it finds the added libs correctly, but at runtime, the IBM server takes over and uses this backward method), instead of the new one. And forward compatibility is, by its nature, limited.
0
 
LVL 41

Assisted Solution

by:HonorGod
HonorGod earned 1000 total points
ID: 20466038
Is your code for doing the lookup,  reflection, and introspection something that you would be willing to share here?
0
 
LVL 15

Assisted Solution

by:bpmurray
bpmurray earned 1000 total points
ID: 20467235
This is exactly why the PARENT_LAST/PARENT_FIRST settings are available, but they must be set all over. If you have only set this in one part of your app, it just has no effect.
0
 
LVL 41

Assisted Solution

by:HonorGod
HonorGod earned 1000 total points
ID: 20467516
Well, that's not completely accurate.
The place where the setting exists identifies the scope of the setting.
In general, you probably want to make sure to set the classloader at
the WAR level, so that the appropriate setting is used.

There can be instances of the application classloader setting causing
problems.  For example, if you have static methods in the EAR, and
these get invoked by the application, that can cause the Application
classloader setting to take precidence, and keep you application from
thereafter accessing resources in the WAR.  Unfortunately, the
classloader stuff is sometimes counter intuitive. :-(
0
 
LVL 39

Author Comment

by:abel
ID: 20470477
> If you have only set this in one part of your app, it just has no effect.

See my earlier comments: it costs too many other problems. I assume that when the system was designed like this from the ground up, there weren't any problems at all. But this is an age old system (hence Struts 1.0.2, WSAS 5.1 with an old Jython and Java 1.4 etc) which has been build and changed over the last years.

Though I do think that changing this order should not really cause "class not found" errors, as the only location on the classpath is further away, and nearer libraries are newer and shouldn't give "class not found" errors. Hmm, maybe I should look into that solution again and try to analyse some of the errors. However, the classes not found were hundredths, which is why I didn't bother to look further and I changed it back to "normal" (why IBM thinks this is normal is beyond me).
0
 
LVL 39

Author Comment

by:abel
ID: 20470487
> Unfortunately, the classloader stuff is sometimes counter intuitive. :-(

that's an understatement. I've programmed in Java for years, and Java always sticked with me easily, even if I stopped programming for some time. But classpath issues had the tendency to always come up and once there, were hard to solve.

Yes, I think I can share that code. However, it is quite straightforward. I'm not in the office today, so bear with me a while.
0
 
LVL 15

Assisted Solution

by:bpmurray
bpmurray earned 1000 total points
ID: 20470637
The idea behind the PARENT_LAST solution is that it means that the jars in your app are used first, and these tend to be the ones you want. That's why I originally suggested to watch out for other jars in your project that you actually don't want. If your app works in WSAD/RAD and it fails in WAS, it should revert to the dev behaviour by setting this, but you have to set it everywhere!
0
 
LVL 41

Assisted Solution

by:HonorGod
HonorGod earned 1000 total points
ID: 20471312
By setting it everywhere you are removing any question about the classloader strategy to be used.  Actually, it really only needs to be set in "the right place".  Unfortunately, understanding which places are "necessary and sufficient" is the challenge.  :-(
0
 
LVL 39

Author Comment

by:abel
ID: 20471473
I haven't been clear about my dev env, it isn't WSAD, but XDE from Rational Rose, which is built on top of Eclipse 2.1. To add to the confusion, the dev env uses Java 1.3 and WSAS (WAS?) uses Java 1.4. But that's something I can configure differently.

I think that I am quite aware of the different possibilities of PARTEN_LAST by now. These several options are there for a reason of course. To find out what the culprit is, I reckon I'll have to create a small project and let it use this new library and see how it goes.

If it were only for ORO I probably go with just the workaround solution. But we are planning to moving from Struts 1.0.2 to Struts 1.3.8 (no, not 2.0) and likely we will hit some lib/classpath problems there too. Anyway, I will try a couple of scenario's and suggestions of you two to see how it goes (basically, the suggestion of "everywhere" didn't work out-of-the box and the suggestion of "only war level" didn't either, but I'll see what I find now that I know more of the process).
0
 
LVL 39

Author Comment

by:abel
ID: 20632735
My apologies for not reacting sooner (well, I did react, but I didn't follow up either). I have been moved to another assignment. The solution to this particular problem was and is using my own ClassLoader.

I still don't understand why the class loading strategy of IBM is the reverse of any other class loading strategy I have ever seen (IBM puts most specific, i.e., your own classes, by default at the end of the path), I can't seem to find any use case for that though I'm sure IBM has thought it over (well, there must be someone who found this more useful than other strategies).

Anyway, what shall we do about the points? I can share them, of course, but in the end, I feel that I have actually answered my own question, but I don't know whether I can accept my own question (actually, I think I can't).

Cheers,
-- Abel --
0
 
LVL 41

Expert Comment

by:HonorGod
ID: 20635565
The decision to accept one or more answers, or split the points is entirely up to you.
If you believe that none of the responses warrant acceptance, then you can open a separate question in the General Community Support area, and request a refund.
0
 
LVL 39

Author Comment

by:abel
ID: 20732360
Well, doing this pragmatically, I'll split the points equally to everybody and I'll post a note about the solution.
0
 
LVL 39

Author Closing Comment

by:abel
ID: 31425236
Excellent for the helpful pointers. Though they didn't result in a complete solution, they did help me understand the classpath problems, solutions and strategies available from IBM WS.

The final solution to this particular situation (i.e., changing the load order is not an option as a result of other existing dependencies) is to add your own classpath resolver *and* dynamically load the libs + classes. This has the disadvantage of having no compile time checks anymore, so good error handling, and solid classpath searching (i.e., recursively searching from the WEB-INF dir in my case instead of from a fixed or programmable location) is vital.

After you have a working prototype, encapsulate it using for instance the Facade pattern (depending on your intents, a decorator, or even a proxy may be more appropriate instead).
0
 
LVL 39

Author Comment

by:abel
ID: 20732470
hmm, interesting to find that the grading comments show up on all places where I graded... ;)

thanks people for helping on this rather complex issue!
0
 
LVL 41

Expert Comment

by:HonorGod
ID: 20733303
interesting indeed.

Thanks for the points.  Good luck
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction This article is the second of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers the basic installation and configuration of the test automation tools used by…
In this post we will learn how to connect and configure Android Device (Smartphone etc.) with Android Studio. After that we will run a simple Hello World Program.
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.
Suggested Courses

609 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question