Solved

What language is the faster for large file processing?

Posted on 2006-04-08
17
269 Views
Last Modified: 2012-05-05
What programming language is the fastest, least processor intensive language for processing large files.  An example would be Web Logs.


Thanks!

Randy
0
Comment
Question by:rjohnsonjr
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
  • +6
17 Comments
 
LVL 84

Accepted Solution

by:
ozo earned 66 total points
ID: 16409917
It would depend a lot on what you write in that language, but the assembly language for the processor you are running would allow you to write the fastest, least processor intensive code.
0
 
LVL 24

Assisted Solution

by:fridom
fridom earned 62 total points
ID: 16410306
Languages optimized for such kind of task. ozo is of course right, however the following languages should be near-by
- C
- Ocaml
- Perl (optimized for string manipulatons)

Regards
Friedrich
0
 
LVL 1

Assisted Solution

by:anteusz
anteusz earned 62 total points
ID: 16410855
Of course, if you take the time to write the program into account, assembly language may not
 be the best if you cannot learn it fast enough...
What languages do you know already?
Maybe selecting a language that is close to the one you already know, can give you higher productivity.
0
Optimize your web performance

What's in the eBook?
- Full list of reasons for poor performance
- Ultimate measures to speed things up
- Primary web monitoring types
- KPIs you should be monitoring in order to increase your ROI

 
LVL 3

Assisted Solution

by:marco_delpercio
marco_delpercio earned 62 total points
ID: 16412028
Hi

I suggest Perl. It's easy to learn instead of C/C++ and it's maybe the best text processing language, well on the other hand you'll have to manage regular expressions wich are not so easy but even with simple regular expressions you'll save much much development time for file parsing. Moreover Perl is cross-platform and you will easily find a perl interpreter on most web servers.

Marco Del Percio

0
 
LVL 6

Author Comment

by:rjohnsonjr
ID: 16413660
I know some c++, some java, some perl, php, coldfusion, cobol, not enough of assembly language to write any program in it.  I was thinking c or Perl.  I have never heard of Ocaml
0
 
LVL 24

Expert Comment

by:fridom
ID: 16414946
Ocaml is a multi paradigmn language (with static types)
It a functionalk and object oriented language in one go (others in that class are:
Common Lisp, Mozart/Oz)

Ocamls homepage is at:
http://caml.inria.fr/

You can see how well it scales regarding speed at:
http://shootout.alioth.debian.org/

It's one of those totalyl underestimated fine languages, which people simple do not even try. It's a pitty really.

Regards
Friedrich
0
 
LVL 35

Assisted Solution

by:TimYates
TimYates earned 62 total points
ID: 16415200
I process massive files in Java...

The speed is more than likely going to depend on how you do it, rather than what language you use...

Tim
0
 
LVL 1

Expert Comment

by:anteusz
ID: 16415433
Actually, C/C++ is one of the fastest languages after assembly, I think.
So you can use it if you are comfortable with it.
On the other hand, it is more harder to maintain C/C++ programs as they have to be compiled.
Perl is almost available everywhere and it can be maintained faster.
What is more important to you, speed or maintainanability?

The question for me is what an acceptable running time for you and how complicated the processing is.
0
 
LVL 6

Author Comment

by:rjohnsonjr
ID: 16416714
The main thing would be a program that is not processor intensive, followed by speed.  Maintainability is not really an issue as I would not have to change it much.

-Randy
0
 
LVL 35

Expert Comment

by:TimYates
ID: 16416823
>> Maintainability is not really an issue as I would not have to change it much.

Those are famous last words ;-)

Tim
0
 
LVL 6

Author Comment

by:rjohnsonjr
ID: 16416884
lol yep,  that's for sure,  
0
 
LVL 11

Assisted Solution

by:cup
cup earned 62 total points
ID: 16419070
I don't know what Cobol compilers are like nowadays.  In the early 80s they were the compilers which generated the best code.  They were geared towards processing large batches of data and most of them are quite well optimized for that.  However, I haven't used Cobol in anger since the early 80s and certainly not on the same machine as C/C++ so I can't compare the two.
0
 
LVL 35

Expert Comment

by:TimYates
ID: 16424312
I think these days, all main languages are pretty much as fast as each other.  Especially for something as "run of the mill" as file processing...

Tim
0
 
LVL 1

Assisted Solution

by:i_mahesh
i_mahesh earned 62 total points
ID: 16460700
Perl is quite optimized for this purpose.
0
 
LVL 8

Assisted Solution

by:Leo Eikelman
Leo Eikelman earned 62 total points
ID: 16478908
If you are talking about LARGE files then you should use a tree structure to parse threw the file.

You can do this using ANSI C.

look at this link for code examples:

http://www.devincook.com/goldparser/engine/c.htm

cheers,

Leo
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

There is an easy way, in .NET, to centralize the treatment of all unexpected errors. First of all, instead of launching the application directly in a Form, you need first to write a Sub called Main, in a module. Then, set the Startup Object to th…
If you haven’t already, I encourage you to read the first article (http://www.experts-exchange.com/articles/18680/An-Introduction-to-R-Programming-and-R-Studio.html) in my series to gain a basic foundation of R and R Studio.  You will also find the …
The viewer will learn how to implement Singleton Design Pattern in Java.
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.

624 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question