Avatar of Ciprian Niculescu
Ciprian Niculescu
Flag for Germany asked on

what programing language is best for lots of read/writes of small files? do it count?

Hello,
i need to process lots of small text files (1k to ~1M in size) millions per day.
they are syslogs files. the processing logic can be summarized in: filter out unwanted lines.
i did some tests with: cat File | grep -v bla | grep -v foo | grep -v bar > File.2
and i get huge space and clearness benefits. But the piping have some limits, after 20 not good any more :)
i'm not a day to day programmer, but kept playing around over the years.
i know/remember C,C++,Perl, some basic bash, started to learn Python some years back. i see that the fancy thing now is Go :)
do any of the language would offer a performance plus or minus, in this scenario? or the limiting factor for all will be the disk access?
Thank you for your thoughts

Ciprian
PythonPerlGolangUnix OS

Avatar of undefined
Last Comment
Suhas .

8/22/2022 - Mon
ASKER CERTIFIED SOLUTION
arnold

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
wilcoxon

I agree with arnold.  If you want to do text processing, use perl.
SOLUTION
David Favor

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
Suhas .

No comment has been added to this question in more than 21 days, so it is now classified as abandoned.

I have recommended this question be closed as follows:

Split:
-- arnold (https:#a42374520)
-- David Favor (https:#a42374692)


If you feel this question should be closed differently, post an objection and the moderators will review all objections and close it as they feel fit. If no one objects, this question will be closed automatically the way described above.

suhasbharadwaj
Experts-Exchange Cleanup Volunteer
Your help has saved me hundreds of hours of internet surfing.
fblack61