Link to home
Create AccountLog in
Avatar of Ciprian Niculescu
Ciprian NiculescuFlag for Germany

asked on

what programing language is best for lots of read/writes of small files? do it count?

Hello,
i need to process lots of small text files (1k to ~1M in size) millions per day.
they are syslogs files. the processing logic can be summarized in: filter out unwanted lines.
i did some tests with: cat File | grep -v bla | grep -v foo | grep -v bar > File.2
and i get huge space and clearness benefits. But the piping have some limits, after 20 not good any more :)
i'm not a day to day programmer, but kept playing around over the years.
i know/remember C,C++,Perl, some basic bash, started to learn Python some years back. i see that the fancy thing now is Go :)
do any of the language would offer a performance plus or minus, in this scenario? or the limiting factor for all will be the disk access?
Thank you for your thoughts

Ciprian
ASKER CERTIFIED SOLUTION
Avatar of arnold
arnold
Flag of United States of America image

Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
I agree with arnold.  If you want to do text processing, use perl.
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
No comment has been added to this question in more than 21 days, so it is now classified as abandoned.

I have recommended this question be closed as follows:

Split:
-- arnold (https:#a42374520)
-- David Favor (https:#a42374692)


If you feel this question should be closed differently, post an objection and the moderators will review all objections and close it as they feel fit. If no one objects, this question will be closed automatically the way described above.

suhasbharadwaj
Experts-Exchange Cleanup Volunteer