CUT command is only processing the first line

I am trying to cut the first 10 columns of a csv file that has about 30 million lines.

My command is:
cut -d "," -f 11- data.txt>cutdata.txt

Open in new window


It only processes the first line. It runs  for a while, and it does write out the new file, but with only the first line cut properly.

If I take only the first 100 or so lines of the data.txt and use the same cut command, it works perfectly.

Can this be done with sed or awk? If so, can you provide the exact syntax?

Thanks.
jasgotAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

jmcgOwnerCommented:
My first impression is that your original file has line endings different from what the cut command expects, while your extract  differs in this respect so the cut command works on it just fine.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
woolmilkporcCommented:
This is awk:

awk  -F, '{for(n=11;n<NF;n++) {printf "%s,",$n} print $NF}' data.txt > cutdata.txt

No idea why "cut" shouldn't work, though. Maybe it's a temp space or buffer handling issue.
Does the concerned filesystem have enough space to hold the output file?
ozoCommented:
Can you try other lengths between 100 and 30 million lines to find exactly where it starts to fail?
Does it make any difference if you do
cut -d "," -f 11- <data.txt>cutdata.txt
jasgotAuthor Commented:
Thank you. I converted all of the LF to CRLF and it worked a treat!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Unix OS

From novice to tech pro — start learning today.