Solved

Need PERL or BASH script the can convert \[cr][lf] to [lf]

Posted on 2014-01-14
11
557 Views
Last Modified: 2014-01-18
Hi Team,

   Just need some assistance for a query that I have in the PHP section.   -->

http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28336410.html

I just want a way to do a global search and replace of every occurence of \[cr][lf] in a text file (i.e. the backslash character + carrier return character + line feed character) and convert it to [lf]   (just the linefeed character by itself).  

I'm sure this is just a one liner in perl...just can't get my head around this.

Thanks for all the help.
0
Comment
Question by:rleyba828
11 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
Comment Utility
tr -d '\r' < inputfile > outputfile
0
 
LVL 84

Accepted Solution

by:
ozo earned 400 total points
Comment Utility
perl -i -pe 's/\\\r\n/\n/' file
0
 
LVL 48

Expert Comment

by:Tintin
Comment Utility
Depending on your system

dos2unix file
0
 
LVL 11

Assisted Solution

by:tel2
tel2 earned 60 total points
Comment Utility
Hi rleyba828,

woolmilkporc's:
    tr -d '\r' < inputfile > outputfile
Will remove all CR chars, regardless of context, and will not remove the leading "\"s.

Tintin's:
    dos2unix file
will also not remove the leading "\"s.

ozo's:
    perl -i -pe 's/\\\r\n/\n/' file
should do what you have asked for.

Here's an alternative:
    sed -i 's/\\\r$//' file
which is slightly more concise, and on my system is more efficient...in case that matters.
0
 
LVL 34

Assisted Solution

by:Duncan Roe
Duncan Roe earned 40 total points
Comment Utility
I just tested ozo's solution and it does exactly what you asked for.
But ... you are left with a file which has a mixture of lines ending CrLf and ending Lf. Is this really what you want?
13:59:09$ cat -A t5
aaa^M$
bbb\^M$
ccc^M$
ddd\^M$
eee^M$
13:59:14$ perl -i -pe 's/\\\r\n/\n/' t5
13:59:28$ cat -A t5
aaa^M$
bbb$
ccc^M$
ddd$
eee^M$
13:59:30$ 

Open in new window

The above assumes all lines end CrLf now. If you've already fixed all lines except those ending backslash, ozo is 100% on the money.
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 11

Expert Comment

by:tel2
Comment Utility
> But ... you are left with a file which has a mixture of lines ending CrLf and ending Lf. Is this really what you want?

True, but only if the input data contains a mixture of
    \[CR][LF]
and just
    [CR][LF]
as your sample has, Duncan.

What's the situation, rleyba828?
0
 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
Yes I assumed it originated from a DOS / Windows system so all lines would end CrLf
0
 
LVL 11

Expert Comment

by:tel2
Comment Utility
I expect they will, Duncan.  I thought our question is, whether the CrLf will be preceeded by a '\', as only some of your sample input data lines are.
0
 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
@tel2 - your sed script does the same as ozo's perl script. They both do exactly what was asked for. It just seemed slightly odd to me to do a selective dos2unix  - but I was certainly assuming this was a regular Windows text file.
0
 

Author Comment

by:rleyba828
Comment Utility
Hi guys....the response of ozo worked best for me.    

To answer Duncan's question,   the mix of [cr][lf] and [lf] characters is how phpmyadmin exports a mysql table where fields in the text contain line returns.   It is able to export properly such a file to a csv file which will load properly in excel.  I found that simply doing a table dump from the mysql command line doesn't accomplish this....not sure why.
0
 
LVL 11

Expert Comment

by:tel2
Comment Utility
Hi rleyba828,

Glad you have a solution, but when you say the response of ozo worked best for you, did you try my sed solution?  It should do exactly the same as ozo's Perl solution, but faster, as I indicated.  You're running UNIX/Linux, right?
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

Over the years I've spent many an hour playing on hardened, DMZ'd servers, with only a sub-set of the usual GNU toy's to keep me company; frequently I've needed to save and send log or data extracts from these server back to my PC, or to others, and…
Email validation in proper way is  very important validation required in any web pages. This code is self explainable except that Regular Expression which I used for pattern matching. I originally published as a thread on my website : http://www…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

8 Experts available now in Live!

Get 1:1 Help Now