We help IT Professionals succeed at work.

Perl - read line

timbrigham
timbrigham asked
on
I'm reading the contents of a file line by line. Each line is field separated with | characters. The file is coming from Nessus, if it matters.
The odd thing is that some of the time I'm getting part of the input line printed at a random position as shown below. My gut feeling is this only occurs for long lines but I'm not sure yet. I'm running Perl version 5.8.8. Please assist.

    while( $raw = readline(PIPEIN) )
    {
        chomp($raw);
        my @todo;
        @todo=split(/\|/,$raw);
        print Dumper( @todo );
        next;
        ...

Open in new window

Output:
$VAR1 = 'results';
$VAR2 = '100.100.100.100';
$VAR3 = '100.100.100.30';
$VAR4 = 'snmp (161/udp)';
$VAR5 = '35296';
$VAR6 = 'Security Note';
t use it, or\\nfilter incoming UDP packets going to this port.\\n\\nRisk factor :\\n\\nNone\\n\\nPlugin output :\\n\\nNessus has negotiated SNMP communications at SNMPv2c.\\n\\n';

Open in new window


Sample line of input
results|100.100.100|100.100.100.30|snmp (161/udp)|35296|Security Note|\nSynopsis :\n\nThis plugin reports the protocol version negotiated with the remote\nSNMP agent.\n\nDescription :\n\nBy sending an SNMP 'get-next-request', it is possible to determine the\nprotocol version of the remote SNMP agent.\n\nSee also :\n\nhttp://en.wikipedia.org/wiki/Simple_Network_Management_Protocol\n\nSolution :\n\nDisable the SNMP service on the remote host if you do not use it, or\nfilter incoming UDP packets going to this port.\n\nRisk factor :\n\nNone\n\nPlugin output :\n\nNessus has negotiated SNMP communications at SNMPv2c.\n\n

Open in new window

Comment
Watch Question

Commented:
Could be just your terminal having troubles with long lines (doesn't linebreak)? In the upper example, you have printed exactly the rightmost part of the string.
Try redirecting to file or print to file and check that the last element  ($todo[5]) is ok. Perhaps print also the length of the string to make sure. My guess is that the code parses ok, just the printout is garbeled.

Commented:
readline() will read up to the \n
the sample input has a \n in the middle of the line.

                                                                                                                                         
results|100.100.100|100.100.100.30|snmp (161/udp)|35296|Security Note|\nSynopsis :\n\n <<<< the \n in question

Open in new window

Distinguished Expert 2018
Commented:
I'm with boocko. I noticed that this didn't appear random as well, but was the right-most part of the line. I also noticed that your previous lines printed a variable name before each string "$VAR1" ...etc, which is, of course, how Data::Dumper formats arrays when fed procedurally. Line 7, however doesn't show that, and I've never seen Data::Dumper exhibit any such bug, which also has me leaning towards a terminal/display issue and not a data issue.

Author

Commented:
I'm not expecting a line break. The file I'm using is output from Nessus, and each element (some of which can be 10kilobytes long) is on a separate line. It's not the most opportunistic but it's what I've got.  
I tried redirecting to file (at the BASH level - did you mean within Perl itself?) and the same error occurs.


I've looked at some more lines which are having problems - they all appear to be printing only the end of the line, and only for extremely long lines.

Author

Commented:
Edit - the same thing occurs if I do the redirection from within Perl itself.

Author

Commented:
I'm in agreement regarding the part of the line being displayed - I checked a few more lines and they are all exhibiting the same behavior. The extraneous data is also being displayed only for the last variable in each data set (with or without the use of data::dumper).
I've tried this using Putty to SSH to the server, as well as executing from the server's local console. Any suggestions?
Commented:
You printed out exactly 180 characters in the long line. Let's see the lengths of the long fields. I suggest a code like the one below. Do you get numbers larger than 180? If yes, then the problem might be in the printout.

while( $raw = readline(PIPEIN) )
    {
        chomp($raw);
        my @todo;
        @todo=split(/\|/,$raw);

my $len = length($todo[5]);
if ($len >= 180) { print "Long field detected: length = $len\n"}

        next;
        ...
Commented:
You can try wrapping print using the
use Text::Wrap

and then

print wrap("","", $todo[5]);

e.g.
if ($len >= 180) { print "Long field detected: length = $len\n"; print wrap("","", $todo[5]);}

Author

Commented:
I came up with something similar by printing the lengths of all my lines as they came up. Oddly enough the lines display properly if I pull things up with emacs, but Perl cat and less all truncate as described.

Trying the script shown doesn't work so well; the specific field in question changes (in the range of 4 to 7, usually).
I tried running with
my $len = length($raw);
if ($len >= 180) { print "Long field detected: length = $len\n"}
This accurately shows which lines I'm having problems with.

Author

Commented:
A little further monkeying with Text::Wrap has corrected all my problems.
Thanks gentlemen.