Solved

Randomize

Posted on 2011-03-10
20
417 Views
Last Modified: 2012-06-21
Hello,

I'm looking for a command that will shuffle/randomize file contents to create all possible variants.

example:
school store watches

output
school store watches
watches store school
store watches school



Thank you
0
Comment
Question by:faithless1
  • 8
  • 5
  • 4
  • +2
20 Comments
 
LVL 10

Expert Comment

by:APNFSSC
ID: 35102455
#! /usr/bin/perl -w

use strict;

my @array = @ARGV ? @ARGV : qw/school store watches/;

sub fac {
   my $nr = shift;
   return 1 if $nr < 2;
   return $nr*fac($nr-1);
}

my %seen;
my $max = fac(scalar @array);
my $nr  = 0;
while( $nr < $max ) {
    my $need_combo = 1;
    my @combo;
    while( $need_combo ) {
        for( my $i = 0; $i < scalar @array; ++$i ) {
            $combo[$i] = $array[ rand scalar @array ];
        }
        my %unique;
        my $unique = 1;
        for my $c ( @combo ) {
            if( ++$unique{$c} > 1 ) {
                $unique = 0;
            }
        }
        if( $unique == 1 ) {
            $need_combo = 0;
        }
    }

    if( exists $seen{ "@combo" } ) {
        next;
    }
    else {
        local $" = " ";
        print "@combo\n";
    }
    ++$seen{"@combo"};
    ++$nr;
}

Open in new window


result is:

school watches store
school store watches
watches store school
store watches school
watches school store
store school watches

0
 
LVL 84

Expert Comment

by:ozo
ID: 35102476
see
perldoc -q shuffle

But I'm confused because your example does not create all possible variants.
it misses
store school watches
watches school store
and
school watches store
and if you really want all possible variants, then is sounds more like you want to use Algorithm::Permute
rather than to randomize/shuffle

0
 
LVL 31

Expert Comment

by:farzanj
ID: 35102485
Do you mean you want to create all permutations?

Like for

a b c

You will have
a b c
a c b
b a c
b c a
c a b
c b a
Right?
0
 
LVL 16

Accepted Solution

by:
sjklein42 earned 500 total points
ID: 35102635
permute.pl:


@words = @ARGV;

my $level = (-1);
my @Value;
my $N = @words; ## count


sub printIt
{
    my @out;
    for (my $i = 0; $i < $N; $i++)
    {
        push @out, $words[$Value[$i]-1] ;
    }

    print join(" ", @out) . "\n";
}

sub visit
{
    my $k = shift(@_);
    $level = $level+1;
    $Value[$k] = $level;

    if ($level == $N)
    {
        printIt();
    }
    else
    {
        for (my $i = 0; $i < $N; $i++)
        {
            if ($Value[$i] == 0) {  visit($i); }
        }
    }
    $level = $level-1;
    $Value[$k] = 0;
}


for (my $i = 0; $i < $N; $i++)
{
    $Value[$i] = 0;
}
visit(0);

Open in new window



c:\temp>perl permute.pl school store watches
school store watches
school watches store
store school watches
watches school store
store watches school
watches store school

c:\temp>

Open in new window

0
 
LVL 16

Assisted Solution

by:sjklein42
sjklein42 earned 500 total points
ID: 35102640
c:\temp>c:\$\lib\permute.pl how now brown cow
how now brown cow
how now cow brown
how brown now cow
how cow now brown
how brown cow now
how cow brown now
now how brown cow
now how cow brown
brown how now cow
cow how now brown
brown how cow now
cow how brown now
now brown how cow
now cow how brown
brown now how cow
cow now how brown
brown cow how now
cow brown how now
now brown cow how
now cow brown how
brown now cow how
cow now brown how
brown cow now how
cow brown now how

c:\temp>

Open in new window

0
 

Author Comment

by:faithless1
ID: 35103377
Thanks for all the responses. Going to test with what seems to be the easiest solution

Ozo -

I ran the perldoc -q shuffle and here's what I see:

use List::Util 'shuffle';

               @shuffled = shuffle(@list);

The command I then used is:

cat file.txt |  perl -pe List::Util

I also ran perldoc -q shuffle but can't seem to figure out what command to use.

Thanks

0
 

Author Comment

by:faithless1
ID: 35103416
sjklein42

How would I specify a file with keywords? Would I run the code this way?

perl -pe permute.pl file.txt

Thanks
0
 

Author Comment

by:faithless1
ID: 35103438
APNFSSC:

Can you please provide usage details for your script so I can test it as well?

Thank you
0
 
LVL 10

Expert Comment

by:APNFSSC
ID: 35103494
what will the contents of file.txt be?

Will it have multiple lines?

0
 
LVL 10

Expert Comment

by:APNFSSC
ID: 35103554
Here is an example as the code stands now. You just pass the words to that you want to the script and it does the rest. if you pass nothing then the words 5 of the script are used.

# ./test.pl the quick brown fox
quick brown the fox
fox the brown quick
fox brown quick the
quick brown fox the
fox the quick brown
quick fox brown the
the quick brown fox
brown quick fox the
quick the brown fox
quick the fox brown
the brown fox quick
brown fox the quick
fox quick the brown
the fox brown quick
the quick fox brown
fox brown the quick
fox quick brown the
brown the fox quick
the brown quick fox
quick fox the brown
brown quick the fox
brown the quick fox
brown fox quick the
the fox quick brown

If you give the structure of your file and what you expect to happen then I'll alter the code to read the file in.
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 16

Expert Comment

by:sjklein42
ID: 35103679
Lots of ways to skin a cat.

To run my version so it takes the words from an input file. replace the first line with this:

$words = join(' ',<>);
$words =~ s/[\r\n]//g;
$words =~ s/^\s+//;
$words =~ s/\s+$//;
@words = split(/\s+/, $words);

Open in new window


This will use all the words in the input keyword file, whether they are on one line or many.

Run it like this:

perl permute.pl keywords.txt
0
 
LVL 10

Expert Comment

by:APNFSSC
ID: 35103735
I'm wondering if the file could be,

this is the first text to jumble
this is the second text to jumple and needs to be kept seperate to first text
this is the third text

will wait for response rather than jumping to conclusions.
0
 

Author Comment

by:faithless1
ID: 35104454
Thanks,

APNFSSC:

The file will have records on each line (length varies some rows have sentences others 3-5 words, no set format)

word word word word
word word word

0
 
LVL 10

Expert Comment

by:APNFSSC
ID: 35104457
will there be any punctuation in the sentences? like question marks or full stops?
0
 

Author Comment

by:faithless1
ID: 35104561
I removed them all, just phrases sentences. thanks
0
 
LVL 10

Expert Comment

by:APNFSSC
ID: 35104696
this works but I've noticed that as the amount of words get's longer it's quite slow, you may be better off with sjklein42 or ozo's suggestions. I only tested with a small amount of words originally.

so give it a go with your data and see.

Just run it and pass the filename that you want to pass in.

#./test.pl filename.txt

#! /usr/bin/perl -w

use strict;

if( -e "$ARGV[0]"){
        my $infile=$ARGV[0];
        # you have passed a file to the script let's process that
         open(INFILE, "$infile") || die "Error opening input $infile: $!";
                while (<INFILE>){
                        chomp($_);
                        my $line = $_;
                        my @array = split(/ /, $line);
                        process_array(@array);
                }
}else{
        my @array = @ARGV ? @ARGV : qw/school store watches/;
        process_array(@array);
}

sub fac {
   my $nr = shift;
   return 1 if $nr < 2;
   return $nr*fac($nr-1);
}

sub process_array {
my @array = @_;
my %seen;
my $max = fac(scalar @array);
my $nr  = 0;
while( $nr < $max ) {
    my $need_combo = 1;
    my @combo;
    while( $need_combo ) {
        for( my $i = 0; $i < scalar @array; ++$i ) {
            $combo[$i] = $array[ rand scalar @array ];
        }
        my %unique;
        my $unique = 1;
        for my $c ( @combo ) {
            if( ++$unique{$c} > 1 ) {
                $unique = 0;
            }
        }
        if( $unique == 1 ) {
            $need_combo = 0;
        }
    }

    if( exists $seen{ "@combo" } ) {
        next;
    }
    else {
        local $" = " ";
        print "@combo\n";
    }
    ++$seen{"@combo"};
    ++$nr;
}
}

Open in new window

0
 
LVL 10

Expert Comment

by:APNFSSC
ID: 35104794
oh dear.... my bad I forgot to close the file... on my server it works quick until 6 words. then it's slow.

 
#! /usr/bin/perl -w

use strict;

if( -e "$ARGV[0]"){
        my $infile=$ARGV[0];
        # you have passed a file to the script let's process that
         open(INFILE, "$infile") || die "Error opening input $infile: $!";
                while (<INFILE>){
                        chomp($_);
                        my $line = $_;
                        my @array = split(/ /, $line);
                        process_array(@array);
                }
            close(INFILE);
}else{
        my @array = @ARGV ? @ARGV : qw/school store watches/;
        process_array(@array);
}

sub fac {
   my $nr = shift;
   return 1 if $nr < 2;
   return $nr*fac($nr-1);
}

sub process_array {
my @array = @_;
my %seen;
my $max = fac(scalar @array);
my $nr  = 0;
while( $nr < $max ) {
    my $need_combo = 1;
    my @combo;
    while( $need_combo ) {
        for( my $i = 0; $i < scalar @array; ++$i ) {
            $combo[$i] = $array[ rand scalar @array ];
        }
        my %unique;
        my $unique = 1;
        for my $c ( @combo ) {
            if( ++$unique{$c} > 1 ) {
                $unique = 0;
            }
        }
        if( $unique == 1 ) {
            $need_combo = 0;
        }
    }

    if( exists $seen{ "@combo" } ) {
        next;
    }
    else {
        local $" = " ";
        print "@combo\n";
    }
    ++$seen{"@combo"};
    ++$nr;
}
}

Open in new window

0
 
LVL 16

Assisted Solution

by:sjklein42
sjklein42 earned 500 total points
ID: 35104897
This version takes an input file as you described and permutes the words on each line:

# usage:   perl permute.pl input.txt

my @words;
my $level;
my @val;
my $N;

sub permute
{
    my $k = shift(@_);
    $level = $level+1;
    $val[$k] = $level;

    if ($level == $N)
    {
        my @out;
        for (my $i = 0; $i < $N; $i++) { push @out, $words[$val[$i]-1] ; }
        print join(" ", @out) . "\n";
    }
    else
    {
        for (my $i = 0; $i < $N; $i++) { if ($val[$i] == 0) { permute($i); } }
    }

    $level = $level-1;
    $val[$k] = 0;
}


while ( <> )
{
    s/[\r\n]//g;
    s/^\s+//;
    s/\s+$//;
    @words = split(/\s+/);

    $level = (-1);
    $N = @words; ## count

    for (my $i = 0; $i < $N; $i++) { $val[$i] = 0; }
    permute(0);
}

Open in new window

0
 
LVL 10

Expert Comment

by:APNFSSC
ID: 35104911
sjklein42's code is much faster than mine I would use his.
0
 
LVL 84

Expert Comment

by:ozo
ID: 35105964
Could you clarify whether you want a random shuffle, or all permutations?
And, if you have a line that repeats the same word, would swapping those words be  considered a different permutation or not?
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Perl Script to append xml file 32 184
perl script 4 94
perl to display a matrix report for 3-dimenional array 3 56
PERL - Find newest folder 12 101
Many time we need to work with multiple files all together. If its windows system then we can use some GUI based editor to accomplish our task. But what if you are on putty or have only CLI(Command Line Interface) as an option to  edit your files. I…
A year or so back I was asked to have a play with MongoDB; within half an hour I had downloaded (http://www.mongodb.org/downloads),  installed and started the daemon, and had a console window open. After an hour or two of playing at the command …
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This tutorial demonstrates a quick way of adding group price to multiple Magento products.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now