Solved

Comparison problem

Posted on 2003-11-14
25
319 Views
Last Modified: 2010-03-31
For VK33:

There were actually 2 things i needed:

Method1:

The first was simply to calculate the average rating given to every cd.

Method2:

The second was to calculate a users average rating.

Step 1 Then for user 1, cd 1, take the average rating from the rating given to cd1.
Step 2 Do this for user2 as well.
Step 3 Then mulitply these 2 values together.

This will give total1

Repeat step1, only this time square the result
Repeat step2, only this time square the result
Repeat step3
Step 4 - Get the square root of this value

This will give total2

Divide total1 by total2

Do this for every cd that both users have rated and add all the values together

0
Comment
Question by:override_y2k
  • 14
  • 8
  • 2
  • +1
25 Comments
 
LVL 15

Expert Comment

by:jimmack
ID: 9751632
override_y2k.  Can you post a link to the other question that you are referring to.  This is a collaborative site.  For a variety of reasons (not least of which is that you didn't mention vk33 in the question title ;-)), vk33 may not see this.  Someone else may be able to help ;-)

Thanx.
0
 
LVL 15

Expert Comment

by:JakobA
ID: 9752504
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9753733
Oh sorry jimmack.The link has been provided by JakobA.
0
 
LVL 4

Expert Comment

by:vk33
ID: 9757993
Hi!

1. Calculating average cd's rating:

// first we accumulate rating sums here, then we'll divide them by number of rating records
HashMap averageRatings = new HashMap();
// number of rating records
HashMap numRatings = new HashMap();

// getting sums and rating numbers...
Iterator iter = userArrayList.iterator();
while (iter.hasNext()) {
   User user = (User)iter.next();
   Iterator iter2 = user.getpairArray().iterator();
   while (iter2.hasNext()) {
      Pair pair = (Pair)iter2.next();
      Float average = (Float)averageRatings.get(new Integer(pair.getId()));
      if (average == null) {
         averageRatings.put(new Integer(pair.getId()), new Float(pair.getRatings()));
      } else {
         averageRatings.put(new Integer(pair.getId()), new Float(pair.getRatings() + average.floatValue()));
      }
      Integer num = (Integer)numRatings.get(new Integer(pair.getId()));
      if (num == null) {
         numRatings.put(new Integer(pair.getId()), new Integer(1));
      } else {
         numRatings.put(new Integer(pair.getId()), new Integer(num + 1));
      }
   }
}
iter = averageRatings.keySet().iterator();

// dividing sums by rating numbers...
while (iter.hasNext()) {
   Integer id = (Integer)iter.next();
   float sum = ((Float)averageRatings.get(id)).floatValue();
   int num = ((Integer)numRatings.get(id)).intValue();
   averageRatings.put(id,new Float(sum/num));
}

That's it. You have your average ratings in the averageRatings map. To display the results do the following:

Iterator iter = averageRatings.keySet().iterator();
while (iter.hasNext()) {
   Integer id = (Integer)iter.next();
   Float average = (Float)averageRatings.get(id);
   System.out.println (id.intValue() + ": " + average.floatValue());
}

Regards!
0
 
LVL 4

Expert Comment

by:vk33
ID: 9758023
2. The second task looks very much similar to the one given before in the previous topic. So,  only some changes needed. Ok, let's assume that you've done calculating average ratings for all cd's before...

private static double compareUsers(User user1, User user2) {
   double sum = 0;
   Iterator iter = user1.getpairArray().iterator();
   while (iter.hasNext()) {
      Pair pair1 = (Pair)iter.next();
      Iterator iter2 = user2.getpairArray().iterator();
      while (iter2.hasNext()) {
         Pair pair2 = (Pair)iter.next();
         if (pair1.getId() == pair2.getId()) {

            // Step 1-1: getting average rating for cd1
            float average1 = ((Float)averageRatings.get(new Integer(pair1.getId()))).floatValue();

            // Step 1-2: getting average rating for cd2
            float average2 = ((Float)averageRatings.get(new Integer(pair2.getId()))).floatValue();

            // Step 1-3: multiplying -> total1
            double total1 = average1*average2;

            // Step 2-1: getting (average rating for cd1)^2
            double average1sqr = average1*average1;

            // Step 2-2: getting (average rating for cd2)^2
            double average2sqr = averate2 * average2;

            // Step 2-3: multiplying, getting square root -> total2
            double total2 = Math.sqrt(average1sqr*average2sqr);

            // Step 3: sum += total1/total2
            sum += total1/total2;
         }
      }
   }
   return sum;
}

This method processes all the cd's rated by both users. If you need to process every user with every other one, just add the process method from the previous topic, it'll work without changes.

I tried to make comments all along the code. But if you have any questions - feel free to ask.

Regards!
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9759027
Just a small change required:

In Step 1.1 the users average rating is the average in question.So the average rating a user gave to a cd should be calculated and subtracted from the rating they gave to a cd.Its not actually the average cd rating that is being taken away.That is just a different part of the problem.

So for example:

User Rating
  1       2
  1       3
  1       1
...............

Users average rating is 2 and then it'll be 2-2, 2-3, 2-1 ......

Thx!
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9759035
You were dying to get that HashMap in werent you! :)
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9759123
Also, it would be preferable not to use a hashmap when calculating a users average rating.

Thx for the help.
0
 
LVL 15

Expert Comment

by:JakobA
ID: 9759205
There is a simple formula for calculating a running average:

      newAverage =  ( newCount + ( oldNrOfCounts * oldAverage ) ) / newNrOfCounts;
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9759765
Erm yes i have no problem in working out an average.My problem is iterating through the ArrayList of users searching for a given cd and extracting its rating, then looping throuh every user and checking if they have rated it and then repeating this process for every cd.

That is the problem.
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9759848
So this is the order in which things should occur, a slight change of your step by step:

    private static double compareUsers(User user1, User user2) {
      double sum = 0;
      Iterator iter = user1.getpairArray().iterator();
      while (iter.hasNext()) {
        Pair pair1 = (Pair)iter.next();
        Iterator iter2 = user2.getpairArray().iterator();
        while (iter2.hasNext()) {
          Pair pair2 = (Pair)iter.next();
          if (pair1.getId() == pair2.getId()) {
           
       
            // Step 1-1: get average rating for user1
            // Step 1-2: subtract this from the rating given by user1 to cd1 -> total1

            // Step 1-3: get average rating for user2
            // Step 1-4: subtract this from the rating given by user2 to cd1 -> total2

            // Step 1-5: multiplying -> total3
            //double total3 = total1*total2;

            // Step 2-1: getting average rating for user1
            // Step 2-2: subtract this from the rating given by user1 to cd1 -> total4
            // Step 2-3: square this value

            // Step 2-4: getting average rating for user2
            // Step 2-5: subtract this from the rating given by user2 to cd1 -> total5
            // Step 2-6: square this value

            // Step 2-7: multiplying, getting square root -> total2
            //double total6 = Math.sqrt(total4*total5);

            // Step 3: sum += total3/total6
            //sum += total3/total6;

            //Repeat this for every cd both users have rated
            //Repeat this for every user
         }
      }
   }
   return sum;
 }
0
 
LVL 4

Accepted Solution

by:
vk33 earned 150 total points
ID: 9761576
Hi!

1. To get the average rating for specified user:

private static double averageRating(User user) {
   double sum = 0;
   int num = 0;
   Iterator iter = user.getpairArray().iterator();
   while (iter.hasNext()) {
      Pair tmp = (Pair)iter.next();
      sum += tmp.getRatings();
      num++;
   }
   if (num == 0)
      return 0;
   return sum/num;
}

2. Changes in the compareUsers() method:

    private static double compareUsers(User user1, User user2) {
      double sum = 0;
      Iterator iter = user1.getpairArray().iterator();
      while (iter.hasNext()) {
        Pair pair1 = (Pair)iter.next();
        Iterator iter2 = user2.getpairArray().iterator();
        while (iter2.hasNext()) {
          Pair pair2 = (Pair)iter.next();
          if (pair1.getId() == pair2.getId()) {
           
       
            // Step 1-1: get average rating for user1
            double average1 = averageRating(user1);

            // Step 1-2: subtract this from the rating given by user1 to cd1 -> total1
            double total1 = pair1.getRatings() - average1;
           
            // Step 1-3: get average rating for user2
            double average2 = averageRating(user2);

            // Step 1-4: subtract this from the rating given by user2 to cd1 -> total2
            double total2 = pair2.getRatings() - average2;

            // Step 1-5: multiplying -> total3
            double total3 = total1*total2;

            // Step 2-1: getting average rating for user1
            // Step 2-2: subtract this from the rating given by user1 to cd1 -> total4
            // Step 2-3: square this value
            double total4 = total1*total1;

            // Step 2-4: getting average rating for user2
            // Step 2-5: subtract this from the rating given by user2 to cd1 -> total5
            // Step 2-6: square this value
            double total5 = total2*total2;

            // Step 2-7: multiplying, getting square root -> total6
            double total6 = Math.sqrt(total4*total5);

            // Step 3: sum += total3/total6
            sum += total3/total6;
         }
      }
   }
   return sum;
 }

3. So, this processing is already repeated for all the cd's both users have rated (see the nested while-loops). To apply this to all the users call the process() method. It's only mission is iterating through all possible pairs of users and calling compareUsers() method for them:

public static double[][] process(ArrayList userArrayList) {
Object[] users = userArrayList.toArray();
double[][] users_diffs = new double[users.length-1][users.length-1];
for (int i=0; i<users.length-1; i++)
   for (int j=i+1; j<users.length; j++) {
      User user1 = (User)users[i];
      User user2 = (User)users[j];
      users_diffs[i][j] = compareUsers(user1,user2);
   }
return users_diffs;
}

4. As for the HashMap... Hmm, a good programmer is a lazy programmer. :) I just simplify the code. The ONLY purpose of my using HashMap here is simplifying futher access to the results. And one more thing. Look, I need the number of records processed for every cd, right? Then I'll divide the sum by this number and get the average. OK, to keep this numbers for all the cd's I need either new class (another pair, but cd->num) or a HashMap which is actually a set of pairs too. Of course there's one more workaround: 2 arrayLists (one for cdId's and one for numbers). But isn't it too complicated for such a simple task? I don't really understand why not using the provided functionality, especially if we do not change anything in the interface between your code and mine. :)

Best regards!
0
What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

 
LVL 2

Author Comment

by:override_y2k
ID: 9763763
Ok, i just have 1 more silly question for you and that's how where will i put the hashmap into my fileparser class.

I've awarded the points already, but if you could tell me that would be great, as i'm not exactly an expert when it come to hashmaps and hashtables :)

BTW: Point taken bout lazy programming :)
0
 
LVL 4

Expert Comment

by:vk33
ID: 9763953
Well, HashMap is not actually very different from ArrayList. The only difference is that it has two associated lists - keys and values. When you put a new record you specify both key and value: HashMap.put(Object key, Object value). When you get the value back, you just specify the key: HashMap.get(Object key). And you can iterate through both keys and values as if they were separate collections: HashMap.keySet().iterator() and HashMap.values().iterator();

But I don't see any HashMaps left in the last release (the accepted answer)... :)

Best regards!
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9764254
There were  none in that , put i did still need a mhod for calculating the average rating given to every cd(as well as a users average rating).

Thats why i was looking at your average method using hashmaps
0
 
LVL 4

Expert Comment

by:vk33
ID: 9770070
ok, no problem, define a HashMap in your parser class and create a private method for calculating cd-s averages:

class Parser {
   private HashMap averageRatings = new HashMap();
   private userArrayList = new ArrayList();
   ...

private calculateAverageRatings() {
// number of rating records
HashMap numRatings = new HashMap();

// getting sums and rating numbers...
Iterator iter = userArrayList.iterator();
while (iter.hasNext()) {
   User user = (User)iter.next();
   Iterator iter2 = user.getpairArray().iterator();
   while (iter2.hasNext()) {
      Pair pair = (Pair)iter2.next();
      Float average = (Float)averageRatings.get(new Integer(pair.getId()));
      if (average == null) {
         averageRatings.put(new Integer(pair.getId()), new Float(pair.getRatings()));
      } else {
         averageRatings.put(new Integer(pair.getId()), new Float(pair.getRatings() + average.floatValue()));
      }
      Integer num = (Integer)numRatings.get(new Integer(pair.getId()));
      if (num == null) {
         numRatings.put(new Integer(pair.getId()), new Integer(1));
      } else {
         numRatings.put(new Integer(pair.getId()), new Integer(num + 1));
      }
   }
}
iter = averageRatings.keySet().iterator();

// dividing sums by rating numbers...
while (iter.hasNext()) {
   Integer id = (Integer)iter.next();
   float sum = ((Float)averageRatings.get(id)).floatValue();
   int num = ((Integer)numRatings.get(id)).intValue();
   averageRatings.put(id,new Float(sum/num));
}
}

That's it. Your averageRatings hashmap is referenced from the class and thus you can use this reference in your methods. Remember that you need to do the same with your userArrayList. But you can also pass it as an argument of calculateAverageRatings() method.

Regards!
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9770551
When i layout the code like this, it throws up errors

class FileParser {
  public static void main(String[] args) throws Exception {

    private HashMap averageRatings = new HashMap();
    private userArrayList = new ArrayList();

    FileReader fr =
      new FileReader("C:/dataset.txt"); // to open the file
      BufferedReader inFile = new BufferedReader(fr);     // to read the file
      String line;     // String variable used to hold each line of the file
      StringTokenizer tokenizer;     // string tokenizer to parse each file line
      ArrayList userArrayList = new ArrayList();
      int id = 0;
      int index = 0;     // Index that we increment each time through the loop
      User user = null;
      int movieId;     // value to hold movie id
      float ratings;     // value to hold movie ratings
      ArrayList pair = null;
      while((line = inFile.readLine()) != null) {
        tokenizer = new StringTokenizer(line);     // tokenize the file line
        // now add the user id to the user id array
        id = Integer.parseInt(tokenizer.nextToken());
        // initialize the movie id variable
        movieId = Integer.parseInt(tokenizer.nextToken());
        // initialize the cd ratings variable
        ratings = Float.parseFloat(tokenizer.nextToken());
        // Check out if the user has already been added to the list
        Iterator iter = userArrayList.iterator();

        while (iter.hasNext()) {
          User tmp = (User)iter.next();
          if (tmp.getUserId() == id) {
            user = tmp;
            break;
          }
        }

        if (user == null) {
          pair = new ArrayList();
          pair.add(new Pair(movieId, ratings));
          user = new User(id,pair);
          userArrayList.add(user);
        } else
          user.getpairArray().add(new Pair(movieId, ratings));

          // incrememnt index
          index++;
        }
       
    }

    private calculateAverageRatings() {
      // number of rating records
      HashMap numRatings = new HashMap();
   
      // getting sums and rating numbers...
      Iterator iter = userArrayList.iterator();
      while (iter.hasNext()) {
         User user = (User)iter.next();
         Iterator iter2 = user.getpairArray().iterator();
         while (iter2.hasNext()) {
            Pair pair = (Pair)iter2.next();
            Float average = (Float)averageRatings.get(new Integer(pair.getId()));
            if (average == null) {
               averageRatings.put(new Integer(pair.getId()), new Float(pair.getRatings()));
            } else {
               averageRatings.put(new Integer(pair.getId()), new Float(pair.getRatings() + average.floatValue()));
            }
            Integer num = (Integer)numRatings.get(new Integer(pair.getId()));
            if (num == null) {
               numRatings.put(new Integer(pair.getId()), new Integer(1));
            } else {
               numRatings.put(new Integer(pair.getId()), new Integer(num + 1));
            }
         }
      }
      iter = averageRatings.keySet().iterator();
   
  // dividing sums by rating numbers...
      while (iter.hasNext()) {
         Integer id = (Integer)iter.next();
         float sum = ((Float)averageRatings.get(id)).floatValue();
         int num = ((Integer)numRatings.get(id)).intValue();
         averageRatings.put(id,new Float(sum/num));
      }
      }

"FileParser.java": Error #: 215 : invalid method declaration; return type required at line 84, column 36

The calculateAverage Ratings method is looking for a return type. I'm unsure on how to fix this error.
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9770558
Should i be returning sum/num?
0
 
LVL 4

Expert Comment

by:vk33
ID: 9770582
your averageRatings and userArrayList should be defined in the class, not in the main method:

class FileParser {
    private HashMap averageRatings = new HashMap();
    private userArrayList = new ArrayList();

  public static void main(String[] args) throws Exception {
    FileReader fr =
      new FileReader("C:/dataset.txt"); // to open the file
      BufferedReader inFile = new BufferedReader(fr);     // to read the file
      String line;     // String variable used to hold each line of the file
      StringTokenizer tokenizer;     // string tokenizer to parse each file line
      int id = 0;
      int index = 0;     // Index that we increment each time through the loop
      User user = null;
      int movieId;     // value to hold movie id
      float ratings;     // value to hold movie ratings
      ArrayList pair = null;
      while((line = inFile.readLine()) != null) {
        tokenizer = new StringTokenizer(line);     // tokenize the file line
        // now add the user id to the user id array
        id = Integer.parseInt(tokenizer.nextToken());
        // initialize the movie id variable
        movieId = Integer.parseInt(tokenizer.nextToken());
        // initialize the cd ratings variable
        ratings = Float.parseFloat(tokenizer.nextToken());
        // Check out if the user has already been added to the list
        Iterator iter = userArrayList.iterator();

        while (iter.hasNext()) {
          User tmp = (User)iter.next();
          if (tmp.getUserId() == id) {
            user = tmp;
            break;
          }
        }

        if (user == null) {
          pair = new ArrayList();
          pair.add(new Pair(movieId, ratings));
          user = new User(id,pair);
          userArrayList.add(user);
        } else
          user.getpairArray().add(new Pair(movieId, ratings));

          // incrememnt index
          index++;
        }
        calculateAverageRatings();    
    }

   private calculateAverageRatings() {
      // number of rating records
      HashMap numRatings = new HashMap();
   
      // getting sums and rating numbers...
      Iterator iter = userArrayList.iterator();
      while (iter.hasNext()) {
         User user = (User)iter.next();
         Iterator iter2 = user.getpairArray().iterator();
         while (iter2.hasNext()) {
            Pair pair = (Pair)iter2.next();
            Float average = (Float)averageRatings.get(new Integer(pair.getId()));
            if (average == null) {
               averageRatings.put(new Integer(pair.getId()), new Float(pair.getRatings()));
            } else {
               averageRatings.put(new Integer(pair.getId()), new Float(pair.getRatings() + average.floatValue()));
            }
            Integer num = (Integer)numRatings.get(new Integer(pair.getId()));
            if (num == null) {
               numRatings.put(new Integer(pair.getId()), new Integer(1));
            } else {
               numRatings.put(new Integer(pair.getId()), new Integer(num + 1));
            }
         }
      }
      iter = averageRatings.keySet().iterator();
   
  // dividing sums by rating numbers...
      while (iter.hasNext()) {
         Integer id = (Integer)iter.next();
         float sum = ((Float)averageRatings.get(id)).floatValue();
         int num = ((Integer)numRatings.get(id)).intValue();
         averageRatings.put(id,new Float(sum/num));
      }
   }
}

Regards!
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9770818
It still throws the same error, ie it is looking for a return type in the calculateAverageRatings method
0
 
LVL 4

Expert Comment

by:vk33
ID: 9770903
ah, yes,

private static void calculateAverageRatings()

And please, change
    private HashMap averageRatings = new HashMap();
    private userArrayList = new ArrayList();
to
    private static HashMap averageRatings = new HashMap();
    private static ArrayList userArrayList = new ArrayList();

Regards!
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9770973
The compiler doesn't like this line:

 numRatings.put(new Integer(pair.getId()), new Integer(num + 1));


Gives this error:
"FileParser.java": Error #: 375 : operator + cannot be applied to (java.lang.Integer, int) at line 110, column 74
0
 
LVL 4

Expert Comment

by:vk33
ID: 9771756
numRatings.put(new Integer(pair.getId()), new Integer(num.intValue() + 1));
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9771903
Great thx v much for the help.
0
 
LVL 2

Author Comment

by:override_y2k
ID: 9797576
Maybe you can help we with this problem:

http://www.experts-exchange.com/Programming/Programming_Languages/Java/Q_20805158.html

Bit unsure on how to do it.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Introduction Java can be integrated with native programs using an interface called JNI(Java Native Interface). Native programs are programs which can directly run on the processor. JNI is simply a naming and calling convention so that the JVM (Java…
Introduction This article is the last of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers our test design approach and then goes through a simple test case example, how …
Viewers learn about the “for” loop and how it works in Java. By comparing it to the while loop learned before, viewers can make the transition easily. You will learn about the formatting of the for loop as we write a program that prints even numbers…
This tutorial covers a practical example of lazy loading technique and early loading technique in a Singleton Design Pattern.

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now