C++ memory use explosion

Wrote a C++ wrapper class to the MySQL/C API that gets a row of DB data and turns it into vector of floats.

Unfortunately, I'm doing about 1x10^6 all-against-all row comparisons, and the memory use of the program as it runs absolutely explodes, eventually crashing due to lack of memory. The whole point of interfacing the code to a db was, of course, to avoid this sort of excessive memory use. Below is my DB.h class. I don't see any obvious extravagant uses of memory, but nonetheless, upon invoking the getPriceVector() method a few thousands times, my system monitor indicates spiraling memory usage. The compiler is  gnu 3.2.2, the system is RH 9, and it's mysql 4.1.



#include <mysql/mysql.h>
#include <stdio.h>
#include <string.h>
#include <DataSeries.h>
using namespace std;
class DB{
 public:
  string name;
  MYSQL mysql;

  DB(char *dbname){
   
    mysql_init(&mysql);
    if (!mysql_real_connect(&mysql,"localhost.localdomain","agm","chacal", dbname,0,NULL,0))
      {
      printf( "Error connecting to database: %s\n",mysql_error(&mysql));
      }
    //    else printf("Connected...\n");
   
  }

  vector<string> getTickers(){
    char query[255];
    sprintf(query,"show tables");
    char **row;
    vector<string> result;
    MYSQL_RES *res;
    if(mysql_real_query(&mysql, query, strlen(query))){
      printf(mysql_error(&mysql));
      exit(1);
    }
    res=mysql_store_result(&mysql);
    row=mysql_fetch_row(res);
    char *ticker;
    while(row!=NULL){
      string s(row[0]);
      result.push_back(s);
      row=mysql_fetch_row(res);

    }
    return result;
   
   
   
  }
 
  vector<float> getPriceVector(string &t, string &d1, string &d2){
   
    char *startQuery=(char*) malloc(255);
    char *endQuery=(char*) malloc(255);
    char *dataQuery=(char*) malloc(1023);
    //dataQuery=malloc(255);
    //endQuery=malloc(255);
    //startQuery(255);
    sprintf(startQuery, "select orderIndex from `%s` where date=%s", t.c_str(), d1.c_str());
    sprintf(endQuery, "select orderIndex from `%s` where date=%s", t.c_str(), d2.c_str());
    //    cout<<startQuery<<endl;
    //cout<<endQuery<<endl;
    char **row;
    MYSQL_RES *resStart;
    MYSQL_RES *resEnd;
    MYSQL_RES *resData;
    bool startSuccess=false;
    bool endSuccess=false;
    char *startIndex;
    char *endIndex;
    if(mysql_real_query(&mysql, startQuery, strlen(startQuery))){
      //cout<<"error\n";
      printf(mysql_error(&mysql));
      //DataSeries ds();
      //return ds;
      //exit(1);
    }else{
      startSuccess=true;
     
      resStart=mysql_store_result(&mysql);
      row=mysql_fetch_row(resStart);
     
      if(row!=NULL){

      startIndex=row[0];
      //      cout<<"start "<<startIndex<<endl;
      
      }else{
      //      cout<<"couldn't find entry for "<<d1<<endl;
      }
    }

    if(mysql_real_query(&mysql, endQuery, strlen(endQuery))){
      printf(mysql_error(&mysql));
      //      DataSeries ds();
      //return ds;
      //exit(1);
    }else{
      endSuccess=true;
      resEnd=mysql_store_result(&mysql);
      row=mysql_fetch_row(resEnd);

      if(row!=NULL){
      
      endIndex=row[0];
      //      cout<<"end "<<endIndex<<endl;
      }else{
      //      cout<<"couldn't find entry for "<<d2<<endl;
      }    
    }
    //vector<string> dates;
    vector<float> prices;
    if(startSuccess && endSuccess){
      sprintf(dataQuery, "select date, close from `%s` where orderIndex>=%s and orderIndex<=%s order by orderIndex",
            t.c_str(), startIndex, endIndex);
      //      cout<<"start: "<<startIndex<<" stop "<<endIndex<<endl;
   
      if(mysql_real_query(&mysql, dataQuery, strlen(dataQuery))){
      //cout<<"error\n";
      printf(mysql_error(&mysql));
      //      DataSeries ds();
      //return ds;
      //exit(1);
      }else{
      
      resData=mysql_store_result(&mysql);
      row=mysql_fetch_row(resData);
     
      while(row!=NULL){
        string s(row[0]);
        //dates.push_back(s);
        prices.push_back((float) strtod(row[1], NULL));
        row=mysql_fetch_row(resData);
      
      }
      }
    }else{
                 cout<<"couldn't find either start or end\n";
    }
    /*for(int i=0; i<dates.size(); i++){
      cout<<dates[i]<<"\t"<<prices[i]<<endl;
      }*/
    //cout<<"prices "<<prices.size()<<endl;
    //cout<<"dates "<<dates.size()<<endl;
    free(startQuery);
    free(endQuery);
    free(dataQuery);
    return prices;
   
   
   
    //startQ<<"select orderIndex from "<<t.c_str()<<" where date="<<d1.c_str();

  }


};
antoniogmAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

jkrCommented:
Correect me if I got that wrong, but you're never freeing the data that is returned as 'row'. That for sure will lead to memory leaks.
0
grg99Commented:
Are you returning the "prices" vector you return once you're done using the values?

You have several places (commented out) where the function could prematurely return before freeing what it has malloced.
That's not good.

Either fix up the code so it all exits out the bottom, or perhaps better, don't malloc those strings, just declare them as static char arrays, then you don't have to allocate or free them or worry about leaks.

Also it wouldnt hurt to use snprintf just in case one of the strings gets too long and threatens to overflow the char arrays.



0
antoniogmAuthor Commented:
Tried changing the declaration to the following:

char **row=(char **)malloc(2055);

followed by:

free(row).

Is this even the correct way to malloc a char**?

In any case, it doesn't help the memory problem....memory use is still off the charts. In fact, it throw a seg fault at a seemingly random point in the running (i.e. not immediately...but halfway through).
0
Get expert help—faster!

Need expert help—fast? Use the Help Bell for personalized assistance getting answers to your important questions.

jkrCommented:
>>Is this even the correct way to malloc a char**?

Usually, yes. But, you should not use a char** here. According to the docs (http://dev.mysql.com/doc/mysql/en/mysql_fetch_row.html), it is a 'MYSQL_ROW', e.g.

MYSQL_ROW row;
unsigned int num_fields;
unsigned int i;

num_fields = mysql_num_fields(result);
while ((row = mysql_fetch_row(result)))
{
   unsigned long *lengths;
   lengths = mysql_fetch_lengths(result);
   for(i = 0; i < num_fields; i++)
   {
       printf("[%.*s] ", (int) lengths[i], row[i] ? row[i] : "NULL");
   }
   printf("\n");
}

But, what I am also missing is a call to 'mysql_free_result()' (http://dev.mysql.com/doc/mysql/en/mysql_free_result.html), which is necessary to free the memory allocated by a query. When you are done with a result set, you must free the memory it uses by calling 'mysql_free_result()'

0
antoniogmAuthor Commented:
I changed it to it's a static declartion:

char startQuery[255];
char endQuery[255];
char dataQuery[1023];
char **row;

Same memory problem. Could it be the "row"?
0
jkrCommented:
>>Could it be the "row"?

Try adding the necessary calls to 'mysql_free_result()' first. And use

MYSQL_ROW row;

instead of 'char**'.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C++

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.