thready
asked on
Write database table to a stream and read stream back into other equivalent database table
Hi Experts,
I would like to design a fast algorithm that can take items in a database table and spit them out to a stream. I'm new with stream programming - but I know there is a way to write to a file as a stream or treat sockets as the stream. Can someone point out how to set this up? And how would I delimit my records?
Many thanks,
Mike
I would like to design a fast algorithm that can take items in a database table and spit them out to a stream. I'm new with stream programming - but I know there is a way to write to a file as a stream or treat sockets as the stream. Can someone point out how to set this up? And how would I delimit my records?
Many thanks,
Mike
ASKER
Hi jkr, it's more complicated than that - I'm asking a more basic question than what I need - I am using Postgres. Basically, I will use this with streams and to do synchronization between databases (sort of). I have database objects smart enough to know how to save their own data, etc.... I've started by using XML - I'm hoping it's not too slow with all the text and extra lengths because of tag identifiers, etc. Do you know of a library that can zip as I write to a file?
Mike
Mike
>> Basically, I will use this with streams and to do synchronization between databases (sort of).
Why don't you just use database replication?
http://www.postgresql.org/about/news.233
Why don't you just use database replication?
http://www.postgresql.org/about/news.233
ASKER
I can't replicate the database tables - some tables will need to have minor differences (private data not to be shared with others)....
BTW: Since you are using Postgres, why did you post to the MySQL Zone? There is a Postgres zone, I'm sure you would have been better off posting to there (I know MySQL but not Postgres!).
https://www.experts-exchange.com/Database/PostgreSQL/
https://www.experts-exchange.com/Database/PostgreSQL/
>> I can't replicate the database tables - some tables will need to have minor differences (private data not to be shared with others)....
Trying to synchronize manually it not going to be a trivial task. Writing the code to read from one and write to another is pretty simply. The problem is going to be managing concurrency. Have you given any thought to how you are going to do this?
Trying to synchronize manually it not going to be a trivial task. Writing the code to read from one and write to another is pretty simply. The problem is going to be managing concurrency. Have you given any thought to how you are going to do this?
BTW: You might want to consider using ADO.Net for this.
http://en.wikipedia.org/wiki/ADO.NET
http://en.wikipedia.org/wiki/ADO.NET
ASKER
evilrix - it's not full synchronization... It's a simplification - and it took me about a year to figure out how to do it (to answer your question about whether or not I thought about how I was going to do this)... . I've also got 3 years of database objects coded that use ODBC... can't switch to nicer newer APIs this late in the game unfortunately.... :-) Thanks for your ideas & suggestions.
Mike
Mike
To be honest I'm a little at a lose as to what you actual question is then? You just want to know how to open and write a file stream in C++? See below.
http://www.cplusplus.com/doc/tutorial/files.html
http://www.cplusplus.com/reference/iostream/
http://www.cplusplus.com/doc/tutorial/files.html
http://www.cplusplus.com/reference/iostream/
#include <fstream>
int main()
{
std::fstream fs("c:\\temp\\test.txt", std::ios::out | std::ios::in | std::ios::app);
fs << "Hello World" << std::endl;
}
ASKER
Hi evilrix:
I want to know how to link up functions - so I can pipe a stream one way or the other (kind of like OpenSSL BIO objects in case you're familiar with them).. It may be a silly question, but I've just never piped one function's input stream to another.... I just need some background and maybe if you know about a compression "stream" that I can write to as well....
Mike
I want to know how to link up functions - so I can pipe a stream one way or the other (kind of like OpenSSL BIO objects in case you're familiar with them).. It may be a silly question, but I've just never piped one function's input stream to another.... I just need some background and maybe if you know about a compression "stream" that I can write to as well....
Mike
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
exactly what I was looking for! Thank you very much! Many thanks! :-) (I think that's the one that a colleague had used a few years back)...
One last thing. Building Boost (on Windows) can be a painful experience since it uses its own proprietary build system (BJam). Fortunately, you can download precompiled binaries for Windows from the following site: http://www.boost-consulting.com/products/free
I hope this helps and good luck.
-Rx.
I hope this helps and good luck.
-Rx.
ASKER
Thanks for the luck - I've got a serious load of work to do and only a few months to get it all done & tested.... It's good not to feel so alone doing it.... And thanks again.
ASKER
Oh and best to you as well.
>> It's good not to feel so alone doing it...
Always here to help -- well, unless my wife is giving me grief for being at the PC "again" :)
Seriously though, if you get stuck just shout.
>> Oh and best to you as well.
And the best to you too my friend.
-Rx.
Always here to help -- well, unless my wife is giving me grief for being at the PC "again" :)
Seriously though, if you get stuck just shout.
>> Oh and best to you as well.
And the best to you too my friend.
-Rx.
ASKER
>> Always here to help -- well, unless my wife is giving me grief for being at the PC "again" :)
Haha - Don't I hear ya.... :-)
Cheers,
Mike
Haha - Don't I hear ya.... :-)
Cheers,
Mike
BTW, which DBMS and library are you using?