Best format for History of values?

Hi Experts,

I am creating a model of (at least) one financial market and need to store a history of the date/times and values at those times.

So my question is: "What is the best way to store this kind of information?"

The values will be doubles, and there may be varying number of data points per day.
I will need to perform varying analyses of the data, and naturally, create various charts of the data.

TIA, jazzle
LVL 1
JazzleAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

CEHJCommented:
In a database?
JazzleAuthor Commented:
hmm, good thought - not something I'd considered.
I haven't yet used a DB in any Java programming - any recommendations on what to use? (good documentation a def' req')
Open-Source preferred, but to be used in a commercial app, so license must allow this.
however, to do any of the numerical processing (trend spotting for instance) wouldn't I need to copy the data into a Java format?

Still open to other format suggestions BTW.
CEHJCommented:
MySql and Postgresql come to mind. The latter has a rich function library already

>> to do any of the numerical processing (trend spotting for instance) wouldn't I need to copy the data into a Java format?

Possibly yes.
Announcing the Winners!

The results are in for the 15th Annual Expert Awards! Congratulations to the winners, and thank you to everyone who participated in the nominations. We are so grateful for the valuable contributions experts make on a daily basis. Click to read more about this year’s recipients!

JazzleAuthor Commented:
Any suggestions on what Java format?
I'm loathe to use plain arrays for performance issues - Is this a valid concern?
Some data may originate from XML - should it stay as XML or be converted?

(I realise I'm asking more questions now, and will happily increase the points if you deem it necessary.)
CEHJCommented:
Give us a brief view of the data and how you're going to process them
JazzleAuthor Commented:
I don't have any real data as yet, but it will be very basic.

A time-date and a double value representing a price in Euro.
The date needn't be more precise than a particular minute of the day (probably an hour is precise enough, possibly just a day).
I was going to keep the price as a double.

The processing will be intended to predict possible future prices. (So there will be other data created/imported.)

I'm starting to think that a database is the right way to go since I realised I will have various data for each time-point, past and future.

Though actually, it may be complicated by the fact that some data will be daily, some may be weekly, etc.
i.e. the granularity may vary.

(have you noticed I'm thinking aloud here?)

perhaps the values should actually be matched to time-periods.

(argh)
CEHJCommented:
Well you could have a foreign key of 'snapshot_period' or some such to represent granularity, but i don't want to think aloud too ;-)
JazzleAuthor Commented:
I was thinking simply have a start and end date for each data point.
I think I will come back to this question in the next couple of days, let my mind do some thinking while I work on other things.

If I don't come back in this time please do give me a nudge.

Thanks for your help so far.
CEHJCommented:
OK
JazzleAuthor Commented:
It looks like I may be able to use a data format* from the charting package** I'll be using.

I'm not sure how to deal with this question, can you advise?
(NB: I will not be giving points to any further participants, unless something amazingly useful is said.)

* The cunningly named 'TimeSeries'
** jfreechart, www.jfree.org
CEHJCommented:
>>I'm not sure how to deal with this question, can you advise?

What i usually do in these circs is to get the points lowered and then award useful contribs

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
JazzleAuthor Commented:
I (tbh) cba getting the points lowered, so you can have them all.
Thanks for asking the questions which answered mine!
jazzle
CEHJCommented:
:-)
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java

From novice to tech pro — start learning today.