Link to home
Start Free TrialLog in
Avatar of websss
websssFlag for Kenya

asked on

NewSQL database for IOT sensors (high write rate) - with HA/ horizontal scaling ?

I'm currently looking at massive scaling challenges

I need a database system that supports the following

here are my requirements
1) High insert rates for timeseries data (60,000 records per second - could be batch inserted)
2) NewSql style database (not NoSql)
3) One main table takes the LOG data, the rest are things like users etc that I need to join to the LOG table, therefore i need full SQL support
4) be able to scale horizontally and also put different months data on different servers so I dont run out of diskspace (currently its all on one large disk)
5) where possible just throw new hardware at the database and it work
6) support OLAP and OLTP work loads - i.e. high insert rates, and reporting (ideally under one system)
7) Open source / free

I've looked at:
Postgres & Citus
MemSql
Vitess

I'm wondering if anyone has any input or experience that could help recommending a good option or considerations
Avatar of Tomas Helgi Johannsson
Tomas Helgi Johannsson
Flag of Iceland image

Hi,

Take a look at MariaDB and the Spider engine for this usage.
Example on Spider sharding setup can be seen here.

Regards,
    Tomas Helgi
Avatar of websss

ASKER

thanks
It doesn't really look suitable for me
Any other options
ASKER CERTIFIED SOLUTION
Avatar of skullnobrains
skullnobrains

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of websss

ASKER

Thanks all