We have api which gets hits of 10000 per minute during mid october till jan end which we process with third party tool and save it in db.
Last year we were unable to process all records.
1> Due to huge no of hits
2> While storing the api details we process these data via third party tool which is too slow.
1> We can't do any thing for the tool nor we can look for its substitute for these third party tool.
2> we can't use help of load balancer due to some reason
Solution what we thought
To monitor the system after periodic time and check how much data has been received within 15 min of time and to know
what is the min and max time that has taken to process 1,00,000 records.
But it would get slow as record in db gets more 1,00,00,000 records and fetching data after every 15 min would surely affect the system
CREATE TABLE mobile_record (
id int NOT NULL auto_increment,
Is there some good method to resolve these issue.
which would process and store the hits in db and command line tool which would give me list of records received and stored in db and
other tool which would give me list of record which are not saved in db