Mehrsa Jalalizadeh
asked on
I cannot figure out a model that accurately predicts outcome, what am I missing
I have a large data (about 44,000 entries) with 10 variables and one outcome. I am trying to figure out a model that can predict the outcome (it is a true/false outcome). So far nothing can predict the outcome more accurately than 65%.
What am I doing wrong?
The outcome is whether the company car needs maintenance (true/false).
The variables are the ID number of person leasing the car, the age of the person leasing the car, starting odometer, ending odometer, starting time, ending time, area the car is being used and car model.
So far the best model that works is the average velocity and end odometer combined but it is not more than 65% accurate in predicting true.
The following variables have no predictive value: age, area, car model, time of day, day of the week
What am I doing wrong?
The outcome is whether the company car needs maintenance (true/false).
The variables are the ID number of person leasing the car, the age of the person leasing the car, starting odometer, ending odometer, starting time, ending time, area the car is being used and car model.
So far the best model that works is the average velocity and end odometer combined but it is not more than 65% accurate in predicting true.
The following variables have no predictive value: age, area, car model, time of day, day of the week
From that data I would not expect better than 65% accuracy.
How are you determining if a service is required?
If you are using objective measures (such as number of miles until service required according to manufacturer, number of miles since oil change, etc) then you should be able to improve on this and it may be your sample size is too small even at 40,000 observations.
However if the determinant of whether a service is required is subjective (eg assessment by mechanic based on their decision) this is partly the limiting factor in your accuracy and in fact what you may be measuring is the accuracy of the mechanic’s assessment (35% of the time they pass a vehicle which, according to the measures you are using, should actually be serviced). Which may mean you need to look again at what factors move the vehicle from “service required = false” to “service required = true”.
If you are using objective measures (such as number of miles until service required according to manufacturer, number of miles since oil change, etc) then you should be able to improve on this and it may be your sample size is too small even at 40,000 observations.
However if the determinant of whether a service is required is subjective (eg assessment by mechanic based on their decision) this is partly the limiting factor in your accuracy and in fact what you may be measuring is the accuracy of the mechanic’s assessment (35% of the time they pass a vehicle which, according to the measures you are using, should actually be serviced). Which may mean you need to look again at what factors move the vehicle from “service required = false” to “service required = true”.
This question needs an answer!
Become an EE member today
7 DAY FREE TRIALMembers can start a 7-Day Free trial then enjoy unlimited access to the platform.
View membership options
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.