asked on python for analysing data over 300,000,000 rows and 20 columns
i would look like ro analyse/describe data that is big over 300,000,000 rows do you python can be able to help me do this. how efficiently can i do thos
Statistical Analysis System (SAS)PythonMongoDBDatabasesNoSQL Databases
Last Comment
noci
ASKER CERTIFIED SOLUTION
THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
ASKER
thank you very much. what is i summaries and reduce the number of rows to not more than 12,000,000 uaing sql server or any DBMS before i do extraction to where i will need python. i want to use python for analysis. do you think this will help
SOLUTION
THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
SOLUTION
THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
The reason is that Python is an interpreted language, not a compiled language. Given the nature of interpreted languages, Python must run slower than an equivalent compiled program that does the same job. At a guess it would run at best 1/3 the speed of a compiled program.
If the intended application needs to handle a 300 million row table, you probably want to get all the performance possible out of the application. C or C++ are excellent possibilities if the data involves multiple data types, and if it is purely numeric number crunching it is hard to beat good old FORTRAN.