Python - exporting results to .csv file

Dear Experts,

I have the following Python code which predicts result on the iris dataset in the frame of machine learning.

# -*- coding: utf-8 -*-

# Load libraries
import pandas
from pandas.tools.plotting import scatter_matrix
import matplotlib.pyplot as plt
from sklearn import model_selection
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.naive_bayes import GaussianNB
from sklearn.svm import SVC

# Load dataset
path = "C:\\Users\\zc\\Downloads\\Daily files\\Machine Learning\\IrisGenerated.csv"
names = ['sepal-length', 'sepal-width', 'petal-length', 'petal-width', 'class']
dataset = pandas.read_csv(path, names=names)

# Split-out validation dataset
array = dataset.values
X = array[:,0:4]
Y = array[:,4]
validation_size = 0.20
seed = 7
X_train, X_validation, Y_train, Y_validation = model_selection.train_test_split(X, Y, test_size=validation_size, random_state=seed)

# Test options and evaluation metric
seed = 7
scoring = 'accuracy'

# Fit the model
model = KNeighborsClassifier()
model.fit(X_validation, Y_validation)

# save the model to disk
filename = 'finalized_model.sav'
joblib.dump(model, filename)

# load the model from disk
loaded_model = joblib.load(filename)
result = loaded_model.score(X_validation, Y_validation)
print(result)

detailed_result = model.predict(X)

print(detailed_result)

Open in new window


Basically the results are in the 'detailed_result' variable at the end, could you please advise with the syntax how that could be exported as .csv file from Python?

Thanks in advance,
LVL 1
csehzIT consultantAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

peprCommented:
Basically, you should use the standard csv module. For that, you should be able to get rows of values from your result (lists or tuples). (Actually, I did not look at your code, so consider the following a hint.)
import csv
fname = 'output.csv'
with open(fname, 'w', newline='') as f:
    writer = csv.writer(f)
    for row in detailed_result:
        writer.writerow(row)

Open in new window

0
csehzIT consultantAuthor Commented:
Thank you that looks working basically, although having a small issue that the 'detailed_result' looks like this on the Python screen:

['Iris-setosa' 'Iris-setosa' 'Iris-setosa' ..., 'Iris-setosa'
 'Iris-virginica' 'Iris-virginica']


while those 'Iris-setosa' texts are exported as 'I,r,i,s,-,s,e,t,o,s,a'
Importing screen from Excel
Could you please advise how to change the exporting syntax to bring the texts in the original format so 'Iris-setosa'?
0
peprCommented:
I see. So, should your detailed_result be exported as a single row into a CSV file, or should it be appended to the existing file?

In the first case, the code should be changed to...
import csv
fname = 'output.csv'
with open(fname, 'w', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(detailed_result)

Open in new window


In the later case it should use the append mode of the file...
import csv
fname = 'output.csv'
with open(fname, 'a', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(detailed_result)

Open in new window


The reason for the data you observed (unwanted splitting to the characters) is that the for loop iterates through the given data. If it were a list of rows, the items would be rows. If it is a string, the items are characters.

A side note: The csv.writer can be parametrized to fit better with Excel.

Or is the detailed_result just a list of strings? Do you actually want to write them as lines to a normal text file?
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Become a CompTIA Certified Healthcare IT Tech

This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

csehzIT consultantAuthor Commented:
Thank you for your explanation, yes the 'detailed_result is a list of strings, which can be finally re-worked in the following format also by using your originally proposed code (with renaming the file to .txt, importing to Excel and replacing the commas with nothing)

The imported and formatted results in Excel
In this way eventually they could be written also to a normal text file, if that is easier to export and maybe the best would be in tab limited format.
0
csehzIT consultantAuthor Commented:
Anyway testing the proposed code in your last answer:

import csv
fname = 'output.csv'
with open(fname, 'w', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(detailed_result)

Open in new window


that is although not being perfect for the target yet, but already better than the original one, because with a simple Copy and Paste Special as Transponse can be formatted:

Excel re-formatting with Transponse
Accordingly accepting your solution, thanks a lot
0
csehzIT consultantAuthor Commented:
.
0
csehzIT consultantAuthor Commented:
For later reference so the whole working code:

# -*- coding: utf-8 -*-

# Load libraries
import pandas
from pandas.tools.plotting import scatter_matrix
import matplotlib.pyplot as plt
from sklearn import model_selection
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.naive_bayes import GaussianNB
from sklearn.svm import SVC
from sklearn.externals import joblib

# Load dataset
path = "C:\\Users\\cz\\Downloads\\Daily files\\Machine Learning\\IrisGenerated.csv"
names = ['sepal-length', 'sepal-width', 'petal-length', 'petal-width', 'class']
dataset = pandas.read_csv(path, names=names)

# Split-out validation dataset
array = dataset.values
X = array[:,0:4]
Y = array[:,4]
validation_size = 0.20
seed = 7
X_train, X_validation, Y_train, Y_validation = model_selection.train_test_split(X, Y, test_size=validation_size, random_state=seed)

# Test options and evaluation metric
seed = 7
scoring = 'accuracy'

# Fit the model
model = KNeighborsClassifier()
model.fit(X_validation, Y_validation)

# save the model to disk
filename = 'finalized_model.sav'
joblib.dump(model, filename)

# load the model from disk
loaded_model = joblib.load(filename)
result = loaded_model.score(X_validation, Y_validation)
print(result)

detailed_result = model.predict(X)

print(detailed_result)

import csv
fname = 'output.csv'
with open(fname, 'w', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(detailed_result)

Open in new window

0
peprCommented:
When having a list of strings, you can just open file for writing, and then join the list to multiline string using \n, and write the result to the file. Like this:
fname = 'output.txt'
with open(fname, 'w') as f:
    f.write('\n'.join(detailed_result))

Open in new window

0
csehzIT consultantAuthor Commented:
Thank you very much, for this particular case your last code works perfectly, but also through the others I learnt
0
peprCommented:
You are welcome. :)

If you know that your actual case IS the CSV file with a single column, and you will want to add more columns later, you can still use the csv.writer with the loop through the detailed_result. You know that the detailed_result is the list of strings. You can easily convert a string to a single element list by enclosing the string variable into square braces. In other words, you can construct the row on the fly like this:
import csv
fname = 'output.csv'
with open(fname, 'w', newline='') as f:
    writer = csv.writer(f)
    for a_string in detailed_result:
        row = [a_string]    # converting a string into a single element list with that string as the element
        # Later, you may want to modify it like ... row = [a_string, 'the other column value', 5]
        writer.writerow(row)

Open in new window

0
csehzIT consultantAuthor Commented:
Thank you this one also works great, you are so helpful
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Python

From novice to tech pro — start learning today.