asked on
'Unpivot' in LINQ/C#
Hi, I have a similar situation to the question linked below. Basically I have a wide CSV file which looks like (basically):
Person, Status, Date, Answer1, Answer2 ... Answer 200.
Bob, Done, 1/1/10,5,3, 6,4,2...and so on
Joe, Done, 1/1/10,5,2,2,1,5...and so on
Most answers are numeric, but some questions allow text.
I'd like to end up with:
Person,Status,Date,AnswerN
Bob, Done, 1/1/10, 1, 5
Bob, Done, 1/1/10, 2, 3
and so on.
The volume of data is very manageable (maybe a few thousand respondents) and people will probably click an 'upload' button to import the result files- so I'd prefer to avoid SSIS/stored procedures if possible. Is there a method in LINQ, or a reasonably easy procedure in C# to rework this and append it to a database?
Many thanks for your thoughts!
https://www.experts-exchange.com/Microsoft/Development/MS-SQL-Server/SQL-Server-2005/Q_23464968.html?sfQueryTermInfo=1+unpivot
Person, Status, Date, Answer1, Answer2 ... Answer 200.
Bob, Done, 1/1/10,5,3, 6,4,2...and so on
Joe, Done, 1/1/10,5,2,2,1,5...and so on
Most answers are numeric, but some questions allow text.
I'd like to end up with:
Person,Status,Date,AnswerN
Bob, Done, 1/1/10, 1, 5
Bob, Done, 1/1/10, 2, 3
and so on.
The volume of data is very manageable (maybe a few thousand respondents) and people will probably click an 'upload' button to import the result files- so I'd prefer to avoid SSIS/stored procedures if possible. Is there a method in LINQ, or a reasonably easy procedure in C# to rework this and append it to a database?
Many thanks for your thoughts!
https://www.experts-exchange.com/Microsoft/Development/MS-SQL-Server/SQL-Server-2005/Q_23464968.html?sfQueryTermInfo=1+unpivot
.NET ProgrammingC#
Last Comment
Nathan08
ASKER
Thanks for the comment. The 'old' way I'd do this is to create an array and iterate through it, but the number of columns will vary and I get the feeling linq could do it a bit more elegantly than nested loops.
ASKER
By the way, I'd imagine I'll be using this CSV reader to bring them in- it's a great little tool.
http://www.codeproject.com/KB/database/CsvReader.aspx
It will let me iterate through the fields as needed, I hope. Or preferably LINQ, as mentioned!
http://www.codeproject.com/KB/database/CsvReader.aspx
It will let me iterate through the fields as needed, I hope. Or preferably LINQ, as mentioned!
Hi,
With LINQ what you can do is to create a Class to store the data from the csv and then query the csv using LINQ and return the data as a list of the class.
You can have a look at these article which can get you started
http://www.onedotnetway.com/tutorial-reading-a-text-file-using-linq/
http://www.codeproject.com/KB/linq/LINQtoCSV.aspx
Thanks and regards,
Venki
With LINQ what you can do is to create a Class to store the data from the csv and then query the csv using LINQ and return the data as a list of the class.
You can have a look at these article which can get you started
http://www.onedotnetway.com/tutorial-reading-a-text-file-using-linq/
http://www.codeproject.com/KB/linq/LINQtoCSV.aspx
Thanks and regards,
Venki
ASKER
Hi, thanks for that. I do understand the idea of each record being a class instance - ie one instance per row and each 'person' and their answers would be typed fields in the class. However then you need to output them iteratively into their normalised/unpivoted state. If you use a class structure and strong types etc, it will be hard for the number of questions to vary.
If there are 100 columns or 300, it should just convert all those to individual rows. I could code this up with arrays, my question is whether a set of queries and then a push to the DB is possible (and efficient!) in LINQ.
Thanks
If there are 100 columns or 300, it should just convert all those to individual rows. I could code this up with arrays, my question is whether a set of queries and then a push to the DB is possible (and efficient!) in LINQ.
Thanks
yes you are correct, I overlooked your question
ASKER CERTIFIED SOLUTION
THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
Create a simple datatable with number of cols same as number of cols in first row and then keep on adding all the rows as and you read the lines.