search HL7 message in vb.net and print the entire message based on search.

Hello experts,

I am trying to write a vb.net application that will read a text file that has several hl7 messages in it, search for keywords and then print the entire hl7 message based on getting hits from those key words.  So far my code I can read in the text file and using  line.IndexOf and  I can find each segment that has the keyword I am looking for and I can print that entire line, but I am having trouble putting the logic together to print the entire hl7 message.  I bring everything in using streamreader and the doing a Do while loop.   I am not sure how or if I am supposed to compare the values in the array that is creating to go from one msh segment to another -1.

Below is a snippet of the code I have, I could really use the help.

Thanks in advance!

        Dim disList As String() = {"hepatitis", "HCV RNA", "HCV RNA", "HCV screening test with signal-to-cutoff", "HCV Genotype", "ALT", "AST", "Bilirubin", "Hepatitis B Surface Antigen", "Hepatitis B virus Core Ab IgM", "Hepatitis B Core Ab Total", "Hepatitis B e Ab", "Hepatitis B virus DNA", "Hepatitis B e antigen", "ALT", "AST", "Bilirubin", "Rubeola Antibodies", "Viral culture", "Measles PCR", "Bordetella pertussis PCR", "Bordetella pertussis Culture", "B pertussis", "B. Pertussis", "Rubella virus", "Rubella PCR", "Rubella viral blood culture", "Virus identified Culture", "Mumps Viral Culture", "Mumps Antibodies", "Mumps PCR", "Blood Culture", "Histopath tes", "Blood Culture", "Blood Culture", "PCR", "latex agg", "immunohisto", "Culture from sterile site", "H flu PCR", "Culture from sterile site", "Lyme IgG", "Lyme IgM", "Lyme antibody", "IgG IFA", "IgG EIA", "IgM EIA", "Parasite Exam", "Plasmodium sp. PCR", "Plasmodium sp. PCR", "Babesia microti PCR", "Babesia microti IgM", "Babesia microti IgG", "DNA", "DNA", "DNA", "IgM", "Ab IgG", "Ab. IgM", "Calif Enc Virus IgM", "IgM", "PCR", "IgG", "PCR", "Culture", "Immunohistochemistry", "Culture", "Serology", "IFA", "Antibody IgM", "IgG", "PCR", "Giardia Antigen EIA", "O&P", "Campy Antigen EIA", "Culture", "Leg Penumo Abs", "Urine antigen", "Culture", "IgM antibodies", "Shiga toxin EIA", "Culture", "PCR", "Culture", "culture", "EIA", "culture", "PCR", "PCR", "Respiratory Virus Panel", "Viral Culture", "Respiratory Virus Panel", "Organism name", "Antibiotic susceptibility pattern", "Vuiral culture", "sedimentation rate", "PCR", "Hepatitis C", "Hepatitis B", "Measles", "Pertussis", "Rubella", "Mumps", "Diphtheriae", "Group B", "Neisseria meningitidis", "H.flu", "Strep pneumo", "Lyme Disease", "Rocky Mtn Fever", "Malaria", "Babesia", "Ehrlichia/Anaplasmosis", "Lacrosse", "West Nile", "Q Fever", "Tularemia", "Brucella", "Giardiasis", "Campylobacteriosis", "Legionellosis", "Hepatitis A", "STEC", "Salmonella", "Parainfluenza", "Human metapneumo virus", "Rotavirus", "Vibrio", "Enterovirus", "Influenza", "RSV", "CRE", "Psittacosis", "Varicella-Zoster Virus"}
        For Each items As String In disList

            Using r As StreamReader = New StreamReader(strSelect)
                ' Store contents in this String.
                Dim line As String
                ' Read first line.
                line = r.ReadLine
                Dim hlMessage As New List(Of String)
                ' Loop over each line in file, While list is Not Nothing.
                Do While (Not line Is Nothing)
                    ' Add this line to list.
                    list.Add(line)

                    If line.IndexOf("msh|", 0, StringComparison.CurrentCultureIgnoreCase) > -1 Then
                        ' The string you are searching.
                        Dim str As String = line
                        ' Find index of msh| segment 
                        Dim i As Integer = str.IndexOf("msh|", StringComparison.CurrentCultureIgnoreCase)
                        ' This new string contains the substring starting at msh|
                        Dim mshTrim As String = str.Substring(i)
                        ' See new string
                        msh = mshTrim.ToString
                        counter = 1
                    End If

                    ' Display to console.
                    If line.IndexOf(items, 0, StringComparison.CurrentCultureIgnoreCase) > -1 Then
                        output = My.Computer.FileSystem.OpenTextFileWriter(strSelect2 & "\HL7log.txt", True)
                        If counter = 1 Then
                            Dim msh2 As String
                            msh2 = msh
                            output.WriteLine(" ")
                            output.WriteLine("=========================================================")
                            output.WriteLine(items)
                            output.WriteLine("=========================================================")
                            output.WriteLine(" ")
                            output.WriteLine(msh2)
                            counter = 0
                        End If
                        output.WriteLine(line)
                        output.Close()
                    End If
                    ' Read in the next line.
                    line = r.ReadLine
                Loop
            End Using
        Next

Open in new window

thomashospitalAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Fernando SotoRetiredCommented:
Can you provide samples of what a HL7 message looks like so that we can see how we can help you.
thomashospitalAuthor Commented:
Here is an example of an HL7 message.  The program that I have wrote so far, reads a text file with multiple messages such as these line by line.  They always start with MSH| and I am trying to figure out basically how to go from one MSH to another and to "print" the data per message.  However from one MSH to another there can be X amount of lines depending on the information sent in the message.  Sometimes you can have a message with only 10 lines, sometimes you can have a message with 50 lines.  Thanks for the reply!

EDIT: I attached a sample .txt file so you can see the formatting better than pasting in the comment box
hl7Sample.txt
Fernando SotoRetiredCommented:
Hi thomashospital;

Here is some sample code that should do what you need.
'' Load the HL7 messages into a String variable. Note that if the file is not
'' in the same directory you must supply the complete path as well
Dim hl7File As String = File.ReadAllText("hl7Sample.txt")
'' Create a list of search words that each HL7 message must contain one of.
'' Sample list of search words
Dim keyWords As New List(Of String) From {"CPT-4", "TRIGLYCERIDES", "CHOLESTEROL"}
'' Regex pattern to parse a complete HL7 message
Dim pattern As String = "(MSH(?:.+\r\n(?!MSH))+)"
'' This list of HL7 messages are the ones you want  
Dim result As New List(Of String)
'' Regex collection of HL7 messages
Dim mc = Regex.Matches(hl7File, pattern)
'' Iterate through the HL7 messages to find the ones you want
For Each m As Match In mc
    For Each word As String In keyWords
        If m.Value.ToUpper().Contains(word.ToUpper()) Then
            result.Add(m.Value)
        End If
    Next
Next

'' The result list is the HL7 messages you are looking for.

Open in new window

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Microsoft Azure 2017

Azure has a changed a lot since it was originally introduce by adding new services and features. Do you know everything you need to about Azure? This course will teach you about the Azure App Service, monitoring and application insights, DevOps, and Team Services.

thomashospitalAuthor Commented:
Thank you for the response Fernando, It looks like I am almost there.  Your code example will produce a result similar to what I need when the incoming text file is one to a couple of those messages. However, the text files that are generated have hundreds of those messages contained in it which contains thousands of lines.  When I used the same code and use it on a large text file, it simply does not work, and gives me no results.  So we are close.
Fernando SotoRetiredCommented:
Hi thomashospital;

I just created a text file with 2000+ records which ended up with 16000+ lines of text and ran the code I posted and it ran in 0.56 seconds to process.

To your statement, "When I used the same code and use it on a large text file, it simply does not work, and gives me no results. ", In order to proceed I need more then "it simply does not work".

Do you get any runtime exception? If yes please post the exception and inner exception messages.

Is the large text file in the same format as what you posted?

Can you post the hl7 message file that is causing the problem.

Please post how you implemented the code in your application.
thomashospitalAuthor Commented:
Sorry Fernando about being vague.  When I said it does not work, I mean it would yield no results(results | count = 0).  The large text file I am using has over 5500 line of text in it, all in the same format.  I can not post the file I am using because it has a mix of test data and patient data. I will send you a screenshot of what I am seeing.  Thank you for responding.
capturre1.png
Fernando SotoRetiredCommented:
Place the following line of code just after the current breakpoint in the screen shot and move the breakpoint to the For Each line and see what the count is in the new line of code.
Dim hl7Count = mc.Count()

Open in new window

thomashospitalAuthor Commented:
Hl7Count = 0
Fernando SotoRetiredCommented:
That tells me that not all HL7 records do not follow the same format. For example each record must start on a new line with the first three characters being MSH and may contain any number of additional lines with additional data until the start of the next record.
thomashospitalAuthor Commented:
That is correct. Maybe I did a bad job of explaining that. The first line will always be MSH with any number of additional lines afterwards depending on the data sent across.  I am sorry if I did not convey that correctly before.  I really appreciate any additional help you can spare.
Fernando SotoRetiredCommented:
I think I understood it correctly, for example if there are 100 HL7 records in the file you will have 100 lines that start with MSH and each one of those line may be followed by any number of information for that MSH record. The next record will start on a new line with the first three characters being MSH.
thomashospitalAuthor Commented:
That is correct. Any additional help is appreciated, I just can not figure out how to get this to work.
Fernando SotoRetiredCommented:
At this point I can not think of what else to have you do. In a case like this if I can not reproduce the problem I can not come up with a solution. I will keep looking at it and see if I can come up with something.
thomashospitalAuthor Commented:
Fernando thanks for all your help!  Your solution was correct after all and I figured out my mistakes.  What was happening was that I was  bringing the messages in to notepad just by copying and pasting them.  However, the original file for the hl7 message was on a unix box that I would ftp over.  Somehow even though I had FTP'd the file and copied the text out and pasted into a windows notepad, it changed the encoding of the windows text file, and vb.net was not able to read it correctly.  I noticed this when I pasted the original hl7 mesage into word and went to save it as text and was alerted via a selection box to choose the encoding that I wanted to use.  So that was my problem on why it was not being read.  I want to thank you Fernando for all of your help!!!
Fernando SotoRetiredCommented:
Not a problem thomashospital, glad it worked out for you. 👍
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Visual Basic.NET

From novice to tech pro — start learning today.