Extraction of data from Notepad file and calculating the average in real-time as it is generated

Best post one such text file (if nothing valuable or confidential is in the file). Then tell us what data you need extracted.
Probably some scripting, maybe in Excel is what's best.

I need to extract the average of the last column  (3rd column) over a time duration of 1 minute.
The 2nd column is the timestamps based on a parent application timestamps.

I have attached the .txt file.
2017-09-18T11_28_19.txt

Assuming the time stamps are in seconds:

start:
52335 41344.767 12.70
end:
2532 41388.057 22.80

This file is 44 seconds?

But what to do if a file is 5 minutes. You want the output of 5 averages?

For these solutions, I usually write some VB.net code. That way you can drag and drop the text file in the program you made.
It's free btw: https://www.visualstudio.com/thank-you-downloading-visual-studio/?sku=Community&rel=15#
Or you want a more basic solution (VBS, powershell)?

Hello,
I uploaded a test file of the data I gathered.
Yes, the duration of the HRV measurement was 44 seconds.
I will be conducting an experiment for a duration of 30 minutes and 15 minutes. I need to selectively take averages of the HRV values (third column) for first 8 minutes, from 10-18 minutes and 20-28 minutes. This is the duration, the participant interacts with another component of the experiment.
I am not good at coding, so yes, a program which would enable me to drop my text file in and get the averages for those specific durations would help me a lot.  Anything simple would be good. I have not used such solutions earlier so any assistance would be appreciated.

Download Visual Studio, install, create VB.net project with Windows Form.
Form, enable drag drop
Add textbox, allow multiline, vert toolbar, then add this code:

Imports System.IO

Public Class Form1

    Public Class rec_value

        Public minute As Integer
        Public sample As Integer
        Public avg As Double

    End Class


    Dim input_list As String()
    Dim TempFiles As String
    Dim start_time As Double
    'Public avg_values As List(Of KeyValuePair(Of Double, Double)) = New List(Of KeyValuePair(Of Double, Double))
    Public avg_values As List(Of rec_value) = New List(Of rec_value)

    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load

    End Sub

    Private Sub Form1_DragDrop(sender As Object, e As DragEventArgs) Handles MyBase.DragDrop


        If e.Data.GetDataPresent(DataFormats.FileDrop) Then
            Dim MyFiles() As String
            ' Assign the files to an array.
            MyFiles = e.Data.GetData(DataFormats.FileDrop)
            ' Loop through the array and add the files to the list.
            If MyFiles.Length > 1 Then
                MsgBox("You selected too many files!")
            Else
                TempFiles = MyFiles(0)

                Dim sr As New StreamReader(TempFiles)
                Dim input_text = sr.ReadToEnd
                sr.Close()
                sr = Nothing
                input_list = Split(input_text, vbCrLf)
                start_time = get_time(input_list(0))
                'MsgBox(start_time.ToString)
                For Each foundline In input_list
                    If foundline <> "" Then
                        add_value(get_minute(get_time(foundline)), get_value(foundline))
                    End If
                Next

                For Each foundvalue In avg_values
                    TextBox1.AppendText("Minute " & foundvalue.minute & " has average " & foundvalue.avg & vbCrLf)
                Next


            End If
        End If

    End Sub

    Private Sub Form1_DragEnter(sender As Object, e As DragEventArgs) Handles MyBase.DragEnter
        If e.Data.GetDataPresent(DataFormats.FileDrop) Then
            e.Effect = DragDropEffects.All
        End If
    End Sub

    Function get_time(input As String) As Double
        get_time = Double.Parse(Split(input, " ")(1))
    End Function


    Function get_value(input As String) As Double
        get_value = Double.Parse(Split(input, " ")(2))
    End Function

    Function get_minute(input As Double) As Integer
        get_minute = Math.Floor((input - start_time) / 60)
    End Function

    Sub add_value(minute As Integer, value As Double)
        Dim found As Boolean = False
        For i = 0 To avg_values.Count - 1 Step 1
            If avg_values(i).minute = minute Then
                avg_values(i) = New rec_value With {.minute = minute, .sample = avg_values(i).sample + 1, .avg = ((avg_values(i).avg * avg_values(i).sample) + value) / (avg_values(i).sample + 1)}
                found = True
            End If
        Next
        If Not found Then
            avg_values.Add(New rec_value() With {.minute = minute, .sample = 1, .avg = value})
        End If
    End Sub
End Class

Select allOpen in new window

It works by dragging and dropping a text file with the info you provided earlier.
Since your example only had 44 seconds, the output is only one line:

Minute 0 has average 17.3381538461538

Also, it will return ALL minutes, not just 8 minutes, from 10-18 minutes and 20-28 minutes. You'll just have to ignore the extra data, and select what you need. If you find that my program is working perfectly, we might think about adjusting the output to really only have those minutes.

Hello,
It took me a while as I needed to install and follow your instructions on Visual Basic. I have no experience working on VB. Luckily, few tutorials and online search helped me get to a stage where I dropped the text file and got some averages and it made sense.
I have attached the text file which has data from 60+ minutes. If you could help me out with creating a change in the VB script for calculating averages for the desired duration, I would be really grateful.

Minutes   Task
    0-1       Silence - No Average
    1-8        Average for 8 minutes.  
   9-10      Silence - No Average.
   11-18    Average for 8 minutes.  
   19-20    Silence - No Average.
   21-28    Average for 8 minutes.  
   29-30     Silence - No Average.

This is how i need to take the average of the HRV data from the text file.
2017-09-25T11_06_31.txt

Imports System.IO

Public Class Form1

    Public Class rec_value

        Public minute As Integer
        Public sample As Integer
        Public avg As Double
        Public test_string As String
    End Class


    Dim input_list As String()
    Dim TempFiles As String
    Dim start_time As Double
    'Public avg_values As List(Of KeyValuePair(Of Double, Double)) = New List(Of KeyValuePair(Of Double, Double))
    Public avg_values As List(Of rec_value) = New List(Of rec_value)

    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
    End Sub

    Private Sub Form1_DragDrop(sender As Object, e As DragEventArgs) Handles MyBase.DragDrop


        If e.Data.GetDataPresent(DataFormats.FileDrop) Then
            Dim MyFiles() As String
            ' Assign the files to an array.
            MyFiles = e.Data.GetData(DataFormats.FileDrop)
            ' Loop through the array and add the files to the list.
            If MyFiles.Length > 1 Then
                MsgBox("You selected too many files!")
            Else
                TempFiles = MyFiles(0)

                Dim sr As New StreamReader(TempFiles)
                Dim input_text = sr.ReadToEnd
                sr.Close()
                sr = Nothing
                input_list = Split(input_text, vbCrLf)
                start_time = get_time(input_list(0))
                'MsgBox(start_time.ToString)
                For Each foundline In input_list
                    If foundline <> "" Then
                        add_value(get_minute(get_time(foundline)), get_value(foundline))
                    End If
                Next

                For Each foundvalue In avg_values
                    'debug data only
                    'TextBox1.AppendText("Minute " & foundvalue.minute & "-8 has average " & foundvalue.avg & vbCrLf & foundvalue.test_string & vbCrLf)
                    TextBox1.AppendText("Minute " & foundvalue.minute & "-8 has average " & foundvalue.avg & vbCrLf)
                Next


            End If
        End If

    End Sub

    Private Sub Form1_DragEnter(sender As Object, e As DragEventArgs) Handles MyBase.DragEnter
        If e.Data.GetDataPresent(DataFormats.FileDrop) Then
            e.Effect = DragDropEffects.All
        End If
    End Sub

    Function get_time(input As String) As Double
        get_time = Double.Parse(Split(input, " ")(1))
    End Function


    Function get_value(input As String) As Double
        get_value = Double.Parse(Split(input, " ")(2))
    End Function

    Function get_minute(input As Double) As Integer
        get_minute = Math.Floor((input - start_time) / 60)
    End Function

    Sub add_value(minute As Integer, value As Double)
        If ((minute Mod 10) > 0) And ((minute Mod 10) < 9) Then

            minute = minute - (minute Mod 10) + 1

            Dim found As Boolean = False
            For i = 0 To avg_values.Count - 1 Step 1
                If avg_values(i).minute = minute Then
                    avg_values(i) = New rec_value With {.minute = minute, .sample = avg_values(i).sample + 1, .avg = ((avg_values(i).avg * avg_values(i).sample) + value) / (avg_values(i).sample + 1), .test_string = avg_values(i).test_string & "/val=" & value & "/avg=" & ((avg_values(i).avg * avg_values(i).sample) + value) / (avg_values(i).sample + 1).ToString}
                    found = True
                End If
            Next
            If Not found Then
                avg_values.Add(New rec_value() With {.minute = minute, .sample = 1, .avg = value, .test_string = "/val=" & value})
            End If


        End If
        
    End Sub
End Class

Select allOpen in new window

Adding excel so you can see calculated output is correct. (only first 2 calculated sections shown). Scroll to cell F634 and F1308 to check, it's the same as the output of my program.
input.xlsx

Your solution has been working fine for now.
I was busy with my conducting experiments and data gathering for more than a month.
I have been using the VB file you gave to get an average for every minute.
But if you can change it a bit like stated earlier for every 8 minutes and 2 minutes. I would really reduce the data entry work for me.

Thanks in advance.

It was already as you specified:

0-1       Silence - No Average
    1-8        Average for 8 minutes.  
   9-10      Silence - No Average.
   11-18    Average for 8 minutes.  

Didn't you even test or check the excel?

Hello,
I did check the excel file.  For now, from VB, I need to scroll from the Window to get the average for every minute and make a manual entry for every minute. Instead, I was looking forward to a table for 30-31 minutes with averages for every minute. I am unaware if that is possible. I hope you can understand what I am requesting for.

Thank you very much. I was missing the Multiline, that was creating the problem all this while.