Solved

Remove rows from csv file based on date column

Posted on 2013-11-18
14
443 Views
Last Modified: 2013-11-20
I have a csv file with columns below.  The second column is the only field that is an actual date.  What we would like to do is remove all rows from the csv file where the date in that column is < 3 days from today whenever run.  Don't have any special preference, but figured some type of bat file utilizing DOS commands, or VB script, powershell, etc.  We've used things like 'FINDSTR' etc in the past for certain strings, but not a date so not sure where to start.  Any help would greatly be appreciated.  
Columns:
PackageReference1,ShipmentInformationCollectiondate,ShipmentInformationActualWeight,ShipmentInformationLeadTrackingNumber,ShipmentInformationServiceType,ThirdPartyUPSAccountNumber
0
Comment
Question by:ibgadmin
14 Comments
 

Author Comment

by:ibgadmin
ID: 39657443
Forgot to provide a few actual rows of data....

PackageReference1,ShipmentInformationCollectiondate,ShipmentInformationActualWeight,ShipmentInformationLeadTrackingNumber,ShipmentInformationServiceType,ThirdPartyUPSAccountNumber
"15231622","11/6/2013","1.00","581089305388","3","444444444"
"15242251","11/6/2013","1.00","581089305399","92","555555555"
"15282587","11/16/2013","1.00","581089305403","92","555555555"
"15278493","11/18/2013","1.00","581089305414","92","444444444"
0
 
LVL 43

Expert Comment

by:Saqib Husain, Syed
ID: 39657558
Try

Sub Macro1()
    Range("A2").AutoFilter
    With ActiveSheet.AutoFilter.Range
        .AutoFilter Field:=2, Criteria1:=">" & Date - 3
        Application.DisplayAlerts = False
        On Error Resume Next
        .Offset(1).Resize(.Rows.Count - 1).SpecialCells(xlCellTypeVisible).Delete
        On Error GoTo 0
        Application.DisplayAlerts = False
    End With
    Range("A2").AutoFilter
End Sub
0
 

Author Comment

by:ibgadmin
ID: 39657585
hello - is this vbs or powershell, etc.?
0
 

Author Comment

by:ibgadmin
ID: 39657592
I see now this looks to be MS Excel macro.  How would I invoke that w/o actually running Excel?  I need to automate this process through a bat file, etc. to read the same input file every day that deletes the records.
0
 
LVL 8

Expert Comment

by:stevepcguy
ID: 39657807
If I may jump in:

Yes, the code is visual basic, and can be run in Excel in seconds. If you double click on the file in Windows, it will open in Excel. you can assign the macro to a keystroke or a button, then let it fly. Click on save, and keep the same CSV format. Depending on size, I see this taking less than a minute or two in total.
0
 

Author Comment

by:ibgadmin
ID: 39659186
I need this to run fully unattended like through a bat file of sorts.  Is there some way to automate the input filename and code to run?
0
 
LVL 43

Expert Comment

by:Saqib Husain, Syed
ID: 39659305
Try this macro. I have not tested it.
Sub Macro1()
dim fnam as string
dim fpath as string
dim wb as workbook
set fpath = "C:\............" 'Put something here
fnam= dir(fpath & "\*.csv")
do while fnam<>""
set wb = workbooks.open(fpath & "\" & fnam)
    Range("A2").AutoFilter
    With ActiveSheet.AutoFilter.Range
        .AutoFilter Field:=2, Criteria1:=">" & Date - 3
        Application.DisplayAlerts = False
        On Error Resume Next
        .Offset(1).Resize(.Rows.Count - 1).SpecialCells(xlCellTypeVisible).Delete
        On Error GoTo 0
        Application.DisplayAlerts = False
    End With
    Range("A2").AutoFilter
    wb.close true
    fnam=dir
loop
End Sub 

Open in new window

0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 31

Expert Comment

by:Rob Henson
ID: 39659773
How is the csv file getting generated in the first place? Is it some sort of query extract?

If so can the query not be adjusted to not extract those dates so the csv file gets created with only the dates required.

Can you clarify what you mean by dates < 3 days from today?

Today + 3 days? Keep or lose.
Today - 3 days? Keep or lose.

Thanks
Rob H
0
 

Author Comment

by:ibgadmin
ID: 39659810
I get the error below...
C:\Users\jdunn\Desktop\TestFolder>REM ******************************************
***********************************

C:\Users\jdunn\Desktop\TestFolder>REM *                           remove lt 3 da
ys                                   *

C:\Users\jdunn\Desktop\TestFolder>REM *
            *

C:\Users\jdunn\Desktop\TestFolder>REM ******************************************
***********************************

C:\Users\jdunn\Desktop\TestFolder>cscript Lt3days.vbs
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Users\jdunn\Desktop\TestFolder\Lt3days.vbs(2, 10) Microsoft VBScript compilat
ion error: Expected end of statement


C:\Users\jdunn\Desktop\TestFolder>PAUSE
Press any key to continue . . .
0
 

Author Comment

by:ibgadmin
ID: 39659883
the file is like a log file.   i don't extract the records. i would like to lose all records lt todays date.  hope that helps.
0
 
LVL 43

Expert Comment

by:Saqib Husain, Syed
ID: 39659999
I do not know how and where you have got those errors. What I have provided is a VBA macro which you have to run from within excel. To run the macro

Open a new file in excel
Right-click on the sheet tab name
Select View code
From the menu      Insert > Module
Paste the given code in the VBA window
Close the VBA window
Press Alt-F8
Select the macro name
Click on run
0
 

Author Comment

by:ibgadmin
ID: 39660028
an excel macro will not work as it will need to run unattended from a windows server.  anyone have just vbs or powershell, etc. that could be used?
0
 
LVL 43

Expert Comment

by:Saqib Husain, Syed
ID: 39660042
In that you need to ask this question in some other topic area. This is excel zone.
0
 
LVL 34

Accepted Solution

by:
Dan Craciun earned 500 total points
ID: 39663442
This is the answer from the other thread, for reference. Powershell script.

Param([Int32]$history = 3,                 # number of days to keep
      [string]$log = "Y:\path\to\log.csv") # path to log file
$dte = Get-Date             # current date
$searchstring = ""          # 
$period = @(0) * $history   # array to store formatted dates

for ( $i = 0; $i -lt $history; $i++ ) {
  $period[$i] = $dte.Month, $dte.Day, $dte.Year -join "\/"
  $searchstring += "\|" + $period[$i]
  $dte = $dte.AddDays(-1)
}
$searchstring = $searchstring.Substring(2) # remove first \| 

&"X:\path\to\sed\sed.exe" -i -e1p -e `""/"$searchstring"/!d"`" $log

Open in new window

Replace path\to\sed, save it as whatever.ps1, then run it as
whatever.ps1 -history 5 -log "Y:\path\to\log.csv"

Also, for the script to work, you'll need sed: http://gnuwin32.sourceforge.net/packages/sed.htm .
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Sparklines have been introduced with Excel 2010 and are a useful tool for creating small in-cell charts, used for example in dashboards. Excel 2010 offers three different types of Sparklines: Line, Column and Win/Loss. What it does not offer is a…
Drop Down List with Unique/Distinct Values (enhancing the Combo-Box with a few steps and a little code) David miller (dlmille) Intro Have you ever created a data validation list from a database field or spreadsheet column (e.g., Zip Codes or Co…
Viewers will learn the basics of slicers and timelines for both PivotTables and standard Excel tables in Excel 2013.
The viewer will learn how to create a normally distributed random variable in Excel, use a normal distribution to simulate the return on an investment over a period of years, Create a Monte Carlo simulation using a normal random variable, and calcul…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now