Solved

SAS-Update history records type 2

Posted on 2012-03-14
5
305 Views
Last Modified: 2012-03-30
Hi,
We need to fix some history data right

This is type 2 dimension we have issue with first 2 rows


VALID_FROM_DTTM          VALID_TO_DTTM        emp_id  emp_key
26AUG2011:23:59:59      12OCT2011:16:26:56     101     1
10OCT2011:23:59:59      07NOV2011:23:59:58     101     201
07NOV2011:23:59:59      12DEC2011:23:59:58    101      302
12DEC2011:23:59:59      13DEC2011:23:59:58    101      801
13DEC2011:23:59:59      22DEC2011:23:59:58    101      10001
22DEC2011:23:59:59      23DEC2011:23:59:58    101      1000005
23DEC2011:23:59:59      25DEC2011:23:59:58    101     1000008
25DEC2011:23:59:59      27DEC2011:23:59:58    101     10000011
27DEC2011:23:59:59      01JAN2012:23:59:58   101     10000013
01JAN2012:23:59:59      02JAN2012:23:59:58   101     10000022
02JAN2012:23:59:59      20JAN2012:23:59:58   101     10000045
20JAN2012:23:59:59      22JAN2012:23:59:58   101     10000067
22JAN2012:23:59:59      31DEC9999:00:00:00   101     100000987


Isuue is Second row VALID_FROM_DTTM should be 12OCT2011:16:26:57
so it should looks like

26AUG2011:23:59:59      12OCT2011:16:26:56     101     1
12OCT2011:16:26:57      07NOV2011:23:59:58     101     201

We have issue with some other emp_id too so we want to update
VALID_FROM_DTTM where VALID_TO_DTTM  <> to VALID_TO_DTTM -1 sec
based upon emp_id
0
Comment
Question by:sam2929
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
5 Comments
 
LVL 14

Expert Comment

by:Aloysius Low
ID: 37723494
you appear to know what your problem is already, and your solution, so you are looking for the code to implement your solution?

anyway have you tried loading duplicate records at initial load for example? I remembered encountering this problem when I did this some time back, the SCD Type2 Generator has a code which will adjust the date/datetime if it detects duplicate records being loaded...

i would do something like:
proc sort data = [input]; by EMP_ID VALID_FROM_DTTM; run;

data [output];
  set [input];
  by EMP_ID;
  retain PREV_TO_VAL;
  if first.EMP_ID then do;
    PREV_TO_VAL = VALID_TO_DTTM;
  end;
  else do;
    VALID_FROM_DTTM = PREV_TO_VAL + 1;
  end;
run;

as usual, please do back up your original data so that it can be restored if need be. also, please do test the code and check the output to ensure that this is what you really wanted
0
 
LVL 11

Expert Comment

by:theartfuldazzler
ID: 37723800
Hi

Iowa's code will work with a slight change - one needs to reset the value of PREV_TO_VAL:

proc sort data = [input]; by EMP_ID VALID_FROM_DTTM; run;

data [output];
  set [input];
  by EMP_ID;
  retain PREV_TO_VAL;
  if first.EMP_ID then do;
    PREV_TO_VAL = VALID_TO_DTTM;
  end;
  else do;
    VALID_FROM_DTTM = PREV_TO_VAL + 1;
     PREV_TO_VAL = VALID_TO_DTTM;
  end;
DROP PREV_TO_VAL;
run;

Open in new window


I actually prefer using the LAG function for these types of code:

proc sort data = [input]; by EMP_ID VALID_FROM_DTTM; run;

DATA [output];
   set [input];
  by EMP_ID;
   IF NOT First.EMP_ID THEN
      VALID_FROM_DTTM = LAG(VALID_TO_DTTM) + 1;
RUN;
      

Open in new window

0
 
LVL 14

Expert Comment

by:Aloysius Low
ID: 37723814
oh yes, theartfuldazzler thanks for pointing that out :)
0
 

Author Comment

by:sam2929
ID: 37726146
can't we modify this code to do changes just where
where VALID_TO_DTTM  <> to VALID_TO_DTTM -1 sec
0
 
LVL 14

Accepted Solution

by:
Aloysius Low earned 500 total points
ID: 37727494
yes, you can, but it will depend on the underlying value... if there's decimals, although not displayed, the value may not match and hence every value will be updated...

for further understanding, SAS stores date and datetime values as numbers. however, these are not entire whole numbers, but decimals are involved as well. hence, even if we were to put a check for VALID_FROM_DTTM <> to VALID_TO_DTTM -1, it might end up all values being updated, although you might not see it.

to add in this condition, do it within the else do statements:
else do;
    if VALID_FROM_DTTM + 1 ne PREV_TO_VAL then
        VALID_FROM_DTTM = PREV_TO_VAL + 1;
    PREV_TO_VAL = VALID_TO_DTTM;
end;
0

Featured Post

Free Tool: Postgres Monitoring System

A PHP and Perl based system to collect and display usage statistics from PostgreSQL databases.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Azure Functions is a solution for easily running small pieces of code, or "functions," in the cloud. This article shows how to create one of these functions to write directly to Azure Table Storage.
This article shows the steps required to install WordPress on Azure. Web Apps, Mobile Apps, API Apps, or Functions, in Azure all these run in an App Service plan. WordPress is no exception and requires an App Service Plan and Database to install
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

740 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question