i would look like ro analyse/describe data that is big over 300,000,000 rows do you python can be able to help me do this. how efficiently can i do thos

Hi everyone!
I am trying to do a few things in my bash script (script1.sh):  pass logicals/variables to my SAS program (program1.sas), execute SAS code (program1.sas), check with "errstat" for specific errors related to SAS code conditions that I set up in my SAS code, and also check with "logerrchk" SAS log file for any errors in the SAS log.  
I think most of the script is correct but I am not sure about both checking errors (errstat" and "logerrchk").

 Please note: variable/logicals (file1, file2, plog, and errstat) are also defined in the SAS program.  

 Please provide any suggestions or examples if you can.
 Here is what I have so far:

#!/bin/bash
#script1.sh

# Creating variables for directories
export DATADIR1=/root/alldirs/2018/data1
export DATADIR2=/root/alldirs/2018/data2
export DATADIR3=/root/alldirs/2018/data3
export PROGDIR1=/root/alldirs/2018/prog1

#Creating variables for data files that are used in program1.sas and final log file 
export file1=${DATADIR1}/sasdata1.sas7bdat
export file2=${DATADIR2}/sasdata2.sas7bdat
export plog=${DATADIR3}/sas_program.log

#executing SAS program
sas ${PROGDIR1}/program1.sas -log ${DATADIR3}/program1.log

#checking for errors in SAS log file
logerrchk=$?     
   if [[ "$logerrchk" !=0 ]]; 
   then 
       echo "--- Error: Abnormal end in program program1.sas. Check log file! ---" >> $plog
       exit 2
   else
       echo "--- Program1.sas run successfully! ---" >> $plog
   fi
  
#checking for specific errors 

Select all Open in new window

…

Need to convert below sas code to pseudocode.

sum ((case when (mod_r in ('02') and CAL IN ('P') and fec_in <= intnx('month', "PAR_FEC"d, -1, 'end') and EST not in ('7' '07' ' 7' '7 ' '70')
              and temp>0) then temp*SAL_OBL else . end)) as con_temp.

How can I write an export command which gives me the same result as a manual export in SAS. When I do the manual export each record is on a separate line.  When I try using Proc Export my output is one continuous line.  I am trying to output a comma delimited text file.

I need help with writing an explicit pass through query in SAS eg.

I am two tables from the test schema, test.tableA and test.tableB.   TableB is a huge table and I need to pull 5 records from tableB  where they match in tableA.  I am thinking that an explicit pass through would be the best way to execute this because of the run time.  So from tableB, I need sample_A, sample_B, sample_C, sample_D, sample_E.  The sampleA column is a date field and I would need to select a particular day to shorten the run time and the result set.  From tableA, I need columnT, columnX, columnAA.  In tableB, the sampleE column is the primary key and in tableB, columnX is the key that can be used to join on.

Any help is appreciated.

I have created two sheets with date field, wrkorder no , hours and invoice count of technicians. One sheet shows day and other shows week numbers. The problem I’m facing is in the weeks sheet where even though the wrkorder happened twice in two dates the hours are adding up and sum of hours for each work order is showing. The total hours are accurate thought but invoicecount is getting wrong. Since a wrkorder happened in both days it is counted  as two but it showing me one.

Hello everyone,
I am trying to create XML record counts in the XML data file header.
Below code creates a XML data file "FinalXMLDataFile" from a SAS dataset "NewData" and the code works fine except counts are not working at all. Current record counts "previousCount" creates "0" counts in the XML Header in the counts section.  

Please let me know where the issue with record counts might be?

libname   mydir '/mydirectory'; 
filename outxml  '/mydirectory/FinalXMLDataFile.xml';

/* Macro variables to provide appropriate tab spacing */
%let tab="   ";
%let tab2="      ";
%let tab3="         ";
%let tab4="            ";
%let previousCount = 0; 


/*XML file information and headers*/
data _null_;
    file outxml;

if _N_ = 1 then
    do;
    
    put '<?xml version="1.0" encoding="UTF-8"?>';
    put '<XMLFile >';	
    put &tab. '<Header>';  
    put &tab2. "<CaseCount>%trim(&previousCount)</CaseCount>";     /* get record count of all XML records */ 
    put &tab2. '<Year>2017</Year>';
    put &tab. '</Header>';
    put &tab. '<Payload>';
    end;
run;

/* Using SAS dataset to create similar XML data file */
data _null_;
    file outxml mod;
    set mydir.NewData NOBS=lst;

    record = _N_ + &previousCount;
    
    if _n_=1 then do;
        put &tab2. '<MasterFile>'; 
    end;

    /*All Data records */
    put &tab3. '<AllData>'; 
    put &tab4. '<unit:VAR1>' VAR1+(-1)'</unit:VAR1>';
    put &tab4. '<unit:VAR2>' 

Select all Open in new window

…

sas date in teradata

date1
20171101
20171101
How do you convert a date column stored as sas date value in teradata to a sas date format using proc sql? I am getting an error message.

Here is my code:

   

  proc sql;
  Create table test
   AS
   select cast(cast(date1 as date format 'YYYY/MM/DD') as char(10)) as 'date_d'
   from td2.db;

Select all Open in new window

Why Statistics is important for Data Analysis?
What is the role of Statistics in Data Analysis?

*Please share a real life example.*

I am trying to embed automatically today's date (DDMMYYD10.) and XML records counts in the XML data file header. Is there an automatic way in SAS how to do that?

This is where today's date and record counts would go:
 

put &tab2. '<SampleDate>2017-06-27</SampleDate>';
 put &tab2. '<CaseCount>10</CaseCount>';

Select all Open in new window

Full SAS code:

filename DirectX  "&basepath\examples\XMLData2.xml";
filename Complete "&basepath\examples\XMLComplete2.xml";

libname xmldir xmlv2 xmlfileref=DirectX xmltype=generic;

/* Macro variables to provide appropriate tab spacing */
%let tab="    ";
%let tab2="        ";

/* Writing two SAS datsets to xmldir library and creating XMLData.xml data file */ 
data xmldir.class(label="dataset class from sashelp library");
       set sashelp.class(obs=10);
run;

data xmldir.cars(label="dataset cars from sashelp library");
       set sashelp.cars(obs=10); 
run; 

libname xmldir clear;

/*  XML file information and header and XML File End*/
data _null_;
 infile	DirectX end=last;
file	Complete;

input;				
 
if _N_ = 1 then
    do;  
	put _infile_;	
    put '<Schema Name="CarsAndClass">';
    put &tab. '<Header>';
    put &tab2. '<SampleDate>2017-06-27</SampleDate>';
    put &tab2. '<CaseCount>10</CaseCount>';
    put &tab2. '<Action>Initial</Action>';
    put &tab2. '<YearPeriod>2017</YearPeriod>';
    put &tab. '</Header>';
    put &tab. '<Payload>';
    end;
else
	put &tab2 _infile_;	
 
if last then
    do;
    put &tab'</Payload>';		
    put '</Schema>';
    end;    
run;

Select all Open in new window

Hello All Experts,
I am a student enthusiast in learning "Data Analytics" , which is the best platform to learn for FREE?
I want to Learn 'Data Science (Statistics)' & 'SAS/R' from scratch?
Any videos? Any websites? Any Blogs?

Thanks,

Regards,
Satish Kumar G N

Hello,
For testing and verification purposes, I am trying to compute the count of good duplicates found in both Data1 and Data2  and in Data3, and in one or the other. These are the steps I am trying to accomplish to get to my final results:

#1.  Create a list (Data4) of unique Data1 and Data2 where variable rec_flag=A, B, C, D.
#2.  Build a duplicate (Data5) file from the list in #1 by generating a record linking each two records where Var3 and Var4 are linked to the same Var6 and Var7.
#3.  Match Data5 file with Data3 where variable rec_flag=B, C, D using key variables Var3, Var4, Var6, and Var7.
#4.  Display in the output .lis the number of records in:
     #4.1 Both Data5 and Data3
     #4.2 Data5 only
     #4.3 Data3 only
 
The issue that I am having is with #2 where I need to build a duplicate file Data5.

Here is my SAS code below with missing step #2:

libname mydir "/home/myprofile/links"; 
%macro cnt_lnks;

 /*  #1 */
proc sort data=mydir.Data1
          out=Data1out(where=(rec_flag='A', 'B', 'C', 'D') keep=Var3 Var4 Var6 Var7 rec_flag);
          by Var3 Var4 Var6 Var7;
run;

proc sort data=mydir.Data2
          out=Data2out(where=(rec_flag='A', 'B', 'C', 'D') keep=Var3 Var4 Var6 Var7 rec_flag);
          by Var3 Var4 Var6 Var7;
run;

/*********************************************
 Datastep: merge Data1 & Data2
           by Var3 Var4 Var6 Var7
 ********************************************/
data all_links1;
  merge 

Select all Open in new window

…

hi,

both SAS and IBM Cognos is analytical SW, any diff between them? I am wondering why company use both?