[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x

Shell Scripting

10K

Solutions

6K

Contributors

The term 'shell' refers to a general class of text-based command interpreters most often associated with the UNIX and Linux operating systems. Popular shells include Bourne, Debian Almquist (dash), Korn (ksh), Bourne Again (bash) and the C shell family (csh). Some view the DOS 'cmd' prompt as a minimal shell of sorts. It is also possible to install Cygwin on Windows and emulate a full Unix environment with complete shell capabilities. Terminal emulators, such as xterm, GNOME Terminal and OS X Terminal, can be used to access shell.

Share tech news, updates, or what's on your mind.

Sign up to Post

I'll need a Shell (Bash) script (rather an exact command) that outputs

a)  files' name in the Solaris system, one file per line in the UNIX systems
b) that were modified/created the last 1470 minutes
c) exclude FIFO files, symbolic links, sockets (ie *.sock)
d) names of files of between 1 byte to 20MB in size
e) files in /dev, /devices, /kernel, /cdrom, /platform, /proc, /net
f) files mounted on NFS

I have about 1million files so hoping the command/script of outputting
the file can complete in 30mins, so may need efficient coding.


I need to amend the following script to read (ie for AV to scan) the above output file:

#!/bin/bash
LOGFILE="/var/log/clamav/`hostname`-$(date +'%Y-%m-%d').log";
## suggest to change dirs below to root but exclude databases
DIRTOSCAN="/var /opt /home /etc /tmp /export";

for S in ${DIRTOSCAN}; do
DIRSIZE=$(du -sh "$S" |grep -v "/proc" |grep -v "/dev" |grep -v ...  /2>/dev/null | cut -f1);
## add to grep -v for any other file types to exclude

echo "Starting a daily scan of "$S" directory.
Amount of data to be scanned is "$DIRSIZE".";

clamscan -ri "$S" >> "$LOGFILE";
0
OWASP Proactive Controls
LVL 12
OWASP Proactive Controls

Learn the most important control and control categories that every architect and developer should include in their projects.

Dear Experts,

I am thinking to use SSIS to do the following procedure.

(1) copy many files to Linux machines. then
(2) execute a Linux program (command) to analysis those files. The program will output the result to the local Linux drive
(3) when (2) is done, then pull the result back to Windows machine

For (1) I am thinking of FTP task (by FTP or SFTP)

how about the task of (2) and (3) ?  Could you please help ... Thanks
0
I'll need to monitor several "privilege escalation related" Solaris 10 & RHEL6 files using
ACLs (Access Ctrl Lists) :

a) /etc/group, /etc/sudoers, /etc/cron.daily (or .weekly or any crons owned by root):
    ACL to send to syslog (so that we can pipe to SIEM) when permissions, ownership
    or contents of the above files are changed

b)visudo, sudo, usermod, useradd    command binary files :
   when these are being executed/run, ACL to send to syslog (who & when it's being
   executed)

Appreciate an exact  setacl (or the actual commands/settings in RHEL6 & Solaris 10
x86  samples
0
I am trying to get a specific pid and ONLY that pid, not others that might have that pid embedded in them, i.e.

345
1345
5345

I only want to get the 345.  If they would let me use Perl it would be easy, but I have to use ksh.

Any ideas?

Thanks!
David
0
Batch scripting

Experts,

I have run across an old dos batch script that utilizes something I have never seen.

The script is used to copy a file, but the reasons behind and the uses for the [ ] is what I am looking for.

copy h:filename.txt i:[Spreadsheet]\folder1\folder2

Open in new window

1
Hi Expert,

I have a SFTP job which is failing while transfer file for source to dest using "put" command.  As put command only works for single file.

Here my requirement is i want to transfer all the files available in "source/dir1/dir2" and want to merge them into single file then wanna SFTP.

Note:The script should also need to check how many files are there in the source directory before merge them!

thanks in advance.
0
Hi Expert,

How Bundle XML files together into a ZIP using shell script.

Your Comments will be really Appreciate.

Thanks,
0
Need to harden a Solaris 10 that is connecting to Internet  from DMZ.

Anyone has a Solaris 10 hardening script that once run will harden for
a) Level 2 Profile
b) "Scored"

The attached which I got from GitHub doesn't seem quite fit to what's needed
& with all the "printf ...", it's more of listing out than actually doing hardening.


From CIS benchmark:

Scoring Information
================
A scoring status indicates whether compliance with the given recommendation impacts the assessed target's benchmark score. The following scoring statuses are used in this benchmark:
Scored  <==
Failure to comply with "Scored" recommendations will decrease the final benchmark score. Compliance with "Scored" recommendations will increase the final benchmark score.
Not Scored
Failure to comply with "Not Scored" recommendations will not decrease the final benchmark score. Compliance with "Not Scored" recommendations will not increase the final benchmark score.



Profile
=====

 Level 1
Items in this profile intend to:
o be practical and prudent;
o provide a clear security benefit; and
o not inhibit the utility of the technology beyond acceptable means.
 Level 2  <==
This profile extends the "Level 1" profile. Items in this profile exhibit one or more of the following characteristics:
o are intended for environments or use cases where security is paramount
o acts as defense in depth measure
o may negatively inhibit the utility or performance of the …
0
This question is regarding modifying the below code which is supposed to take f string as the path and within the sn variable looks for all the files that exist there.

Two things I need to know is

1.) what is the correct syntax for me to loop through and view each file within 'sn'…  I know it has to do with for each but unable to come up with the correct syntax to make it work.
2.) ...and what should 'sn' be declared originally besides variant?

This coding below was found doing a search on youtube as the fastest method to get any files within a folder at this link:
https://www.youtube.com/watch?v=ddA2_SOaq14

Dim sSQL As String
Dim f As String
Dim sn As Variant
    
f = "C:\Temp\File Sorter\Samples"
sn = Split(CreateObject("wscript.shell").exec("cmd /c Dir """ & f & ".pdf" & """ /s /a /b").stdout.readall, vbCrLf)

Open in new window


I'm really looking for some code to try and paste to see if works.  I'm not a student looking for an answer but have other solutions I'm having to figure out and work on.  Don't have time to try this method but if you have time to solve and know it right away I appreciate the direct answer.   Thank you in advance.
0
Using the last SharePoint site in Office 365, and after some changes (see below), all files in SharePoint Sync as marked Read Only (Padlock icon)
All files are Read OnlyIndeed if you try to save the file after making a change, you are warned that this file is Read Only and to save using a new name or in a different location.

When editing a file in Word, all files have the following error message: "REQUIRED PROPERTIES To save to the server, correct the invalid or missing required properties"
Required PropertiesWhen you click on the "Edit Properties" button, Word is asking to add a Title and a File Type. If you fail to enter a File Type, you cannot proceed.
PropertiesSame "error" appears when checking the file online
Properties while online
ALL the files and folder in that Site are affected
There are 54K+ files & ~6,000 folders affected

Background
1. Files were previously uploaded without issues by dropping them in a SharePoint synced folder on a Windows 10 PC using OneDrive one demand
2. There was no issue initially with Read Only or Properties missing
3. We had to restart the above process because it had been stopped in the middle
4. It is only after this second upload that their Read Only & Properties settings were changed.

Google has not helped so far.
* Only one possible root cause (but no fix): Changing the Default View on the SharePoint Site web page that lists all the Documents in that Site has been shown to create …
0
Build an E-Commerce Site with Angular 5
LVL 12
Build an E-Commerce Site with Angular 5

Learn how to build an E-Commerce site with Angular 5, a JavaScript framework used by developers to build web, desktop, and mobile applications.

Experts,

In a file I need to have the possibility to grep a pattern in the first or in the second column.  If found the entire line is returned.
Comes with it that that pattern is a variable.

Grepping the var in the first column is easy, but I can't get it to work for the second column (sep is >).

var=MSPROXML


MSUPDSEOT>MSPROXMLH
VSRFUMVTT>MSPROXMLH
FT0Y6600_T>MSPROXMLH
MSINSMSGT>MSPROXMLH
MSPROXMLT>XGGENEXTH
MSPROXMLT>CD0E0100_H
MSPROXMLT>FT0Y3600_H
MSPROXMLT>PC0R3420_H
MSPROXMLT>WO0A0020_H

This is what would need to be returned:

MSUPDSEOT>MSPROXMLH
VSRFUMVTT>MSPROXMLH
FT0Y6600_T>MSPROXMLH
MSINSMSGT>MSPROXMLH

Thank you for  your help.
Watnog
0
Let's start with a useless example of input redirection:

less 1< /test.txt

Open in new window


The result is:

Missing filename ("less --help" for help)

This I understand, because:

LESS-PROCESS:
FD 0 <- terminal file (keyboard)
FD 1 <- /test.txt
FD 2 -> terminal file (monitor)

FD 0 needs to get some content from a file, but there is no file in this case. There is /test.txt but it points to the wrong fd. Now let's take a look at a useless example of output redirection:

less 0> /test.txt

Open in new window


LESS-PROCESS:
FD 0 -> /test.txt
FD 1 -> terminal file (monitor)
FD 2 -> terminal file (monitor)

The program doesn't give file descriptor 0 some output, so "nothing" will be written to /test.txt. That why you will always end up with an empty /test.txt file. File descriptor 0 opens /test.txt for writing and not for reading. So the less-process doesn't get any file to read from. Then why the result is not:

Missing filename ("less --help" for help)

Instead, less is acting as it got an empty file as input. The file /test.txt is empty in the end, but this is about output redirection and not about input redirection, so there is no input. That's the reason why I would expect "Missing filename". Why this is not the case?
0
See: https://stackoverflow.com/questions/6170598/can-anyone-explain-to-me-what-the-purpose-of-dev-tty


You can start with the POSIX spec. From there, read about the "controlling terminal" of a process.

But just for example... /dev/tty is how a command like "ssh" can read your password even if its standard input comes from somewhere else:

tar cf - . | ssh dest 'tar xf -'

Open in new window


If ssh decides to prompt you for a password, it will read it from /dev/tty instead of stdin.

Conceptually, /dev/tty is "the keyboard and text terminal". More or less.

Let's say my "terminal-file" of the current session is /dev/pts/1. In such a case, then what's the difference between "/dev/pts/1" and "/dev/tty"? And if they are basically the same, then why  "/dev/tty" is used instead of "/dev/pts/1"?

And:

/dev/tty is how a command like "ssh" can read your password even if its standard input comes from somewhere else

Let's say the standard input comes from somewhere else, so let's say we have:

FD 0 <- file
FD 1 -> /dev/pts/1
FD 2 -> /dev/pts/1

How I see it: the fact that the standard input comes from somewhere else doesn't mean that /dev/pts/1 can not be read? The password comes from the keyboard and /dev/pts/1 represents i.a. the keyboard, right? So I still don't see what exactly the purpose is of /dev/tty?

@noci: I know you know the answer, but I don't understand your explanation so I've made this post so maybe other people can explain it to me in a way that I understand it.
0
On a Redhat Linux system running a bash shell script I need some help with an if then statement that has more than 2 conditions. I basically want to check for this
A AND B or C  
A AND B or D
A AND B or E

Something along these lines but it doesn't work and wondered if I have the correct usage of brackets. It's not what's contained for evaluation that's the issue it's the syntax of the AND and OR where there's more than two conditions that I am struggling with.

if [[ $(find /opt/app -name httptd*.conf | grep -v grep | grep -c http) -eq 0 ] && [ ! -f /etc/init.d/apache ] ||  [ $(find /app -name http*.conf | grep -v grep | grep -c http) -eq 0 ]] || \
[[ $(find /opt/app -name httptd*.conf | grep -v grep | grep -c http) -eq 0 ] && [ ! -f /etc/init.d/apache ] ||  [ $(find /application -name http*.conf | grep -v grep | grep -c http) -eq 0 ]] || \
[[ $(find /opt/app -name httptd*.conf | grep -v grep | grep -c http) -eq 0 ] && [ ! -f /etc/init.d/apache ] ||  [ $(find /application -name manifest* | grep -v grep | grep -c http) -eq 0 ]] ; then
.....
0
I'm reading about "redirection of input" on the internet. I understand what's behind it. For example:

command < file.ext

Open in new window


This is equivalent to:

command 0< file.ext

Open in new window


In general, if you have:

command n< file.ext

Open in new window


then the contents of file.ext go to file descriptor "n" as input. I've checked different websites explaining "input redirection". However, the problem is that I didn't see any good example. I'll discuss some examples I saw:

cat < file.txt

Open in new window


Then I'm thinking, "cat file.txt" does the same, so why do we need it? Another example:

sort < file_list.txt > sorted_file_list.txt

Open in new window


Then I'm thinking, "sort file_list.txt > sorted_file_list.txt" does the same, so why do we need it? Another example:

more < /etc/passwd

Open in new window


Then I'm thinking, "more /etc/passwd" does the same, so why do we need it? That's why these are not really good examples in my opinion. What is a good example to explain the purpose of input redirection in a terminal-window?

Probably internally something like "cat file.txt" is being treated as "cat 0< file.txt" (input redirection), but in a terminal-window ... when it really does make sense to use an "input redirection" in a terminal-window? Does someone have a good example?
0
Why does for loop return only the first entry from my lists?  Stripped down script as follows.

groups=(baremetal production staging)
baremetal=(test1 test2)
production=(test3)
staging=()

for group in ${groups[@]}
  do
        echo $group

    for host in ${!group}   # only adds the 1st
      do
           echo $host

      done
  done


./test.sh
baremetal
test1
production
test3
staging
0
First I create a regular file with some contents (manual page of find command):

man find > test.txt

Open in new window


Then I use the less command to display some of these contents:

less test.txt

Open in new window


Now I press CTRL-Z to suspend the process. The process is still open, so now I can execute this command:

lsof | grep 'less'

Open in new window


By doing this, I get an idea which files are open with respect to the less-process. My result:

COMMAND  PID    USER  FD   TYPE  DEVICE  SIZE/OFF  NODE       NAME
less     24565  root  cwd  DIR   0,38    4096      21473055   /
less     24565  root  rtd  DIR   0,38    4096      21473055   /
less     24565  root  txt  REG   0,38    149944    22143102   /usr/bin/less
less     24565  root  mem  REG   9,1               22143102   /usr/bin/less (path dev=0,38)
less     24565  root  mem  REG   9,1               22135172   /usr/lib/locale/locale-archive-rpm (path dev=0,38)
less     24565  root  mem  REG   9,1               21741879   /lib64/libc-2.12.so (path dev=0,38)
less     24565  root  mem  REG   9,1               22265955   /usr/local/lib/libpcre.so.0.0.1 (path dev=0,38)
less     24565  root  mem  REG   9,1               21741743   /lib64/libtinfo.so.5.7 (path dev=0,38)
less     24565  root  mem  REG   9,1               21741946   /lib64/ld-2.12.so (path dev=0,38)
less     24565  root  0u   CHR   136,1   0t0       4          /dev/pts/1
less     24565  root  1u   CHR   136,1   0t0       4          /dev/pts/1
less     

Open in new window

0
Let's say I type the following "in a terminal":

echo 'bla'

Open in new window


In my case, the shell is bash, so I assume the shell/bash-process receives "echo 'bla'" as standard input? Then it sees "echo", so a child process will be started. So then we will have at least:

ECHO PROCESS:
fd 0 (standard input)   <- terminal-file (keyboard)
fd 1 (standard output)  -> terminal-file (monitor)
fd 2 (standard error)   -> terminal-file (monitor)

Open in new window


I thought that for this process, only "bla" is the standard input. And then the output is also "bla", so I'll see "bla" on my monitor.

I was just a bit playing with "input redirections" and I noticed that the following does not work:

echo < bla-file.txt

Open in new window


After some Google searches, I found out that "echo" does not read from stdin. However, it prints all of its arguments. So it's working differently than normal. So how I have to see/change this:

ECHO PROCESS:
fd 0 (standard input)   <- terminal-file (keyboard)
fd 1 (standard output)  -> terminal-file (monitor)
fd 2 (standard error)   -> terminal-file (monitor)

Open in new window


I thought every process by default has fd's 0,1,2? But if fd 0 would be there something like this:

fd 0 (standard input)   <- nothing

Open in new window


Then it should be still possible to redirect (input) to something. So this means I can not see it like that. Does this mean that the echo process doesn't have a fd 0 at all? Or I must not see "echo" as a process with a fd table et cetera?

But the echo command displays something on my monitor, so at least this should be there:

fd 1 (standard output)  -> terminal-file (monitor)

Open in new window

1
Experts,

I am looking for a script that can take a list of hostnames ($mypcs) and provide the IP, MacAddress, and Hostname of each of the computers in question.

I would like this script to also be able to tell when computers were not reachable..

I have been working on the following but haven't gotten it to work property yet and need this information ASAP.


Function Get-Mac { 
    $ComputerName = $mypcs 
    $ErrorActionPreference = 'Stop'
    
    $COlResults = foreach ($Computer in $ComputerName) {     
        invoke-command -ComputerName $computer -Credential $cred -ScriptBlock {
            Try{
                $WMI = gwmi -class "Win32_NetworkAdapterConfiguration" |? IpEnabled -EQ "True" | select PSComputerName, description, ipAddress, MACAddress | FT -AutoSize;
                $WMI        
            }catch {
                    Write-Warning "System is not reachable : $Computer";Continue
             }
            
        
    }

    }
    $COlResults
}
. Get-Mac

Open in new window

0
Learn Ruby Fundamentals
LVL 12
Learn Ruby Fundamentals

This course will introduce you to Ruby, as well as teach you about classes, methods, variables, data structures, loops, enumerable methods, and finishing touches.

Hello,

I have a bash script that I want to output the variables into a .csv file.  Most of the variable I can have the variable output on one line but I'm having a problme with variables that are multiline. The two variables are passed the process id of a process and the process listing contains new line characters. I thought if I stripped out the newline characters that you could assign it to the variable but it complains "No such file or directory in the variable substition".  The other variable is just netstat -an | grep <port>.  I don't know how to use arrays in bash but ideally i'd like to be able to keep the newlines in the csv file, if it's possible, since i'm sticking with comma as the separator. Any help would be really appreciated. Hitting my head off the wall!

VAR1="$(ps -ef | grep 27656 | tr '\n' ' ' | sed -e '/s   */g')"
(printf '%s\n' $VAR1)
0
File descriptor table:      Open file table
FD 0 (stdin)                ?
FD 1 (stdout)               ?
FD 2 (stderr)               ?

Open in new window


By default file descriptors 0, 1 and 2 are associated with the terminal. The keyboard input is associated with the standard input. The monitor is associated with the standard output and standard error.

The question is: Do fd 0,1,2 all refer to the same entry in the "open file table"? Or do they refer to two entries?

 
FD 0  -> entry A
FD 1    
       > entry B  
FD 2    

Open in new window


Or do they refer to three entries?

 
FD 0  -> entry A
FD 1  -> entry B
FD 2  -> entry C

Open in new window


This seems a pretty basic question, but I'm reading different things about it on the internet.

See: https://www.usna.edu/Users/cs/aviv/classes/ic221/s16/lec/21/lec.html#orgheadline6

If fd 1 and fd 2 refer to a different entry in the open file table, then this should be also the case for fd 0. So according to this website, they refer to three different entries in the open file table.
 
But now see: https://www.enseignement.polytechnique.fr/informatique/INF422/INF422_8.pdf#page=160    (page 160, example of no redirection)

There, it's like:

 
FD 0  -> entry A
FD 1    
       > entry B  
FD 2    

Open in new window


So according to that website, they refer to two different entries in the open file table.

And see: https://www.experts-exchange.com/questions/29119936/How-the-open-file-table-entries-look-like-for-stdin-stdout-stderr.html#a42694025


the fd[0] , fd[1] & fd[2] should all point to the same central entry

According to this, they all refer to the same entry in the open file table.

I can execute the following command:

lsof | grep 'bash'

Open in new window


This i.a. prints:


Open in new window

0
Hi All,

I have about 20 Centos OS servers and I need a script that read a text file that contains a list of server names, then go into each server and find out if a particular file exists, if it does, write the host name to fileexsit.txt and it the file doesn't exist, write to a file named filenotexist.txt.

Please help!!


Thanks!
0
My question is about a pipe/pipeline in Unix. And it's especially about the FD (File descriptor) numbers/integers associated with it.
This is what I would expect:

  PROCESS A           PROCESS B
  0 stdin
  1 stdout -> pipe -> 0 stdin
  2 stderr            1 stdout
                      2 stderr

Open in new window


However, on the internet I'm reading different things.

For example, see: http://pubs.opengroup.org/onlinepubs/009696799/functions/pipe.html


Their integer values shall be the two lowest available at the time of the pipe() call.

Also see the images here: http://www.rozmichelle.com/pipes-forks-dups/#attachment_7362

Why new file descriptors are created? Why they don't use stdout (1) of the parent process? They could just redirect this existing
file descriptor (1/stdout) to the pipe. Do I miss something?
1
Hi,

Im trying to develop a script that will strip the msexchangeguid attribute from a particular OU only, once a week as a scheduled task.
Im almost there.

The first part of the script successfully identifies all users in that OU that has a non-NULL msExchangeGUID
 
Get-ADUser -SearchBase "OU=Disabled-Objects,dc=my,dc=domain,dc=com" -filter * -Properties * | ? {$_.msExchMailboxGuid -ne $null}

Open in new window


The next part of the linked script will just give me the SAMAccountNames of those users with a non-Null msExchangeGUID from the above command
Select-Object SamAccountName

Open in new window


Put the final part of setting the attribute to be Null (0000) doesnt take the SamAccountNames for action
Set-RemoteMailbox  -ExchangeGUID 00000000-0000-0000-0000-00000000000

Open in new window


Here it is in one non-functioning line.

Get-ADUser -SearchBase "OU=Disabled-Objects,dc=my,dc=domain,dc=com" -filter * -Properties * | ? {$_.msExchMailboxGuid -ne $null} | Select-Object SamAccountName | Set-RemoteMailbox  -ExchangeGUID 00000000-0000-0000-0000-000000000000

Open in new window



I also thought about i could put the output of the first two lines into a text file, that I think use as a variable
Get-ADUser -SearchBase "OU=Disabled-Objects,dc=my,dc=domain,dc=com" -filter * -Properties * | ? {$_.msExchMailboxGuid -ne $null} | Select-Object SamAccountName > users.txt

Open in new window


$users=Get-content users.txt
Foreach($user in $users){
get-aduser $user| set-aduser -clear msExchMailboxGuid
}

Open in new window


But the users.txt has trailing whitespaces after the username so the  
get-aduser $user| set-aduser -clear msExchMailboxGuid

Open in new window

looks like
get-aduser "USERNAME        "| set-aduser -clear msExchMailboxGuid

Open in new window

when it runs. It fails on the excess whitespaces.

Any help from Powershell gurus out there?
Thanks!
0
I am, at best, a novice user when it comes to writing Powershell scripts, so any simplified explanation to this question would be appreciated.
I am accessing one of our vendors API services. In order to access it, I need to generate an authentication token. I have the url to send the request to, and I have the username and password. What I am not sure of, is how to write the script to retrieve the token. Is there a "generic" method to do this?

Again, feel free to break it down to the most basic of terms. I won't be offended. :)

Thank you in advance.
0

Shell Scripting

10K

Solutions

6K

Contributors

The term 'shell' refers to a general class of text-based command interpreters most often associated with the UNIX and Linux operating systems. Popular shells include Bourne, Debian Almquist (dash), Korn (ksh), Bourne Again (bash) and the C shell family (csh). Some view the DOS 'cmd' prompt as a minimal shell of sorts. It is also possible to install Cygwin on Windows and emulate a full Unix environment with complete shell capabilities. Terminal emulators, such as xterm, GNOME Terminal and OS X Terminal, can be used to access shell.