Solved

determine the files extension

Posted on 2014-11-12
24
107 Views
Last Modified: 2014-12-16
In linux, we can use the command file "example.txt" to determine the true extension of the file. Even if I rename the original example.zip file to example.txt, the file command still can recognize the true extension by using magic header.
So do we have any way to determine the list of files inside a folder by using bash, powershell or python. I prefer these languages because it's easy for me to read and understand.

Can anyone help me with this?

Thank you so much.
0
Comment
Question by:totoroha
  • 10
  • 10
  • 2
  • +2
24 Comments
 
LVL 29

Expert Comment

by:MikeOM_DBA
ID: 40438423
Try:
 echo ${file_name##*.} 

Open in new window

:p
0
 
LVL 51

Expert Comment

by:Bill Prew
ID: 40438500
Okay, sounds like you are looking for a Windows alternative to the unix "file" command.

I don't think there is any built in way to do that, even with some of the scripting languages you mentioned.

You could try one of the ports of "file" to Windows, like:

http://gnuwin32.sourceforge.net/packages/file.htm

Or there are some utilities that attempt to do this - it's a bit of a guessing game by examining the file contents and trying to determine the file type based on that.  One popular example of that is:

http://mark0.net/soft-trid-e.html

~bp
0
 

Author Comment

by:totoroha
ID: 40438520
can you explain it to me Mike?
Bill: I have linux environment too, so it is not a constraint for only Windows.
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 40438888
if you are using bash then you can always call the command file from bash command prompt or within bash script:

file filename
0
 
LVL 51

Expert Comment

by:Bill Prew
ID: 40439019
On linux can't you use the FILE command?

~bp
0
 

Author Comment

by:totoroha
ID: 40439121
I can use FILE command in linux for a single file. But what if I want to check hundred of files? I cannot type in a hundred time. ^_^
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 40439233
you can put it in loop:

ls | while read myfile
do
    file $myfile
done

the ls command can be used in different way like

ls *.txt | while read myfile
do
    file $myfile
done
0
 

Author Comment

by:totoroha
ID: 40441772
Thank you omar. But my coding skill is nearly 0 ^_^ If you can give me an example that works, i would really appreciate it.
Either powershell or python,bash is ok.
0
 
LVL 84

Expert Comment

by:ozo
ID: 40441777
#!/bin/bash
file *.txt
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 40441968
The example I gave can be put in a file e.g. myscript, run:
echo 'ls *.txt | while read myfile
do
    file $myfile
done' > myscript

(or using a text editor). Then you make the file executable:

chmod +x myscript

Then you can call the script by its name:

./myscript
0
 

Author Comment

by:totoroha
ID: 40451574
I think the example from ozo is really good in this case. The problem is I can use wget -I to get all the http links in the text file and download all of it. After that I can check it with the "file" command. The path that i don't know is how to automate that whole process in python or bash language.
I soul really appreciate if anyone can help me with that.
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 40452524
You started with how to determine file type for files inside folder, and now you are talking about urls and downloading files, etc.

Please elaborate on your exact requirements.
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 

Author Comment

by:totoroha
ID: 40452610
Thanks omar for asking that. At first, I thought I could do the manual job by downloading the files and put it in the folder and check the file type. But after went through the discussion with everyone, I think that If I can automate this process, that would be a great help for me in saving time.
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 40452832
So,  can you till your requirement? you may put steps you want to do and then a script can be provided.
0
 

Author Comment

by:totoroha
ID: 40453204
step 1: download a list of text http links which included in a A.txt file
step 2:  *. while downloading, if the file is text, put it in one sub-folder.
                                                       *. if not text file, put it in one sub-folder.

I think that's all.
0
 

Author Comment

by:totoroha
ID: 40456424
hi Omar,

Do you think my idea is clear enough or we need to clarify more? Thank you ^^
0
 
LVL 40

Accepted Solution

by:
omarfarid earned 500 total points
ID: 40468318
Please try below:

cat A.txt | while read file
do
   wget $file
   file $file | grep text
   if [ $? -eq 0 ]
   then
         mv $file sub-folder
   else
        mv $file other-folder
   fi
done
0
 

Author Comment

by:totoroha
ID: 40501661
Dear Omar,

Thank you for your help. Now I can run my script without any error. However, if I want to create another loop for your code, what should I do? and what kind of language that you're using for the code?

Thank you so much
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 40501862
This is a shell script (sh / ksh / bash).

Where do you want to add the loop?
0
 

Author Comment

by:totoroha
ID: 40502687
If the file is text, I want to move it to text folder. if the file is zip file, I want to move it to zip folder, if it is binary file, move it to binary folder. The rest can move to spam folder. That's what I want to achieve omar.

Thank you for your help!
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 40504069
Do you have the return messages for such file types?
0
 

Author Comment

by:totoroha
ID: 40504088
If we can export it to csv file with file name, and return message, that would be great!
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 40504150
I don't have all the messages that file will return for each file type, but I can show you how the loop can be changed

cat A.txt | while read file
do
   wget $file
   filetype=`file $file`
   echo $filetype | grep text
   if [ $? -eq 0 ]
   then
         mv $file sub-folder1
   elif
   echo $filetype | grep gzip
   then
        mv $file other-folder2
   fi
done
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 40504153
Correction

cat A.txt | while read file
do
   wget $file
   filetype=`file $file`
   echo $filetype | grep text
   if [ $? -eq 0 ]
   then
         mv $file sub-folder1
   else
        echo $filetype | grep gzip
        if [ $? -eq 0 ]
        then
               mv $file other-folder2
        fi
   fi
done
0

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

Introduction: Dialogs (1) modal - maintaining the database. Continuing from the ninth article about sudoku.   You might have heard of modal and modeless dialogs.  Here with this Sudoku application will we use one of each type: a modal dialog …
How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
Learn the basics of modules and packages in Python. Every Python file is a module, ending in the suffix: .py: Modules are a collection of functions and variables.: Packages are a collection of modules.: Module functions and variables are accessed us…
Learn the basics of while and for loops in Python.  while loops are used for testing while, or until, a condition is met: The structure of a while loop is as follows:     while <condition>:         do something         repeate: The break statement m…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now