?
Solved

DoS bug/exploit

Posted on 2006-04-12
37
Medium Priority
?
347 Views
Last Modified: 2012-06-27
Hi, i have a problem. i am running a shell server. Freebsd 5.4 pl13 . i have quotas enabled - 150 mb disk space / 1500 inodes . some user yesterday, built a program, that created 400 folders . in each folders they were 999 files with very long names. The weird thing is that they had all the same inode .

this is ls -lai ... so you see that it has the same inode .. each file . the thing is that server will crash if runs sme periodic tasks like ... updatedb or something .
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx074
1319375 -rw------- 30000 user users 0 Apr 10 23:31 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx075
1319375 -rw------- 30000 user users 0 Apr 10 23:31 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx076
1319375 -rw------- 30000 user users 0 Apr 10 23:31 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx077

the whole idea is that by creating those files with the same inode, the quota stuff saw only 800 inodes, as files had the same inodes. so ... above all my protection was bypassed. any idea how to work this around ?

the thing is i have the proggie that did this ... and i tried some reverse engineering on it, and ... what i got is ... a fragment of procedures ... now im very very very smakll at c coding ..

need help on how to decompie the file ... and to prevent this from happening.

deleting shell access or not giving shell access is not an option for me. i need to secure the box not to delay another happening


_start()
{

(save)ebp;
ebp = esp;
(save)edi;
(save)esi;
(save)ebx;
esp = esp - 0xc;
ecx = edx;
esi = ebp + 8;
ebx = *(esi - 4);
environ = esi + ebx * 4 + 4;
if(ebx > 0 && *(ebp + 8) != 0) {
eax = *(ebp + 8);
__progname = eax;
edx = __progname;
if(*__progname != 0) {
do {
if(*edx == "") {
__progname = edx + 1;
}
edx = edx + 1;
} while(*edx != 0);
}
}
if(134519772 != 0) {
esp = esp - 0xc;
(save)ecx;
L08048584();
esp = esp + 0x10;
} else {
L08048564();
}
esp = esp - 0xc;
(save)_fini;
L08048584();
_init();
esp = esp - 4;
(save)edi;
(save)esi;
(save)ebx;
main();
esp = esp + 0x14;
(save)eax;
L080485B4();
}

/* Procedure: 0x08048664 - 0x080486B5
* Argument size: -16
* Local size: 8
* Save regs size: 0
* Called by:
* _fini()
*/

__do_global_dtors_aux()
{

if(completed.1 == 0) {
eax = *p.0;
do {
p.0 = p.0 + 4;
*eax();
eax = *p.0;
} while(eax != 0);
eax = 0;
if(0 != 0) {
esp = esp - 0xc;
(save)__EH_FRAME_BEGIN__;
eax = L00000000();
}
completed.1 = 1;
}
}

/* Procedure: 0x080486B6 - 0x080486B7
* Argument size: 0
* Local size: 0
* Save regs size: 0
*/

L080486b6()
{

}

/* Procedure: 0x080486B8 - 0x08048701
* Argument size: -8
* Local size: 8
* Save regs size: 0
* Called by:
* _init()
*/

frame_dummy()
{

eax = 0;
if(0 != 0) {
esp = esp - 8;
(save)object.2;
(save)__EH_FRAME_BEGIN__;
eax = L00000000();
}
if(__JCR_LIST__ != 0) {
eax = 0;
if(0 != 0) {
esp = esp - 0xc;
(save)__JCR_LIST__;
L00000000();
}
}
}

/* Procedure: 0x08048702 - 0x08048703
* Argument size: 0
* Local size: 0
* Save regs size: 0
*/

L08048702()
{

}

/* Procedure: 0x08048704 - 0x08048A7D
* Argument size: -1840
* Local size: 920
* Save regs size: 0
* Called by:
* _start()
*/

main()
{
/* unknown */ void Vfffffc74;
/* unknown */ void Vfffffc78;
/* unknown */ void Vfffffce0;
/* unknown */ void Vfffffce4;
/* unknown */ void Vfffffce8;
/* unknown */ void Vfffffde8;
/* unknown */ void Vfffffee8;
/* unknown */ void Vfffffef8;
/* unknown */ void Vffffffed;

esp = esp & -16;
esp = esp - ("ec/ld-elf.so.1" >> 4 > "c/ld-elf.so.1"));
esp = esp - 8;
(save)0x1ed;
(save) & Vfffffee8;
esp = esp + 0x10;
if(L080485C4() < 0) {
esp = esp - 4;
(save) & Vfffffee8;
(save)"mkdir(%s) failed\n";
(save)__stderrp;
L08048544();
esp = esp + 0x10;
esp = esp - 0xc;
(save)1;
L080485B4();
}
}
esp = esp - 0xc;
ecx = Vfffffce0;
eax = 274877907;
asm("imul ecx");
edx = edx >> 6;
edx = edx - (ecx >> "c/ld-elf.so.1");
eax = (edx
0
Comment
Question by:rares_dumitrescu
  • 14
  • 12
  • 9
  • +1
37 Comments
 
LVL 14

Expert Comment

by:ppfoong
ID: 16442495

You should have a proper resource usage policy / security policy that if any user violate, their access will be revoked and they will be penalised for any lost or damage done.

Fine the culprit user and delete his/her account!

If you want to solve it the technical way, report the problem to the FreeBSD development community, or switch to another distribution that does not have the problem.

0
 
LVL 7

Expert Comment

by:wnross
ID: 16442836
Well, if these are all the same inode, then we're looking at a hude number of hardlinks, which are harmless.
As your quota is showing: only 800 inodes and some presumably 24MB of storage.

As for cron, if it's bogging down in these folders, try launching updatedb as a background task
* * * * * updatedb &

that way your cron doesn't hang waiting for the locate subsystem to finish.  Alternately prune the
directories from updatedb...there's a compelling argument that says we don't need to know where our
users files are located and they are free to use conventional tools such as "find".  

(Assuming non-staff environments, such as web accounts, obviously staff members need to locate
those spreadsheets and should not be pruned).

Since these are hardlinks, rm -f will clean up the files for you lickety-split.

As for the program, it looks like your decompile attepts gave up too early, have the jump points
came back blank.  All is not lost, however:

Whats the name of the binary?
most script attacks come from common code which is easy to track in google

ldd ./hackerprog
will dump the shared libraries and give you a clue as to the program intentions
libncurses == console monitoring program
libpcap == network sniffer

and so on

strings ./hackerprog | more
Careful analysis will give text prompts, network addresses, even the author on occasion.

Look for the source code as well, check the /tmp folder (also /var/tmp; /var/cache/apache; and anywhere in
the user folder) and be wary of  folder paths such as ".. " (note the space)

These hidden in plain sight folders can lead you to the original .c files used to create the binary.

Cheers,
-Bill
0
 

Author Comment

by:rares_dumitrescu
ID: 16444816
program name was renamed to eggdrop :) so i guess that won't be of help. source code. ... he wasn't that stupid to let it there. the idea is ... and main question. how do you create 1000 files, which have the same inode ? just how ? so i could prevent that. as i said deleting an user is NOT an option. just delays the pain. i want to solve it not move forward with an insecure system geez.
0
Veeam Disaster Recovery in Microsoft Azure

Veeam PN for Microsoft Azure is a FREE solution designed to simplify and automate the setup of a DR site in Microsoft Azure using lightweight software-defined networking. It reduces the complexity of VPN deployments and is designed for businesses of ALL sizes.

 
LVL 14

Expert Comment

by:ppfoong
ID: 16445046

You can create as many file as you like, with a simple looping function in a simple shell script.

0
 
LVL 14

Expert Comment

by:ppfoong
ID: 16445061

To prevent malicious users, you can mount the user home directory as non-executable and non-suid, provided the home directory of all users is in a separate partition.

0
 
LVL 14

Expert Comment

by:ppfoong
ID: 16445079

To answer your other question: a hard link will create file with same inode.

0
 

Author Comment

by:rares_dumitrescu
ID: 16445244
okay ... but as you see the files are no links. are simply files .

1319375 -rw------- 30000 user users 0 Apr 10 23:31 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx076
1319375 -rw------- 30000 user users 0 Apr 10 23:31 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx077

Second of all . no you can't create as many files as you like if you set limitations in quota for number of inodes. which i already did .. set limit to 1500 files. still this method of creating files has overpassed my protection. those are not hard links as you see. question is how were they created ? :)
0
 
LVL 14

Expert Comment

by:ppfoong
ID: 16445353

A hard link will appear as a file. A soft link will appear as a link.

Try this:

touch file1
ln file1 file2
ln file1 file3
ln file1 file4


0
 

Author Comment

by:rares_dumitrescu
ID: 16445606
fuck you are right .....

 215666 -rw-------    4 user   users           0 Apr 13 13:47 file1
 215666 -rw-------    4 user   users           0 Apr 13 13:47 file2
 215666 -rw-------    4 user   users           0 Apr 13 13:47 file3
 215666 -rw-------    4 user   users           0 Apr 13 13:47 file4


so all i have to do ... is restrict access to the ln command.... and they would not be able to do it aight ?
0
 

Author Comment

by:rares_dumitrescu
ID: 16445634
i restricted access to the ln command, but still works ...
0
 
LVL 7

Expert Comment

by:wnross
ID: 16445646
for ((x = 0; $x <5; x++)) do
   ln bigfile xxxxxxx$x
done

(Assuming bash shell)

Hardlinks are directory entries which use the same inode as another file.
Linux will generally disallow hardlinks of directorys, and since the
files share the same inode, you may not have hardlinks which span
multiple filesystems.

So, your user went nuts in the directory entries department, but actually has not created any more than 800 files.

Ok. so what does ldd and strngs give us?  And assume nothing about the intelligence of your attacker, even
someone brilliant will leave traces behind

Cheers,
-Bill
0
 

Author Comment

by:rares_dumitrescu
ID: 16445679
i restricted access to ln and touch commands, still the exploit uses something else ....

touch file1
-su: /usr/bin/touch: Permission denied

ln file1 file2
-su: /bin/ln: Permission denied



&#9492;&#9472;(~/eggdrop/modules/eggdrop.so)-> ./eggdrop
00/04000...

just built 4000 files ...

0
 

Author Comment

by:rares_dumitrescu
ID: 16445723
ldd eggdrop
eggdrop:
        libc.so.5 => /lib/libc.so.5 (0x28076000)


trings eggdrop
/libexec/ld-elf.so.1
FreeBSD
libc.so.5
__stdoutp
_DYNAMIC
__stderrp
creat
fflush
_init
environ
fprintf
__deregister_frame_info
__progname
memset
_init_tls
_fini
sprintf
atexit
stat
_GLOBAL_OFFSET_TABLE_
link
_Jv_RegisterClasses
mkdir
__register_frame_info
_edata
__bss_start
_end
$FreeBSD: src/lib/csu/i386-elf/crti.S,v 1.6 2002/05/15 04:19:49 obrien Exp $
%02d/%05d...
%02d-%02d
mkdir(%s) failed
%s/%s%03d
creat(%s) failed
stat(%s) failed
link(%s,%s) failed
finished successfully
$FreeBSD: src/lib/csu/i386-elf/crtn.S,v 1.5 2002/05/15 04:19:49 obrien Exp $



0
 
LVL 7

Expert Comment

by:wnross
ID: 16445725
linking is an OS capability, restrict ln all you like, they can still do it via a program using the link() function.

One option is to restrict access to compiler tools

groupadd ctools
chown root:ctools /usr/bin/cc /usr/bin/gcc-3.4.1 /usr/bin/protoize /usr/bin/ld /usr/bin/ar
chmod 750 /usr/bin/cc /usr/bin/gcc-3.4.1 /usr/bin/protoize /usr/bin/ld /usr/bin/ar

then add any real developers to the ctools group.

Ditto for ntools and programs like ping, nmap and the like.

-Bill
0
 

Author Comment

by:rares_dumitrescu
ID: 16445765
hummz the idea is that legit users that have nothing to do with this .. like 99 % of users on the server, need access to compile stuff ...
0
 

Author Comment

by:rares_dumitrescu
ID: 16445873
can i restrict access for example .. to the link function ? in compiler ?
0
 
LVL 7

Expert Comment

by:wnross
ID: 16445906
True, but apache doesn't nor do ftp-only users, nor do managers, administrative staff, etc.
What account did she use to get in? Was it a casual user or one of your development team?

This isn't a very large program, is it?
Looks like your ok, based on the output the program has no network capability,
it just creats a heck of a lot of hard links.

Only one dependency: the standard c library...so he/she is not doing anything special
as for os routines, we see that he calls
mkdir to create the directorties
creat() to create the initial files
sprintf fprintf and fflush to twrite to the creat()'d files
stat() to confirm the file was created (can't be too creful)
then link() to create a hardlink.  

Based on what I've seen above + the ldd and strings, thats about it.

He/she probably wrote this by hand rather than scripting it since its such a simple program

1) This program does what we think it does
2) No further footprint exists (here) as to the attack profile
3) Restricting access to compiler and network tools will hamper future attacks (unless they change languages)

oh, add curl and wget to your ntools groups
Usual attack profile goes:
- gain shell access
- wget somehakersite/tools.tgz
- tar -xzf tools.tgz
- gcc hackme.c
- go to town with the destruction and stuff

Cheers,
-Bill
0
 
LVL 14

Expert Comment

by:ppfoong
ID: 16445998

As long as you allow executable to be put into and run from the user directory, and/or the /tmp directory, there is nothing much you can prevent.

The user can always do the compiling in another machine, and transfer the file over. Even if file transfer is not possible, the user can get the machine code of the compiled program, and produce the same codes in your server with a simple script/program.

0
 
LVL 7

Expert Comment

by:wnross
ID: 16446247
I don't know if *simple* script program is accurate, but you are right there ppfoong,  besides this still does not rule out attacks using other languages such as perl, php, python, and the like, non of which require compilation.
0
 

Author Comment

by:rares_dumitrescu
ID: 16446346
well yes the program is compiled already. this is so damn true. the program was compiled and brought here ...  are there any limitations that i can do to deny the use of those syscalls  for a certain login class of users ?
0
 
LVL 14

Expert Comment

by:ppfoong
ID: 16446417

The simple script I said just need to make use of echo to escape codes.

As I said before, unless you limit them from executing program from their home directory, it is unlikely to stop them running such program.

0
 

Author Comment

by:rares_dumitrescu
ID: 16446427
people must be able to execute programs ... damn ... this means .. it can't be worked around ...
0
 
LVL 14

Expert Comment

by:ppfoong
ID: 16446476

Back to square one. Use resource usage policy / security policy to restrict the user.

0
 
LVL 7

Accepted Solution

by:
wnross earned 1500 total points
ID: 16446517
Lets look at attacker profile and privileges again: was this a general staff account?  Certainly you could do something like

/exports/staff
/exports/developer

mount localhost:/exports/staff  /home/staff  -t nfs -onoexec
mount localhost:/exports/developer /home/team -t nfs -ouser,exec,no_root_squash
mount /tmp -oremount,noexec
rmdir /var/tmp
ln -s /tmp /var/tmp

Good luck creating executables with that setup
0
 
LVL 7

Expert Comment

by:wnross
ID: 16446537
Also was this a break in or an in-house job?
0
 

Author Comment

by:rares_dumitrescu
ID: 16446925
well ... the idea is that users have shell access . it is a shell hosting server. they must be able to execute proggies.  i know that nosuid noexec stuff. but if i do it for all users, server will render useless. ...

this was an in house job. someone bought a shell account and used this program to screw me up. the idea is that he can do it over and over again .. all he has to do is pay and get another shell. that is why i said that deleting the user is not the solution.


back to square one. there are limitations already. 1500 max files and 150 mb disk space. limited by quota. but as he creates each 1000 files with the same inode ... hard links, the quota system will see 400000 files as 400 files ... and will not block creating more ....
0
 
LVL 7

Expert Comment

by:wnross
ID: 16447015
Then I suggest let them do it: there's no harm if they want to clutter their account that way.

In the meantime, as I mentioned before, prune updatedb from looking in your user's files

/etc/updatedb.conf
# PRUNEPATHS="/proc,/tmp,/var/tmp,/usr/tmp,/net,/afs,/mnt"
PRUNEPATHS="/proc,/tmp,/var/tmp,/usr/tmp,/net,/afs,/mnt,/home/customers"

this will keep the cron task out of there.   The user will end up punishing themselves since
simple ls commands will take minutes or even hours to complete.

the locate command is great, but there's no reason at all for it to be used by your clients, not as long as
they can only create files in their home directories anyway.

Cheers,
-Bill
0
 

Author Comment

by:rares_dumitrescu
ID: 16447074
i thought of something ... well .. mounting the /home in noexec, and all legit program binaries to be run from /bin . i mean users should not have the binaries there, just run them from /bin /bin/psybnc /bin/eggdrop ... and mounting the /tmp in noexec/nosuid will fix this aight ?
0
 
LVL 14

Expert Comment

by:ppfoong
ID: 16447203

This is what I suggested in the first place.

Of course you need to have /bin /sbin /usr/bin /usr/sbin /usr/local/bin /usr/local/sbin to run executable, but any directory that is user writable, including the /home and /tmp, should be mounted with noexec and nosuid.

0
 
LVL 7

Expert Comment

by:wnross
ID: 16447217
That was my implication, although if you have people requiring use of compilers, then they will need some kind of bin folder
that they can write to as well.

Personally I would mount  your tmp directory that way anyways.

Here's a thought, although it takes some setup: use SELinux,  then you can fine tune your policies according to need
such as:
stat_file_perms      Permissions to call stat or access on a file.
x_file_perms      Permissions to execute a file.
r_file_perms      Permissions to read a file.
rx_file_perms      Permissions to read and execute a file.
rw_file_perms      Permissions to read and write a file.
ra_file_perms      Permissions to read and append to a file.
link_file_perms      Permissions to link, unlink, or rename a file.   <------------------  looky here
create_file_perms      Permissions to create, access, and delete a file.
r_dir_perms      Permissions to read and search a directory.
rw_dir_perms      Permissions to read and modify a directory.
ra_dir_perms      Permissions to read and add entries to a directory.
create_dir_perms      Permissions to create, access, and delete a directory.
mount_fs_perms      Permissions to mount and unmount a filesystem.

Since RedHat/CentOS/Whitebox linux support SElinux out of hte box, this might not be a bad way of going.
Cheers
-Bill
0
 
LVL 7

Expert Comment

by:wnross
ID: 16447232
0
 
LVL 16

Expert Comment

by:xDamox
ID: 16447888
Hi,

Could you send me the program I want to analysis this problem, anyways I have a small solution:

http://www.grsecurity.net/features.php

grsecurity will provide the following help:

Symlink/hardlink restrictions to prevent /tmp races
0
 
LVL 16

Expert Comment

by:xDamox
ID: 16447971
Heres a little more info, grsecurity looks to be the way:

http://private.addcom.de/nordi/linux/suse_seclink.html

ignore the suse bit it wil lapply for all
0
 

Author Comment

by:rares_dumitrescu
ID: 16448632
i know the grsecurity but as you guys would find out by reading the first post i am using freebsd 5.4 pl13.
0
 
LVL 7

Expert Comment

by:wnross
ID: 16449048
Whoops, sorry rares, long discussion does that sometimes

Check out http://www.trustedbsd.org/
its inspired by the NSA's FLASK/SELinux project.

Cheers,
-Bill
0
 

Author Comment

by:rares_dumitrescu
ID: 16450132
okay problem solved . system will be re4installed and repartitioned.
0
 
LVL 7

Expert Comment

by:wnross
ID: 16450209
Hope we were of help

Cheers
-Bill
0

Featured Post

Nothing ever in the clear!

This technical paper will help you implement VMware’s VM encryption as well as implement Veeam encryption which together will achieve the nothing ever in the clear goal. If a bad guy steals VMs, backups or traffic they get nothing.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

​Being a Managed Services Provider (MSP) has presented you  with challenges in the past— and by meeting those challenges you’ve reaped the rewards of success.  In 2014, challenges and rewards remain; but as the Internet and business environment evol…
BIND is the most widely used Name Server. A Name Server is the one that translates a site name to it's IP address. There is a new bug in BIND (https://kb.isc.org/article/AA-01272), affecting all versions of BIND 9 from BIND 9.1.0 (inclusive) thro…
this video summaries big data hadoop online training demo (http://onlineitguru.com/big-data-hadoop-online-training-placement.html) , and covers basics in big data hadoop .
Whether it be Exchange Server Crash Issues, Dirty Shutdown Errors or Failed to mount error, Stellar Phoenix Mailbox Exchange Recovery has always got your back. With the help of its easy to understand user interface and 3 simple steps recovery proced…

839 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question