DoS bug/exploit

Hi, i have a problem. i am running a shell server. Freebsd 5.4 pl13 . i have quotas enabled - 150 mb disk space / 1500 inodes . some user yesterday, built a program, that created 400 folders . in each folders they were 999 files with very long names. The weird thing is that they had all the same inode .

this is ls -lai ... so you see that it has the same inode .. each file . the thing is that server will crash if runs sme periodic tasks like ... updatedb or something .
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx074
1319375 -rw------- 30000 user users 0 Apr 10 23:31 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx075
1319375 -rw------- 30000 user users 0 Apr 10 23:31 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx076
1319375 -rw------- 30000 user users 0 Apr 10 23:31 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx077

the whole idea is that by creating those files with the same inode, the quota stuff saw only 800 inodes, as files had the same inodes. so ... above all my protection was bypassed. any idea how to work this around ?

the thing is i have the proggie that did this ... and i tried some reverse engineering on it, and ... what i got is ... a fragment of procedures ... now im very very very smakll at c coding ..

need help on how to decompie the file ... and to prevent this from happening.

deleting shell access or not giving shell access is not an option for me. i need to secure the box not to delay another happening


_start()
{

(save)ebp;
ebp = esp;
(save)edi;
(save)esi;
(save)ebx;
esp = esp - 0xc;
ecx = edx;
esi = ebp + 8;
ebx = *(esi - 4);
environ = esi + ebx * 4 + 4;
if(ebx > 0 && *(ebp + 8) != 0) {
eax = *(ebp + 8);
__progname = eax;
edx = __progname;
if(*__progname != 0) {
do {
if(*edx == "") {
__progname = edx + 1;
}
edx = edx + 1;
} while(*edx != 0);
}
}
if(134519772 != 0) {
esp = esp - 0xc;
(save)ecx;
L08048584();
esp = esp + 0x10;
} else {
L08048564();
}
esp = esp - 0xc;
(save)_fini;
L08048584();
_init();
esp = esp - 4;
(save)edi;
(save)esi;
(save)ebx;
main();
esp = esp + 0x14;
(save)eax;
L080485B4();
}

/* Procedure: 0x08048664 - 0x080486B5
* Argument size: -16
* Local size: 8
* Save regs size: 0
* Called by:
* _fini()
*/

__do_global_dtors_aux()
{

if(completed.1 == 0) {
eax = *p.0;
do {
p.0 = p.0 + 4;
*eax();
eax = *p.0;
} while(eax != 0);
eax = 0;
if(0 != 0) {
esp = esp - 0xc;
(save)__EH_FRAME_BEGIN__;
eax = L00000000();
}
completed.1 = 1;
}
}

/* Procedure: 0x080486B6 - 0x080486B7
* Argument size: 0
* Local size: 0
* Save regs size: 0
*/

L080486b6()
{

}

/* Procedure: 0x080486B8 - 0x08048701
* Argument size: -8
* Local size: 8
* Save regs size: 0
* Called by:
* _init()
*/

frame_dummy()
{

eax = 0;
if(0 != 0) {
esp = esp - 8;
(save)object.2;
(save)__EH_FRAME_BEGIN__;
eax = L00000000();
}
if(__JCR_LIST__ != 0) {
eax = 0;
if(0 != 0) {
esp = esp - 0xc;
(save)__JCR_LIST__;
L00000000();
}
}
}

/* Procedure: 0x08048702 - 0x08048703
* Argument size: 0
* Local size: 0
* Save regs size: 0
*/

L08048702()
{

}

/* Procedure: 0x08048704 - 0x08048A7D
* Argument size: -1840
* Local size: 920
* Save regs size: 0
* Called by:
* _start()
*/

main()
{
/* unknown */ void Vfffffc74;
/* unknown */ void Vfffffc78;
/* unknown */ void Vfffffce0;
/* unknown */ void Vfffffce4;
/* unknown */ void Vfffffce8;
/* unknown */ void Vfffffde8;
/* unknown */ void Vfffffee8;
/* unknown */ void Vfffffef8;
/* unknown */ void Vffffffed;

esp = esp & -16;
esp = esp - ("ec/ld-elf.so.1" >> 4 > "c/ld-elf.so.1"));
esp = esp - 8;
(save)0x1ed;
(save) & Vfffffee8;
esp = esp + 0x10;
if(L080485C4() < 0) {
esp = esp - 4;
(save) & Vfffffee8;
(save)"mkdir(%s) failed\n";
(save)__stderrp;
L08048544();
esp = esp + 0x10;
esp = esp - 0xc;
(save)1;
L080485B4();
}
}
esp = esp - 0xc;
ecx = Vfffffce0;
eax = 274877907;
asm("imul ecx");
edx = edx >> 6;
edx = edx - (ecx >> "c/ld-elf.so.1");
eax = (edx
rares_dumitrescuAsked:
Who is Participating?
 
wnrossCommented:
Lets look at attacker profile and privileges again: was this a general staff account?  Certainly you could do something like

/exports/staff
/exports/developer

mount localhost:/exports/staff  /home/staff  -t nfs -onoexec
mount localhost:/exports/developer /home/team -t nfs -ouser,exec,no_root_squash
mount /tmp -oremount,noexec
rmdir /var/tmp
ln -s /tmp /var/tmp

Good luck creating executables with that setup
0
 
ppfoongCommented:

You should have a proper resource usage policy / security policy that if any user violate, their access will be revoked and they will be penalised for any lost or damage done.

Fine the culprit user and delete his/her account!

If you want to solve it the technical way, report the problem to the FreeBSD development community, or switch to another distribution that does not have the problem.

0
 
wnrossCommented:
Well, if these are all the same inode, then we're looking at a hude number of hardlinks, which are harmless.
As your quota is showing: only 800 inodes and some presumably 24MB of storage.

As for cron, if it's bogging down in these folders, try launching updatedb as a background task
* * * * * updatedb &

that way your cron doesn't hang waiting for the locate subsystem to finish.  Alternately prune the
directories from updatedb...there's a compelling argument that says we don't need to know where our
users files are located and they are free to use conventional tools such as "find".  

(Assuming non-staff environments, such as web accounts, obviously staff members need to locate
those spreadsheets and should not be pruned).

Since these are hardlinks, rm -f will clean up the files for you lickety-split.

As for the program, it looks like your decompile attepts gave up too early, have the jump points
came back blank.  All is not lost, however:

Whats the name of the binary?
most script attacks come from common code which is easy to track in google

ldd ./hackerprog
will dump the shared libraries and give you a clue as to the program intentions
libncurses == console monitoring program
libpcap == network sniffer

and so on

strings ./hackerprog | more
Careful analysis will give text prompts, network addresses, even the author on occasion.

Look for the source code as well, check the /tmp folder (also /var/tmp; /var/cache/apache; and anywhere in
the user folder) and be wary of  folder paths such as ".. " (note the space)

These hidden in plain sight folders can lead you to the original .c files used to create the binary.

Cheers,
-Bill
0
How do you know if your security is working?

Protecting your business doesn’t have to mean sifting through endless alerts and notifications. With WatchGuard Total Security Suite, you can feel confident that your business is secure, meaning you can get back to the things that have been sitting on your to-do list.

 
rares_dumitrescuAuthor Commented:
program name was renamed to eggdrop :) so i guess that won't be of help. source code. ... he wasn't that stupid to let it there. the idea is ... and main question. how do you create 1000 files, which have the same inode ? just how ? so i could prevent that. as i said deleting an user is NOT an option. just delays the pain. i want to solve it not move forward with an insecure system geez.
0
 
ppfoongCommented:

You can create as many file as you like, with a simple looping function in a simple shell script.

0
 
ppfoongCommented:

To prevent malicious users, you can mount the user home directory as non-executable and non-suid, provided the home directory of all users is in a separate partition.

0
 
ppfoongCommented:

To answer your other question: a hard link will create file with same inode.

0
 
rares_dumitrescuAuthor Commented:
okay ... but as you see the files are no links. are simply files .

1319375 -rw------- 30000 user users 0 Apr 10 23:31 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx076
1319375 -rw------- 30000 user users 0 Apr 10 23:31 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx077

Second of all . no you can't create as many files as you like if you set limitations in quota for number of inodes. which i already did .. set limit to 1500 files. still this method of creating files has overpassed my protection. those are not hard links as you see. question is how were they created ? :)
0
 
ppfoongCommented:

A hard link will appear as a file. A soft link will appear as a link.

Try this:

touch file1
ln file1 file2
ln file1 file3
ln file1 file4


0
 
rares_dumitrescuAuthor Commented:
fuck you are right .....

 215666 -rw-------    4 user   users           0 Apr 13 13:47 file1
 215666 -rw-------    4 user   users           0 Apr 13 13:47 file2
 215666 -rw-------    4 user   users           0 Apr 13 13:47 file3
 215666 -rw-------    4 user   users           0 Apr 13 13:47 file4


so all i have to do ... is restrict access to the ln command.... and they would not be able to do it aight ?
0
 
rares_dumitrescuAuthor Commented:
i restricted access to the ln command, but still works ...
0
 
wnrossCommented:
for ((x = 0; $x <5; x++)) do
   ln bigfile xxxxxxx$x
done

(Assuming bash shell)

Hardlinks are directory entries which use the same inode as another file.
Linux will generally disallow hardlinks of directorys, and since the
files share the same inode, you may not have hardlinks which span
multiple filesystems.

So, your user went nuts in the directory entries department, but actually has not created any more than 800 files.

Ok. so what does ldd and strngs give us?  And assume nothing about the intelligence of your attacker, even
someone brilliant will leave traces behind

Cheers,
-Bill
0
 
rares_dumitrescuAuthor Commented:
i restricted access to ln and touch commands, still the exploit uses something else ....

touch file1
-su: /usr/bin/touch: Permission denied

ln file1 file2
-su: /bin/ln: Permission denied



&#9492;&#9472;(~/eggdrop/modules/eggdrop.so)-> ./eggdrop
00/04000...

just built 4000 files ...

0
 
rares_dumitrescuAuthor Commented:
ldd eggdrop
eggdrop:
        libc.so.5 => /lib/libc.so.5 (0x28076000)


trings eggdrop
/libexec/ld-elf.so.1
FreeBSD
libc.so.5
__stdoutp
_DYNAMIC
__stderrp
creat
fflush
_init
environ
fprintf
__deregister_frame_info
__progname
memset
_init_tls
_fini
sprintf
atexit
stat
_GLOBAL_OFFSET_TABLE_
link
_Jv_RegisterClasses
mkdir
__register_frame_info
_edata
__bss_start
_end
$FreeBSD: src/lib/csu/i386-elf/crti.S,v 1.6 2002/05/15 04:19:49 obrien Exp $
%02d/%05d...
%02d-%02d
mkdir(%s) failed
%s/%s%03d
creat(%s) failed
stat(%s) failed
link(%s,%s) failed
finished successfully
$FreeBSD: src/lib/csu/i386-elf/crtn.S,v 1.5 2002/05/15 04:19:49 obrien Exp $



0
 
wnrossCommented:
linking is an OS capability, restrict ln all you like, they can still do it via a program using the link() function.

One option is to restrict access to compiler tools

groupadd ctools
chown root:ctools /usr/bin/cc /usr/bin/gcc-3.4.1 /usr/bin/protoize /usr/bin/ld /usr/bin/ar
chmod 750 /usr/bin/cc /usr/bin/gcc-3.4.1 /usr/bin/protoize /usr/bin/ld /usr/bin/ar

then add any real developers to the ctools group.

Ditto for ntools and programs like ping, nmap and the like.

-Bill
0
 
rares_dumitrescuAuthor Commented:
hummz the idea is that legit users that have nothing to do with this .. like 99 % of users on the server, need access to compile stuff ...
0
 
rares_dumitrescuAuthor Commented:
can i restrict access for example .. to the link function ? in compiler ?
0
 
wnrossCommented:
True, but apache doesn't nor do ftp-only users, nor do managers, administrative staff, etc.
What account did she use to get in? Was it a casual user or one of your development team?

This isn't a very large program, is it?
Looks like your ok, based on the output the program has no network capability,
it just creats a heck of a lot of hard links.

Only one dependency: the standard c library...so he/she is not doing anything special
as for os routines, we see that he calls
mkdir to create the directorties
creat() to create the initial files
sprintf fprintf and fflush to twrite to the creat()'d files
stat() to confirm the file was created (can't be too creful)
then link() to create a hardlink.  

Based on what I've seen above + the ldd and strings, thats about it.

He/she probably wrote this by hand rather than scripting it since its such a simple program

1) This program does what we think it does
2) No further footprint exists (here) as to the attack profile
3) Restricting access to compiler and network tools will hamper future attacks (unless they change languages)

oh, add curl and wget to your ntools groups
Usual attack profile goes:
- gain shell access
- wget somehakersite/tools.tgz
- tar -xzf tools.tgz
- gcc hackme.c
- go to town with the destruction and stuff

Cheers,
-Bill
0
 
ppfoongCommented:

As long as you allow executable to be put into and run from the user directory, and/or the /tmp directory, there is nothing much you can prevent.

The user can always do the compiling in another machine, and transfer the file over. Even if file transfer is not possible, the user can get the machine code of the compiled program, and produce the same codes in your server with a simple script/program.

0
 
wnrossCommented:
I don't know if *simple* script program is accurate, but you are right there ppfoong,  besides this still does not rule out attacks using other languages such as perl, php, python, and the like, non of which require compilation.
0
 
rares_dumitrescuAuthor Commented:
well yes the program is compiled already. this is so damn true. the program was compiled and brought here ...  are there any limitations that i can do to deny the use of those syscalls  for a certain login class of users ?
0
 
ppfoongCommented:

The simple script I said just need to make use of echo to escape codes.

As I said before, unless you limit them from executing program from their home directory, it is unlikely to stop them running such program.

0
 
rares_dumitrescuAuthor Commented:
people must be able to execute programs ... damn ... this means .. it can't be worked around ...
0
 
ppfoongCommented:

Back to square one. Use resource usage policy / security policy to restrict the user.

0
 
wnrossCommented:
Also was this a break in or an in-house job?
0
 
rares_dumitrescuAuthor Commented:
well ... the idea is that users have shell access . it is a shell hosting server. they must be able to execute proggies.  i know that nosuid noexec stuff. but if i do it for all users, server will render useless. ...

this was an in house job. someone bought a shell account and used this program to screw me up. the idea is that he can do it over and over again .. all he has to do is pay and get another shell. that is why i said that deleting the user is not the solution.


back to square one. there are limitations already. 1500 max files and 150 mb disk space. limited by quota. but as he creates each 1000 files with the same inode ... hard links, the quota system will see 400000 files as 400 files ... and will not block creating more ....
0
 
wnrossCommented:
Then I suggest let them do it: there's no harm if they want to clutter their account that way.

In the meantime, as I mentioned before, prune updatedb from looking in your user's files

/etc/updatedb.conf
# PRUNEPATHS="/proc,/tmp,/var/tmp,/usr/tmp,/net,/afs,/mnt"
PRUNEPATHS="/proc,/tmp,/var/tmp,/usr/tmp,/net,/afs,/mnt,/home/customers"

this will keep the cron task out of there.   The user will end up punishing themselves since
simple ls commands will take minutes or even hours to complete.

the locate command is great, but there's no reason at all for it to be used by your clients, not as long as
they can only create files in their home directories anyway.

Cheers,
-Bill
0
 
rares_dumitrescuAuthor Commented:
i thought of something ... well .. mounting the /home in noexec, and all legit program binaries to be run from /bin . i mean users should not have the binaries there, just run them from /bin /bin/psybnc /bin/eggdrop ... and mounting the /tmp in noexec/nosuid will fix this aight ?
0
 
ppfoongCommented:

This is what I suggested in the first place.

Of course you need to have /bin /sbin /usr/bin /usr/sbin /usr/local/bin /usr/local/sbin to run executable, but any directory that is user writable, including the /home and /tmp, should be mounted with noexec and nosuid.

0
 
wnrossCommented:
That was my implication, although if you have people requiring use of compilers, then they will need some kind of bin folder
that they can write to as well.

Personally I would mount  your tmp directory that way anyways.

Here's a thought, although it takes some setup: use SELinux,  then you can fine tune your policies according to need
such as:
stat_file_perms      Permissions to call stat or access on a file.
x_file_perms      Permissions to execute a file.
r_file_perms      Permissions to read a file.
rx_file_perms      Permissions to read and execute a file.
rw_file_perms      Permissions to read and write a file.
ra_file_perms      Permissions to read and append to a file.
link_file_perms      Permissions to link, unlink, or rename a file.   <------------------  looky here
create_file_perms      Permissions to create, access, and delete a file.
r_dir_perms      Permissions to read and search a directory.
rw_dir_perms      Permissions to read and modify a directory.
ra_dir_perms      Permissions to read and add entries to a directory.
create_dir_perms      Permissions to create, access, and delete a directory.
mount_fs_perms      Permissions to mount and unmount a filesystem.

Since RedHat/CentOS/Whitebox linux support SElinux out of hte box, this might not be a bad way of going.
Cheers
-Bill
0
 
wnrossCommented:
0
 
xDamoxCommented:
Hi,

Could you send me the program I want to analysis this problem, anyways I have a small solution:

http://www.grsecurity.net/features.php

grsecurity will provide the following help:

Symlink/hardlink restrictions to prevent /tmp races
0
 
xDamoxCommented:
Heres a little more info, grsecurity looks to be the way:

http://private.addcom.de/nordi/linux/suse_seclink.html

ignore the suse bit it wil lapply for all
0
 
rares_dumitrescuAuthor Commented:
i know the grsecurity but as you guys would find out by reading the first post i am using freebsd 5.4 pl13.
0
 
wnrossCommented:
Whoops, sorry rares, long discussion does that sometimes

Check out http://www.trustedbsd.org/
its inspired by the NSA's FLASK/SELinux project.

Cheers,
-Bill
0
 
rares_dumitrescuAuthor Commented:
okay problem solved . system will be re4installed and repartitioned.
0
 
wnrossCommented:
Hope we were of help

Cheers
-Bill
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.