• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2704
  • Last Modified:

Livedump filesystem almost full

I got a error message, "Livedump filesystem almost full",
Livedump filesystem almost full

        Recommended Actions
        Expand filesystem or delete dumps that are not needed.
Live dump threshold may need to be updated. To change the
livedump threshold, use the command 'dumpctrl freespc=[0-100]'.
freespc is in percent.

Detail Data
percent free
desired percent free

What should be the proper procedure to fix this problem.

  • 7
  • 7
1 Solution

are there indeed any livedumps? Please run

ls -l /var/adm/ras/livedump

What do you see? If there is nothing then your /var filesystem is gradually filling up due to other data being accumulated.

Either increase its size by means of the command

chfs -a size=+nnnM /var

where nnn is the desired number of additional megabytes

or clean up /var.

Particularly check /var/tmp (snmpdxxx.log can grow big) or /var/adm (wtmp is usually very big).

Both files can be emptied (if you don't need the data for actual debugging or user tracing purposes) by means of

> /var/tmp/snmpd*.log*
> /var/adm/wtmp

sams20Author Commented:
After run this, " ls -l /var/adm/ras/livedump" command I did see nothing. I took backup for wtmp, sulog, cron log files and did cleanup. After cleanup those files I got more space in /var filesystem.
Sorry, I forgot that the newer AIXes have their own filesystem for livedump.

Anyway, if you see indeed nothing with "ls -l /var/adm/ras/livedump" and get the errpt message nonetheless there must be something wrong with the livedump filesystem.

What do you see with

df -m /var/adm/ras/livedump # Check "%Used"

and with

dumpctrl -s # Any registered dumps?

To repair a possible filesystem damage you could do

umount /dev/livedump # you can always umount it without worries!
fsck -y /dev/livedump
mount /dev/livedump
df -m /var/adm/ras/livedump
# Any change?
Cloud Class® Course: Ruby Fundamentals

This course will introduce you to Ruby, as well as teach you about classes, methods, variables, data structures, loops, enumerable methods, and finishing touches.

sams20Author Commented:
After run those command mentioned below, I got this,

# df -m /var/adm/ras/livedump
Filesystem    MB blocks      Free %Used    Iused %Iused Mounted on
/dev/hd9var      512.00    xxx.xx   64%     xxxx     7% /var


# dumpctrl -s
ODM object not found.

so you're running indeed an early AIX 6.1 version which doesn't yet have its own /dev/livedump filesystem - or did someone delete it, perhaps to make room in rootvg?

Anyway, you should be fine with the free space now available in /var and the errpt message should not appear any longer.

By the way, why did you obfuscate the "Free" and "Iused" values? Do you think that's Top Secret?

sams20Author Commented:
Not top secret, just for safekeeping (security reason) my system data.
I have checked my system os level , AIX 6.1 TL8
When I type lsvg -l rootvg command, there is nothing about /dev/livedump filesystem. But I find out there is a directory in /var/adm/ras/livedump. In this case, shall I need to fix dump device with new filesystem /dev/livedump and mount point /var/adm/ras/livedump.
When I type sysdumpdev -l, I didn't see any dump device named /dev/livedump as a primary or secondary.

"sysdumpdev" is meant for working on legacy/system dump devices. "livedump" is a new feature and its appropriate tool is "dumpctrl":

dumpctrl -ql

shows the "livedump" environment, and

dumpctrl -qs

shows the legacy/system dump environment, similar to "sysdumpdev -l".

You can run "dumpctrl -qc" to check which components use the new livedump feature and which ones use the legacy/system dump way (some use both). Check the third (livedump) and fourth (legacy/system) columns for "ON" or "OFF".

Your AIX 6.1 is at a rather high level (TL 8 is from October 2012, TL 9 from October 2013 is newest), so there should normally be a separate livedump filesystem. It is not imperatively required - having it in place is just a good way to keep /var from getting filled up with dump data.

So creating an own FS is not a bad idea, and if you have sufficient free space in rootvg you should do it.

Standard size is 256 MB, the standard LV is "/dev/livedump", fstype is jfs2, no v2 attributes, no encryption, no inline logging. The mount point is, as you already know, /var/adm/ras/livedump.

If you need more assistance in creating such an FS please let me know.

sams20Author Commented:
After running this dumpctrl command, I got this,

# dumpctrl -ql
Live Dump Enabled:                      yes
Live Dump Directory:                    /var/adm/ras/livedump
Live Dump Free Space Threshold:         25%
Live Dump Global Level:                 3
Live Dump Heap Size:                    0 MB (0 indicates default heap size)
Live Dump Duplicate Suppression Type:   all
Live Dump Max System Freeze Interval:100ms

# dumpctrl -qs
Dump Legacy Components:                 yes
System Dump Global Level:               3
System Dump Copy Directory:             /xxxx_dump
Display Boot Time Menu on Dump Failure: no
Allow Dump on Keyboard Sequence:        yes
Primary Dump Device:                    /dev/lg_dumplv
Secondary Dump Device:                  /dev/sysdumpnull

# sysdumpdev -l
primary              /dev/lg_dumplv
secondary            /dev/sysdumpnull
copy directory       /xxxx_dump
forced copy flag     FALSE
always allow dump    TRUE
dump compression     ON
type of dump         traditional
OK, looks good.

The only thing you'll have to decide is whether to create a separate FS or not.
I'd recommend doing so, but it's your decision.

How much free space is in your rootvg? Check the output of

lsvg rootvg

for "FREE PPs"
sams20Author Commented:
From, lsvg rootvg command I got,

FREE PPs : 390 ( 99840 megabytes)
So I think from your 99840 Megabytes (~ 97.5 Gigabytes) you could well sacrifice 250 MB for livedump.

Go on!
sams20Author Commented:
I am not clear about next steps, according to yours instruction, I will create file system /dev/livdumplv and mount with /var/adm/ras/livedump mount point which already there. What should be the next steps after mounting the mount point.
The LV name is "livedump" (not "livdumplv"!)

There are no next steps. Just make sure that the mountpoint "/var/adm/ras/livedump" is indeed empty before mounting the new FS.

That's all. There are no background processes which must be informed about the new environment.

For your convenience, here are the LV/FS characteristics again:

Standard size is 256 MB, the standard LV is "/dev/livedump", fstype is jfs2, no v2 attributes, no encryption, no inline logging. The mount point is, as you already know, /var/adm/ras/livedump.
sams20Author Commented:
Thank you for your instruction, I will create /dev/livedump filesystem with 256MB size.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: Microsoft Azure 2017

Azure has a changed a lot since it was originally introduce by adding new services and features. Do you know everything you need to about Azure? This course will teach you about the Azure App Service, monitoring and application insights, DevOps, and Team Services.

  • 7
  • 7
Tackle projects and never again get stuck behind a technical roadblock.
Join Now