Livedump filesystem almost full

Posted on 2013-12-17
Last Modified: 2014-01-03
I got a error message, "Livedump filesystem almost full",
Livedump filesystem almost full

        Recommended Actions
        Expand filesystem or delete dumps that are not needed.
Live dump threshold may need to be updated. To change the
livedump threshold, use the command 'dumpctrl freespc=[0-100]'.
freespc is in percent.

Detail Data
percent free
desired percent free

What should be the proper procedure to fix this problem.

Question by:sams20
  • 7
  • 7
LVL 68

Expert Comment

ID: 39724018

are there indeed any livedumps? Please run

ls -l /var/adm/ras/livedump

What do you see? If there is nothing then your /var filesystem is gradually filling up due to other data being accumulated.

Either increase its size by means of the command

chfs -a size=+nnnM /var

where nnn is the desired number of additional megabytes

or clean up /var.

Particularly check /var/tmp (snmpdxxx.log can grow big) or /var/adm (wtmp is usually very big).

Both files can be emptied (if you don't need the data for actual debugging or user tracing purposes) by means of

> /var/tmp/snmpd*.log*
> /var/adm/wtmp


Author Comment

ID: 39725667
After run this, " ls -l /var/adm/ras/livedump" command I did see nothing. I took backup for wtmp, sulog, cron log files and did cleanup. After cleanup those files I got more space in /var filesystem.
LVL 68

Expert Comment

ID: 39726255
Sorry, I forgot that the newer AIXes have their own filesystem for livedump.

Anyway, if you see indeed nothing with "ls -l /var/adm/ras/livedump" and get the errpt message nonetheless there must be something wrong with the livedump filesystem.

What do you see with

df -m /var/adm/ras/livedump # Check "%Used"

and with

dumpctrl -s # Any registered dumps?

To repair a possible filesystem damage you could do

umount /dev/livedump # you can always umount it without worries!
fsck -y /dev/livedump
mount /dev/livedump
df -m /var/adm/ras/livedump
# Any change?
Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.


Author Comment

ID: 39726515
After run those command mentioned below, I got this,

# df -m /var/adm/ras/livedump
Filesystem    MB blocks      Free %Used    Iused %Iused Mounted on
/dev/hd9var      512.00    xxx.xx   64%     xxxx     7% /var


# dumpctrl -s
ODM object not found.
LVL 68

Expert Comment

ID: 39726586

so you're running indeed an early AIX 6.1 version which doesn't yet have its own /dev/livedump filesystem - or did someone delete it, perhaps to make room in rootvg?

Anyway, you should be fine with the free space now available in /var and the errpt message should not appear any longer.

By the way, why did you obfuscate the "Free" and "Iused" values? Do you think that's Top Secret?


Author Comment

ID: 39728334
Not top secret, just for safekeeping (security reason) my system data.
I have checked my system os level , AIX 6.1 TL8
When I type lsvg -l rootvg command, there is nothing about /dev/livedump filesystem. But I find out there is a directory in /var/adm/ras/livedump. In this case, shall I need to fix dump device with new filesystem /dev/livedump and mount point /var/adm/ras/livedump.
When I type sysdumpdev -l, I didn't see any dump device named /dev/livedump as a primary or secondary.

LVL 68

Expert Comment

ID: 39728724
"sysdumpdev" is meant for working on legacy/system dump devices. "livedump" is a new feature and its appropriate tool is "dumpctrl":

dumpctrl -ql

shows the "livedump" environment, and

dumpctrl -qs

shows the legacy/system dump environment, similar to "sysdumpdev -l".

You can run "dumpctrl -qc" to check which components use the new livedump feature and which ones use the legacy/system dump way (some use both). Check the third (livedump) and fourth (legacy/system) columns for "ON" or "OFF".

Your AIX 6.1 is at a rather high level (TL 8 is from October 2012, TL 9 from October 2013 is newest), so there should normally be a separate livedump filesystem. It is not imperatively required - having it in place is just a good way to keep /var from getting filled up with dump data.

So creating an own FS is not a bad idea, and if you have sufficient free space in rootvg you should do it.

Standard size is 256 MB, the standard LV is "/dev/livedump", fstype is jfs2, no v2 attributes, no encryption, no inline logging. The mount point is, as you already know, /var/adm/ras/livedump.

If you need more assistance in creating such an FS please let me know.


Author Comment

ID: 39732181
After running this dumpctrl command, I got this,

# dumpctrl -ql
Live Dump Enabled:                      yes
Live Dump Directory:                    /var/adm/ras/livedump
Live Dump Free Space Threshold:         25%
Live Dump Global Level:                 3
Live Dump Heap Size:                    0 MB (0 indicates default heap size)
Live Dump Duplicate Suppression Type:   all
Live Dump Max System Freeze Interval:100ms

# dumpctrl -qs
Dump Legacy Components:                 yes
System Dump Global Level:               3
System Dump Copy Directory:             /xxxx_dump
Display Boot Time Menu on Dump Failure: no
Allow Dump on Keyboard Sequence:        yes
Primary Dump Device:                    /dev/lg_dumplv
Secondary Dump Device:                  /dev/sysdumpnull

# sysdumpdev -l
primary              /dev/lg_dumplv
secondary            /dev/sysdumpnull
copy directory       /xxxx_dump
forced copy flag     FALSE
always allow dump    TRUE
dump compression     ON
type of dump         traditional
LVL 68

Expert Comment

ID: 39732202
OK, looks good.

The only thing you'll have to decide is whether to create a separate FS or not.
I'd recommend doing so, but it's your decision.

How much free space is in your rootvg? Check the output of

lsvg rootvg

for "FREE PPs"

Author Comment

ID: 39732276
From, lsvg rootvg command I got,

FREE PPs : 390 ( 99840 megabytes)
LVL 68

Expert Comment

ID: 39732344
So I think from your 99840 Megabytes (~ 97.5 Gigabytes) you could well sacrifice 250 MB for livedump.

Go on!

Author Comment

ID: 39737224
I am not clear about next steps, according to yours instruction, I will create file system /dev/livdumplv and mount with /var/adm/ras/livedump mount point which already there. What should be the next steps after mounting the mount point.
LVL 68

Accepted Solution

woolmilkporc earned 500 total points
ID: 39737731
The LV name is "livedump" (not "livdumplv"!)

There are no next steps. Just make sure that the mountpoint "/var/adm/ras/livedump" is indeed empty before mounting the new FS.

That's all. There are no background processes which must be informed about the new environment.

For your convenience, here are the LV/FS characteristics again:

Standard size is 256 MB, the standard LV is "/dev/livedump", fstype is jfs2, no v2 attributes, no encryption, no inline logging. The mount point is, as you already know, /var/adm/ras/livedump.

Author Comment

ID: 39738410
Thank you for your instruction, I will create /dev/livedump filesystem with 256MB size.


Featured Post

Networking for the Cloud Era

Join Microsoft and Riverbed for a discussion and demonstration of enhancements to SteelConnect:
-One-click orchestration and cloud connectivity in Azure environments
-Tight integration of SD-WAN and WAN optimization capabilities
-Scalability and resiliency equal to a data center

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Hello fellow BSD lovers, I've created a patch process for patching openjdk6 for BSD (FreeBSD specifically), although I tried to keep all BSD versions in mind when creating my patch. Welcome to OpenJDK6 on BSD First let me start with a little …
This tech tip describes how to install the Solaris Operating System from a tape backup that was created using the Solaris flash archive utility. I have used this procedure on the Solaris 8 and 9 OS, and it shoudl also work well on the Solaris 10 rel…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

790 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question