Sybase Engine has over 85% kernel usage on rebuilding cluster index: Sybase kernel usage

by: alpmoonPosted on 2009-07-15 at 18:00:59ID: 24865585

If "max parallel degree" is 1,high cpu usage cannot be due to rebuilding a clustered index. There must be a lot of other activities. Have you checked processes by sp_who or any other tool?

Also can you confirm number of CPU's by running $SYBASE/SYSAM-2_0/bin/cpuinfo

by: PinkSandPosted on 2009-07-15 at 18:37:38ID: 24865729

Hi alpmoon,

Thanks for pick up the question.

Here is the output from cpuinfo:
Detected 32 logical processors, 32 cores, in 8 chips

Our client applications are also running on the same box. The plan is to use 1 cpu per sybase engine, and 1 per client application. We checked that the CPU idle is mostly over 70% until we create a clustered index. We repeated the test, and every time the CPU idlle went down to almost 0 when the index is created on a table with more than 1 million rows. We defintely would not create index on a big table during normal operation hours. My main concern is why one Sybase engine can have over 40% CPU usage on a 32 CPU machine if max parallel degree is set to 1.

by: alpmoonPosted on 2009-07-15 at 19:52:42ID: 24865987

It is not really possible to use 40% cpu power for serial index build. Can you provide sp_who output taken during index rebuild. Also how long does it take to build the clustered index?

Sybase might be using cpu power excessively to check whether disk io is completed or not. But, this shouldn't be at that level.

by: Joe_WoodhousePosted on 2009-07-16 at 02:15:26ID: 24867603

Max parallel degree = 1 does not mean Sybase is constrained to use only one CPU. It means only one spid will be running the "create index" job, but that work can migrate from one engine to another... there is no engine affinity in Sybase unless you have explicitly set it up (generally not recommended).

So first things first, the work could actually get put onto every engine, one per time slice, one after each other.

Second - your stats are CPU as reported by the operating system, which is not the same thing as CPU as used by Sybase ASE. The O/S only tells you whether a CPU had an instruction on its register, even if that instruction was a NOP while Sybase sat and spinned and waited.

We really need to see the results of something like sp_sysmon while your create index job is running. In a perfect world you would ensure nothing else could possibly be running so we knew for certain we were seeing only the create index job.

Also - I'm not entirely sure setting max parallel degree to 1 will force this to be serial. Max parallel degree is about limiting the number of child worker processes during a parallel query. I am not convinced parallel create index is considered parallel query. The way to be sure is to also set number of worker processes to 0 to see if that changes anything.

by: PinkSandPosted on 2009-07-16 at 07:29:50ID: 24869462

Hi Joe & alpmoon,
The index coubd be built in about a minute.

The number of worker processes is changed from 40 to 2, and creating a clustered index still causes the high CPU usage.
Unfortunately, I could not take down the other processes while testing. One thing I noticed is the during the index rebuild, it is always the same 2 engines that got very busy, while other engines has very low cpu usage.

Thanks!


A snapshot of TOP:
 
load averages: 87.5, 23.3, 9.65; up 1+16:53:47 16:33:56
133 processes: 130 sleeping, 3 on cpu
CPU states: 12.0% idle, 4.5% user, 83.5% kernel, 0.0% iowait, 0.0% swap
Memory: 32G phys mem, 772M free mem, 33G total swap, 33G free swap
 
PID USERNAME LWP PRI NICE SIZE RES STATE TIME CPU COMMAND
1179 sybase 264 0 0 19G 533M cpu/10 286:31 78.73% dataserver
1182 sybase 264 50 0 19G 533M sleep 350:59 3.88% dataserver
1181 sybase 264 0 0 19G 533M cpu/23 244:48 0.07% dataserver
1193 sybase 264 59 0 19G 533M sleep 322:31 0.06% dataserver
1185 sybase 156 59 0 19G 532M sleep 261:53 0.06% dataserver
1102 prod 3 59 0 22M 7560K sleep 63:08 0.06% tkr_rcv
1177 sybase 264 59 0 19G 535M sleep 367:32 0.03% dataserver
1104 tkr 1 53 0 20M 13M sleep 33:31 0.03% dcd
8813 helenw 1 52 0 3204K 1876K sleep 0:00 0.03% top
1184 sybase 193 59 0 19G 532M sleep 229:46 0.02% dataserver
25843 vap 7 59 0 303M 298M sleep 9:01 0.01% vap_genx
25805 vap 7 59 0 207M 203M sleep 9:45 0.01% vap_genx
25830 vap 7 49 0 207M 203M sleep 8:11 0.01% vap_genx
1187 sybase 89 59 0 19G 531M sleep 377:35 0.01% dataserver
25928 vap 7 52 0 308M 304M sleep 15:55 0.01% vap
1183 sybase 138 59 0 19G 532M sleep 187:04 0.01% dataserver
1192 sybase 260 54 0 19G 533M sleep 277:32 0.01% dataserver
25916 vap 7 59 0 404M 400M sleep 19:06 0.01% vap
1190 sybase 72 59 0 19G 531M sleep 210:43 0.01% dataserver
1180 sybase 86 59 0 19G 531M sleep 212:30 0.01% dataserver
 
This is a sys mon output taken from previous test. But the sample was taken for 10 minutes:
 
===============================================================================
Sybase Adaptive Server Enterprise System Performance Report
===============================================================================
 
Server Version: Adaptive Server Enterprise/15.0.3/EBF 16558 ESD#1/P/Sola
Server Name: amdvapp2
Run Date: Jul 15, 2009
Sampling Started at: Jul 15, 2009 12:00:29
Sampling Ended at: Jul 15, 2009 12:10:30
Sample Interval: 00:10:01
Sample Mode: No Clear
Counters Last Cleared: Jul 13, 2009 23:47:49
 
===============================================================================
 
Kernel Utilization
------------------
 
Your Runnable Process Search Count is set to 2000
and I/O Polling Process Count is set to 5
 
Engine Busy Utilization CPU Busy I/O Busy Idle
------------------------ -------- -------- --------
Engine 0 8.0 % 18.6 % 73.4 %
Engine 1 9.3 % 18.3 % 72.3 %
Engine 2 3.9 % 10.5 % 85.6 %
Engine 3 6.5 % 12.2 % 81.3 %
Engine 4 9.1 % 25.9 % 64.9 %
Engine 5 4.0 % 10.1 % 85.9 %
Engine 6 7.8 % 14.4 % 77.8 %
Engine 7 8.2 % 24.1 % 67.8 %
Engine 8 8.3 % 19.1 % 72.6 %
Engine 9 5.3 % 12.6 % 82.0 %
Engine 10 8.8 % 16.9 % 74.4 %
Engine 11 5.6 % 13.9 % 80.5 %
Engine 12 4.0 % 9.2 % 86.8 %
Engine 13 3.7 % 7.1 % 89.2 %
Engine 14 6.0 % 17.7 % 76.3 %
Engine 15 8.3 % 16.9 % 74.8 %
------------------------ -------- -------- --------
Summary Total 106.9 % 247.6 % 1245.5 %
Average 6.7 % 15.5 % 77.8 %
 
CPU Yields by Engine per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Engine 0 1418.7 37.7 852629 32.5 %
Engine 1 1037.6 27.6 623591 23.8 %
Engine 2 59.1 1.6 35495 1.4 %
Engine 3 48.0 1.3 28832 1.1 %
Engine 4 390.7 10.4 234821 8.9 %
Engine 5 50.2 1.3 30183 1.1 %
Engine 6 291.6 7.7 175253 6.7 %
Engine 7 150.5 4.0 90477 3.4 %
Engine 8 84.5 2.2 50796 1.9 %
Engine 9 157.8 4.2 94856 3.6 %
Engine 10 195.6 5.2 117535 4.5 %
Engine 11 224.6 6.0 134982 5.1 %
Engine 12 58.4 1.6 35106 1.3 %
Engine 13 49.5 1.3 29758 1.1 %
Engine 14 43.8 1.2 26339 1.0 %
Engine 15 107.4 2.9 64534 2.5 %
------------------------- ------------ ------------ ----------
Total CPU Yields 4368.0 116.1 2625187
 
Network Checks
Non-Blocking 1126183.4 29923.3 676836204 99.6 %
Blocking 4368.0 116.1 2625171 0.4 %
------------------------- ------------ ------------ ----------
Total Network I/O Checks 1130551.4 30039.4 679461375
Avg Net I/Os per Check n/a n/a 0.00041 n/a
 
Disk I/O Checks
Total Disk I/O Checks 1131908.7 30075.5 680277134 n/a
Checks Returning I/O 921530.8 24485.6 553840000 81.4 %
Avg Disk I/Os Returned n/a n/a 0.00165 n/a
 
 
===============================================================================
 
Worker Process Management
-------------------------
per sec per xact count % of total
------------ ------------ ---------- ----------
Worker Process Requests
Total Requests 0.0 0.0 0 n/a
 
Worker Process Usage
Total Used 0.0 0.0 0 n/a
Max Ever Used During Sample 0.0 0.0 0 n/a
 
Memory Requests for Worker Processes
Total Requests 0.0 0.0 0 n/a
 
Tuning Recommendations for Worker Processes
-------------------------------------------
- Consider decreasing the 'number of worker processes'
configuration parameter.
 
 
===============================================================================
 
Parallel Query Management
-------------------------
 
Parallel Query Usage per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total Parallel Queries 0.0 0.0 0 n/a
 
Merge Lock Requests per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total # of Requests 0.0 0.0 0 n/a
 
Sort Buffer Waits per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total # of Waits 0.0 0.0 0 n/a
 
===============================================================================
 
Task Management per sec per xact count % of total
--------------------------- ------------ ------------ ---------- ----------
 
Connections Opened 0.0 0.0 30 n/a
 
Task Context Switches by Engine
Engine 0 156.5 4.2 94058 9.7 %
Engine 1 136.9 3.6 82285 8.5 %
Engine 2 68.3 1.8 41031 4.3 %
Engine 3 63.0 1.7 37848 3.9 %
Engine 4 140.3 3.7 84334 8.7 %
Engine 5 56.0 1.5 33662 3.5 %
Engine 6 97.4 2.6 58567 6.1 %
Engine 7 139.6 3.7 83899 8.7 %
Engine 8 162.8 4.3 97843 10.1 %
Engine 9 73.3 1.9 44052 4.6 %
Engine 10 123.1 3.3 73958 7.7 %
Engine 11 109.3 2.9 65660 6.8 %
Engine 12 52.2 1.4 31379 3.3 %
Engine 13 50.2 1.3 30142 3.1 %
Engine 14 90.5 2.4 54395 5.6 %
Engine 15 86.7 2.3 52121 5.4 %
------------------------- ------------ ------------ ----------
Total Task Switches: 1606.0 42.7 965234
 
Task Context Switches Due To:
Voluntary Yields 184.0 4.9 110583 11.5 %
Cache Search Misses 97.8 2.6 58786 6.1 %
Exceeding I/O batch size 8.1 0.2 4865 0.5 %
System Disk Writes 249.8 6.6 150149 15.6 %
Logical Lock Contention 0.0 0.0 13 0.0 %
Address Lock Contention 0.0 0.0 0 0.0 %
Latch Contention 1.6 0.0 985 0.1 %
Log Semaphore Contention 4.1 0.1 2456 0.3 %
PLC Lock Contention 0.7 0.0 398 0.0 %
Group Commit Sleeps 11.1 0.3 6675 0.7 %
Last Log Page Writes 140.4 3.7 84395 8.7 %
Modify Conflicts 19.5 0.5 11719 1.2 %
I/O Device Contention 0.0 0.0 0 0.0 %
Network Packet Received 399.4 10.6 240018 24.9 %
Network Packet Sent 43.6 1.2 26204 2.7 %
Network services 442.1 11.7 265719 27.5 %
Other Causes 3.8 0.1 2269 0.2 %
 
 
===============================================================================
 
Application Management
----------------------
 
Application Statistics Summary (All Applications)
-------------------------------------------------
Priority Changes per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
To High Priority 5.8 0.2 3489 26.0 %
To Medium Priority 11.2 0.3 6724 50.1 %
To Low Priority 5.3 0.1 3205 23.9 %
------------------------- ------------ ------------ ----------
Total Priority Changes 22.3 0.6 13418
 
Allotted Slices Exhausted per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
High Priority 0.0 0.0 0 0.0 %
Medium Priority 2.3 0.1 1363 100.0 %
Low Priority 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ----------
Total Slices Exhausted 2.3 0.1 1363
 
Skipped Tasks By Engine per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total Engine Skips 0.0 0.0 0 n/a
 
Engine Scope Changes 0.0 0.0 0 n/a
 
===============================================================================
 
ESP Management per sec per xact count % of total
--------------------------- ------------ ------------ ---------- ----------
ESP Requests 0.0 0.0 0 n/a
===============================================================================
 
Housekeeper Task Activity
-------------------------
per sec per xact count % of total
------------ ------------ ----------
Buffer Cache Washes
Clean 182.6 4.9 109722 91.9 %
Dirty 16.0 0.4 9645 8.1 %
------------ ------------ ----------
Total Washes 198.6 5.3 119367
 
Garbage Collections 3.0 0.1 1808 n/a
Pages Processed in GC 0.0 0.0 1 n/a
 
Statistics Updates 16.7 0.4 10056 n/a
 
Tuning Recommendations for Housekeeper
--------------------------------------
- Consider increasing the 'housekeeper free write percent'
configuration parameter.
 
===============================================================================
 
Monitor Access to Executing SQL
-------------------------------
per sec per xact count % of total
------------ ------------ ---------- ----------
Waits on Execution Plans 0.0 0.0 0 n/a
Number of SQL Text Overflows 0.0 0.0 20 n/a
Maximum SQL Text Requested n/a n/a 90931 n/a
(since beginning of sample)
 
 
Tuning Recommendations for Monitor Access to Executing SQL
----------------------------------------------------------
- Consider increasing the 'max SQL text monitored' parameter
to at least 45977 (i.e., half way from its current value
to Maximum SQL Text Requested).
 
===============================================================================
 
Transaction Profile
-------------------
 
Transaction Summary per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Committed Xacts 37.6 n/a 22619 n/a
 
Transaction Detail per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Inserts
APL Heap Table 129.0 3.4 77523 7.9 %
APL Clustered Table 0.4 0.0 257 0.0 %
Data Only Lock Table 243.1 6.5 146107 14.9 %
Fast Bulk Insert 1255.9 33.4 754811 77.1 %
------------------------- ------------ ------------ ---------- ----------
Total Rows Inserted 1628.4 43.3 978698 99.0 %
 
Updates
APL Deferred 1.1 0.0 655 73.5 %
APL Direct In-place 0.2 0.0 130 14.6 %
APL Direct Cheap 0.1 0.0 53 5.9 %
APL Direct Expensive 0.0 0.0 0 0.0 %
DOL Deferred 0.0 0.0 0 0.0 %
DOL Direct 0.1 0.0 53 5.9 %
------------------------- ------------ ------------ ---------- ----------
Total Rows Updated 1.5 0.0 891 0.1 %
 
Data Only Locked Updates
DOL Replace 0.1 0.0 53 100.0 %
DOL Shrink 0.0 0.0 0 0.0 %
DOL Cheap Expand 0.0 0.0 0 0.0 %
DOL Expensive Expand 0.0 0.0 0 0.0 %
DOL Expand & Forward 0.0 0.0 0 0.0 %
DOL Fwd Row Returned 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total DOL Rows Updated 0.1 0.0 53 0.0 %
 
Deletes
APL Deferred 1.1 0.0 655 7.3 %
APL Direct 0.5 0.0 274 3.1 %
DOL 13.3 0.4 8006 89.6 %
------------------------- ------------ ------------ ---------- ----------
Total Rows Deleted 14.9 0.4 8935 0.9 %
========================= ============ ============ ==========
Total Rows Affected 1644.8 43.7 988524
 
===============================================================================
 
Transaction Management
----------------------
 
ULC Flushes to Xact Log per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
by Full ULC 7.0 0.2 4231 2.7 %
by End Transaction 49.9 1.3 29993 18.8 %
by Change of Database 0.2 0.0 92 0.1 %
by Single Log Record 21.3 0.6 12785 8.0 %
by Unpin 187.1 5.0 112457 70.5 %
by Other 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ----------
Total ULC Flushes 265.5 7.1 159558
 
ULC Flushes Skipped per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
by PLC Discards 3.5 0.1 2104 100.0 %
------------------------- ------------ ------------ ----------
Total ULC Discards 3.5 0.1 2104
 
ULC Log Records 3463.5 92.0 2081582 n/a
Max ULC Size During Sample n/a n/a 0 n/a
 
ULC Semaphore Requests
Granted 7496.3 199.2 4505302 100.0 %
Waited 0.7 0.0 398 0.0 %
------------------------- ------------ ------------ ----------
Total ULC Semaphore Req 7497.0 199.2 4505700
 
Log Semaphore Requests
Granted 272.5 7.2 163757 98.5 %
Waited 4.1 0.1 2456 1.5 %
------------------------- ------------ ------------ ----------
Total Log Semaphore Req 276.6 7.3 166213
 
Transaction Log Writes 166.7 4.4 100205 n/a
Transaction Log Alloc 53.1 1.4 31926 n/a
Avg # Writes per Log Page n/a n/a 3.13866 n/a
 
===============================================================================
 
Index Management
----------------
 
Nonclustered Maintenance per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Ins/Upd Requiring Maint 2.7 0.1 1615 n/a
# of NC Ndx Maint 2.7 0.1 1614 n/a
Avg NC Ndx Maint / Op n/a n/a 0.99938 n/a
 
Deletes Requiring Maint 0.2 0.0 149 n/a
# of NC Ndx Maint 0.2 0.0 148 n/a
Avg NC Ndx Maint / Op n/a n/a 0.99329 n/a
 
RID Upd from Clust Split 0.0 0.0 1 n/a
# of NC Ndx Maint 0.0 0.0 7 n/a
Avg NC Ndx Maint / Op n/a n/a 7.00000 n/a
 
Upd/Del DOL Req Maint 13.4 0.4 8059 n/a
# of DOL Ndx Maint 15.9 0.4 9563 n/a
Avg DOL Ndx Maint / Op n/a n/a 1.18662 n/a
 
Page Splits 9.5 0.3 5708 n/a
Retries 0.0 0.0 0 0.0 %
Deadlocks 0.0 0.0 0 0.0 %
Add Index Level 0.0 0.0 0 0.0 %
 
Page Shrinks 0.0 0.0 0 n/a
 
Index Scans per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Ascending Scans 199.8 5.3 120091 36.3 %
DOL Ascending Scans 309.1 8.2 185749 56.1 %
Descending Scans 42.0 1.1 25214 7.6 %
DOL Descending Scans 0.0 0.0 0 0.0 %
------------ ------------ ----------
Total Scans 550.8 14.6 331054
 
===============================================================================
 
Metadata Cache Management
-------------------------
 
Metadata Cache Summary per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
 
Open Object Usage
Active n/a n/a 1017 n/a
Max Ever Used Since Boot n/a n/a 1034 n/a
Free n/a n/a 18983 n/a
Reuse Requests
Succeeded n/a n/a 0 n/a
Failed n/a n/a 0 n/a
 
Open Index Usage
Active n/a n/a 363 n/a
Max Ever Used Since Boot n/a n/a 377 n/a
Free n/a n/a 9637 n/a
Reuse Requests
Succeeded n/a n/a 0 n/a
Failed n/a n/a 0 n/a
 
Open Partition Usage
Active n/a n/a 591 n/a
Max Ever Used Since Boot n/a n/a 605 n/a
Free n/a n/a 9409 n/a
Reuse Requests
Succeeded n/a n/a 0 n/a
Failed n/a n/a 0 n/a
 
Open Database Usage
Active n/a n/a 123 n/a
Max Ever Used Since Boot n/a n/a 123 n/a
Free n/a n/a 27 n/a
Reuse Requests
Succeeded n/a n/a 0 n/a
Failed n/a n/a 0 n/a
 
Object Manager Spinlock Contention n/a n/a n/a 0.0 %
 
Object Spinlock Contention n/a n/a n/a 0.0 %
 
Index Spinlock Contention n/a n/a n/a 0.0 %
 
Index Hash Spinlock Contention n/a n/a n/a 0.0 %
 
Partition Spinlock Contention n/a n/a n/a 0.0 %
 
Partition Hash Spinlock Contention n/a n/a n/a 0.0 %
 
===============================================================================
 
Lock Management
---------------
 
Lock Summary per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total Lock Requests 20861.4 554.3 12537709 n/a
Avg Lock Contention 0.0 0.0 13 0.0 %
Deadlock Percentage 0.0 0.0 0 0.0 %
 
Lock Detail per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
 
Table Lock Hashtable
Lookups 407.5 10.8 244892 n/a
Avg Chain Length n/a n/a 0.37744 n/a
Spinlock Contention n/a n/a n/a 0.0 %
 
Exclusive Table
Granted 7.7 0.2 4616 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total EX-Table Requests 7.7 0.2 4616 0.0 %
 
Shared Table
Granted 0.0 0.0 10 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total SH-Table Requests 0.0 0.0 10 0.0 %
 
Exclusive Intent
Granted 73.7 2.0 44305 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total EX-Intent Requests 73.7 2.0 44305 0.4 %
 
Shared Intent
Granted 285.1 7.6 171346 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total SH-Intent Requests 285.1 7.6 171346 1.4 %
 
Page & Row Lock HashTable
Lookups 20471.2 543.9 12303205 n/a
Avg Chain Length n/a n/a 0.02292 n/a
Spinlock Contention n/a n/a n/a 0.0 %
 
Exclusive Page
Granted 184.6 4.9 110974 100.0 %
Waited 0.0 0.0 13 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total EX-Page Requests 184.7 4.9 110987 0.9 %
 
Update Page
Granted 1.2 0.0 718 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total UP-Page Requests 1.2 0.0 718 0.0 %
 
Shared Page
Granted 17193.9 456.9 10333533 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total SH-Page Requests 17193.9 456.9 10333533 82.4 %
 
 
Exclusive Row
Granted 459.4 12.2 276086 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total EX-Row Requests 459.4 12.2 276086 2.2 %
 
Update Row
Granted 29.3 0.8 17580 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total UP-Row Requests 29.3 0.8 17580 0.1 %
 
Shared Row
Granted 2572.5 68.4 1546099 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total SH-Row Requests 2572.5 68.4 1546099 12.3 %
 
 
Next-Key
Total Next-Key Requests 0.0 0.0 0 n/a
 
Address Lock Hashtable
Lookups 54.0 1.4 32429 n/a
Avg Chain Length n/a n/a 0.00274 n/a
Spinlock Contention n/a n/a n/a 0.0 %
 
Exclusive Address
Granted 32.6 0.9 19613 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total EX-Address Requests 32.6 0.9 19613 0.2 %
 
Shared Address
Granted 21.3 0.6 12816 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total SH-Address Requests 21.3 0.6 12816 0.1 %
 
 
Last Page Locks on Heaps
Granted 129.0 3.4 77527 100.0 %
Waited 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ---------- ----------
Total Last Pg Locks 129.0 3.4 77527 100.0 %
 
 
Deadlocks by Lock Type per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total Deadlocks 0.0 0.0 0 n/a
 
 
Deadlock Detection
Deadlock Searches 0.0 0.0 30 n/a
Searches Skipped 0.0 0.0 0 0.0 %
Avg Deadlocks per Search n/a n/a 0.00000 n/a
 
 
Lock Promotions
Ex-Page to Ex-Table 0.0 0.0 0 0.0 %
Sh-Page to Sh-Table 0.0 0.0 0 0.0 %
Ex-Row to Ex-Table 0.5 0.0 309 100.0 %
Sh-Row to Sh-Table 0.0 0.0 0 0.0 %
Sh-Next-Key to Sh-Table 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ----------
Total Lock Promotions 0.5 0.0 309
 
 
Lock Timeouts by Lock Type per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total Timeouts 0.0 0.0 0 n/a
 
Tuning Recommendations for Lock Management
------------------------------------------
- Consider increasing the 'deadlock checking period' parameter
by 50 ms.
 
 
===============================================================================
 
Data Cache Management
---------------------
 
Cache Statistics Summary (All Caches)
-------------------------------------
per sec per xact count % of total
------------ ------------ ---------- ----------
 
Cache Search Summary
Total Cache Hits 29609.5 786.7 17795290 95.5 %
Total Cache Misses 1391.4 37.0 836259 4.5 %
------------------------- ------------ ------------ ----------
Total Cache Searches 31000.9 823.7 18631549
 
Cache Turnover
Buffers Grabbed 875.6 23.3 526255 n/a
Buffers Grabbed Dirty 0.1 0.0 43 0.0 %
 
Cache Strategy Summary
Cached (LRU) Buffers 29459.3 782.7 17705013 99.2 %
Discarded (MRU) Buffers 246.4 6.5 148109 0.8 %
 
Large I/O Usage
Large I/Os Performed 214.1 5.7 128649 99.9 %
 
Large I/Os Denied due to
Pool < Prefetch Size 0.3 0.0 185 0.1 %
Pages Requested
Reside in Another
Buffer Pool 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ----------
Total Large I/O Requests 214.4 5.7 128834
 
Large I/O Effectiveness
Pages by Lrg I/O Cached 1553.0 41.3 933372 n/a
Pages by Lrg I/O Used 987.8 26.2 593640 63.6 %
 
Asynchronous Prefetch Activity
APFs Issued 0.4 0.0 267 0.8 %
APFs Denied Due To
APF I/O Overloads 0.0 0.0 0 0.0 %
APF Limit Overloads 0.0 0.0 0 0.0 %
APF Reused Overloads 0.0 0.0 10 0.0 %
APF Buffers Found in Cache
With Spinlock Held 0.0 0.0 3 0.0 %
W/o Spinlock Held 54.3 1.4 32624 99.1 %
------------------------- ------------ ------------ ----------
Total APFs Requested 54.7 1.5 32904
 
Other Asynchronous Prefetch Statistics
APFs Used 0.4 0.0 267 n/a
APF Waits for I/O 0.3 0.0 194 n/a
APF Discards 0.0 0.0 0 n/a
 
Dirty Read Behavior
Page Requests 0.0 0.0 0 n/a
 
-------------------------------------------------------------------------------
Cache: default data cache
per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Spinlock Contention n/a n/a n/a 0.7 %
 
Utilization n/a n/a n/a 98.3 %
 
Cache Searches
Cache Hits 29077.1 772.6 17475330 95.4 %
Found in Wash 132.6 3.5 79691 0.5 %
Cache Misses 1391.4 37.0 836202 4.6 %
------------------------- ------------ ------------ ----------
Total Cache Searches 30468.4 809.6 18311532
 
Pool Turnover
8 Kb Pool
LRU Buffer Grab 661.3 17.6 397443 75.6 %
Grabbed Dirty 0.0 0.0 0 0.0 %
16 Kb Pool
LRU Buffer Grab 26.0 0.7 15599 3.0 %
Grabbed Dirty 0.0 0.0 0 0.0 %
64 Kb Pool
LRU Buffer Grab 187.5 5.0 112679 21.4 %
Grabbed Dirty 0.1 0.0 43 0.0 %
------------------------- ------------ ------------ ----------
Total Cache Turnover 874.8 23.2 525764
 
Buffer Wash Behavior
Buffers Passed Clean 0.0 0.0 0 0.0 %
Buffers Already in I/O 0.0 0.0 0 0.0 %
Buffers Washed Dirty 20.9 0.6 12573 100.0 %
 
Cache Strategy
Cached (LRU) Buffers 28944.6 769.1 17395730 99.2 %
Discarded (MRU) Buffers 246.4 6.5 148109 0.8 %
 
Large I/O Usage
Large I/Os Performed 213.4 5.7 128278 99.9 %
 
Large I/Os Denied due to
Pool < Prefetch Size 0.0 0.0 1 0.0 %
Pages Requested
Reside in Another
Buffer Pool 0.2 0.0 125 0.1 %
------------------------- ------------ ------------ ----------
Total Large I/O Requests 213.7 5.7 128404
 
Large I/O Detail
16 Kb Pool
Pages Cached 51.9 1.4 31198 n/a
Pages Used 51.9 1.4 31200 100.0 %
64 Kb Pool
Pages Cached 1499.9 39.9 901432 n/a
Pages Used 934.6 24.8 561699 62.3 %
 
Dirty Read Behavior
Page Requests 0.0 0.0 0 n/a
 
Tuning Recommendations for Data cache : default data cache
-------------------------------------
- Consider increasing the 'wash size' of the 64k pool for this cache.
 
-------------------------------------------------------------------------------
Cache: ent_cache
per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Spinlock Contention n/a n/a n/a 0.0 %
 
Utilization n/a n/a n/a 0.0 %
 
Cache Searches
Cache Hits 1.6 0.0 965 94.4 %
Found in Wash 1.2 0.0 749 77.6 %
Cache Misses 0.1 0.0 57 5.6 %
------------------------- ------------ ------------ ----------
Total Cache Searches 1.7 0.0 1022
 
Pool Turnover
8 Kb Pool
LRU Buffer Grab 0.1 0.0 57 100.0 %
Grabbed Dirty 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ----------
Total Cache Turnover 0.1 0.0 57
 
Buffer Wash Behavior
Statistics Not Available - No Buffers Entered Wash Section Yet
 
Cache Strategy
Cached (LRU) Buffers 1.7 0.0 1023 100.0 %
Discarded (MRU) Buffers 0.0 0.0 0 0.0 %
 
Large I/O Usage
Total Large I/O Requests 0.0 0.0 0 n/a
 
Large I/O Detail
No Large Pool(s) In This Cache
 
Dirty Read Behavior
Page Requests 0.0 0.0 0 n/a
 
Tuning Recommendations for Data cache : ent_cache
-------------------------------------
- Consider adding a large I/O pool for this cache.
 
-------------------------------------------------------------------------------
Cache: tempdb_cache
per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Spinlock Contention n/a n/a n/a 0.2 %
 
Utilization n/a n/a n/a 1.7 %
 
Cache Searches
Cache Hits 527.1 14.0 316792 100.0 %
Found in Wash 6.5 0.2 3888 1.2 %
Cache Misses 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ----------
Total Cache Searches 527.1 14.0 316792
 
Pool Turnover
8 Kb Pool
LRU Buffer Grab 0.1 0.0 63 14.5 %
Grabbed Dirty 0.0 0.0 0 0.0 %
16 Kb Pool
LRU Buffer Grab 0.6 0.0 371 85.5 %
Grabbed Dirty 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ----------
Total Cache Turnover 0.7 0.0 434
 
Buffer Wash Behavior
Statistics Not Available - No Buffers Entered Wash Section Yet
 
Cache Strategy
Cached (LRU) Buffers 509.5 13.5 306193 100.0 %
Discarded (MRU) Buffers 0.0 0.0 0 0.0 %
 
Large I/O Usage
Large I/Os Performed 0.6 0.0 371 86.3 %
 
Large I/Os Denied due to
Pool < Prefetch Size 0.0 0.0 0 0.0 %
Pages Requested
Reside in Another
Buffer Pool 0.1 0.0 59 13.7 %
------------------------- ------------ ------------ ----------
Total Large I/O Requests 0.7 0.0 430
 
Large I/O Detail
16 Kb Pool
Pages Cached 1.2 0.0 742 n/a
Pages Used 1.2 0.0 741 99.9 %
 
Dirty Read Behavior
Page Requests 0.0 0.0 0 n/a
 
Tuning Recommendations for Data cache : tempdb_cache
-------------------------------------
- Consider using 'relaxed LRU replacement policy'
for this cache.
 
-------------------------------------------------------------------------------
Cache: vap_cache
per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Spinlock Contention n/a n/a n/a 0.0 %
 
Utilization n/a n/a n/a 0.0 %
 
Cache Searches
Cache Hits 3.7 0.1 2203 100.0 %
Found in Wash 0.4 0.0 241 10.9 %
Cache Misses 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ----------
Total Cache Searches 3.7 0.1 2203
 
Pool Turnover 0.0 0.0 0 n/a
 
Buffer Wash Behavior
Statistics Not Available - No Buffers Entered Wash Section Yet
 
Cache Strategy
Cached (LRU) Buffers 3.4 0.1 2067 100.0 %
Discarded (MRU) Buffers 0.0 0.0 0 0.0 %
 
Large I/O Usage
Total Large I/O Requests 0.0 0.0 0 n/a
 
Large I/O Detail
16 Kb Pool
Pages Cached 0.0 0.0 0 n/a
Pages Used 0.0 0.0 0 n/a
64 Kb Pool
Pages Cached 0.0 0.0 0 n/a
Pages Used 0.0 0.0 0 n/a
 
Dirty Read Behavior
Page Requests 0.0 0.0 0 n/a
 
Tuning Recommendations for Data cache : vap_cache
-------------------------------------
- Consider using 'relaxed LRU replacement policy'
for this cache.
 
- Consider removing the 16k pool for this cache.
 
- Consider removing the 64k pool for this cache.
 
===============================================================================
 
Procedure Cache Management per sec per xact count % of total
--------------------------- ------------ ------------ ---------- ----------
Procedure Requests 2.1 0.1 1287 n/a
Procedure Reads from Disk 0.0 0.0 5 0.4 %
Procedure Writes to Disk 0.0 0.0 0 0.0 %
Procedure Removals 0.0 0.0 8 n/a
Procedure Recompilations 0.0 0.0 2 n/a
 
Recompilations Requests:
Execution Phase 0.0 0.0 2 100.0 %
Compilation Phase 0.0 0.0 0 0.0 %
Execute Cursor Execution 0.0 0.0 0 0.0 %
Redefinition Phase 0.0 0.0 0 0.0 %
 
Recompilation Reasons:
Table Missing 0.0 0.0 0 n/a
Temporary Table Missing 0.0 0.0 0 n/a
Schema Change 0.0 0.0 2 n/a
Index Change 0.0 0.0 0 n/a
Isolation Level Change 0.0 0.0 0 n/a
Permissions Change 0.0 0.0 0 n/a
Cursor Permissions Change 0.0 0.0 0 n/a
 
SQL Statement Cache:
Statements Cached 0.0 0.0 0 n/a
Statements Found in Cache 0.0 0.0 0 n/a
Statements Not Found 0.0 0.0 0 n/a
Statements Dropped 0.0 0.0 0 n/a
Statements Restored 0.0 0.0 0 n/a
Statements Not Cached 2.8 0.1 1666 n/a
 
 
===============================================================================
 
Memory Management per sec per xact count % of total
--------------------------- ------------ ------------ ---------- ----------
Pages Allocated 79.8 2.1 47948 n/a
Pages Released 79.8 2.1 47942 n/a
 
===============================================================================
 
Recovery Management
-------------------
 
Checkpoints per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
# of Normal Checkpoints 0.2 0.0 95 100.0 %
# of Free Checkpoints 0.0 0.0 0 0.0 %
------------------------- ------------ ------------ ----------
Total Checkpoints 0.2 0.0 95
 
Avg Time per Normal Chkpt 2.45263 seconds
 
===============================================================================
 
Disk I/O Management
-------------------
 
Max Outstanding I/Os per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Server n/a n/a 3692 n/a
Engine 0 n/a n/a 290 n/a
Engine 1 n/a n/a 197 n/a
Engine 2 n/a n/a 139 n/a
Engine 3 n/a n/a 1348 n/a
Engine 4 n/a n/a 1925 n/a
Engine 5 n/a n/a 473 n/a
Engine 6 n/a n/a 475 n/a
Engine 7 n/a n/a 241 n/a
Engine 8 n/a n/a 240 n/a
Engine 9 n/a n/a 168 n/a
Engine 10 n/a n/a 229 n/a
Engine 11 n/a n/a 1010 n/a
Engine 12 n/a n/a 137 n/a
Engine 13 n/a n/a 317 n/a
Engine 14 n/a n/a 232 n/a
Engine 15 n/a n/a 2383 n/a
 
 
I/Os Delayed by
Disk I/O Structures n/a n/a 0 n/a
Server Config Limit n/a n/a 0 n/a
Engine Config Limit n/a n/a 0 n/a
Operating System Limit n/a n/a 0 n/a
 
 
Total Requested Disk I/Os 1524.5 40.5 916229
 
Completed Disk I/O's
Asynchronous I/O's
Engine 0 126.0 3.3 75755 8.3 %
Engine 1 85.1 2.3 51127 5.6 %
Engine 2 60.9 1.6 36615 4.0 %
Engine 3 82.8 2.2 49776 5.4 %
Engine 4 128.6 3.4 77307 8.4 %
Engine 5 45.3 1.2 27245 3.0 %
Engine 6 84.1 2.2 50555 5.5 %
Engine 7 119.1 3.2 71564 7.8 %
Engine 8 131.6 3.5 79105 8.6 %
Engine 9 107.5 2.9 64637 7.1 %
Engine 10 96.6 2.6 58029 6.3 %
Engine 11 76.7 2.0 46104 5.0 %
Engine 12 160.0 4.3 96178 10.5 %
Engine 13 48.4 1.3 29113 3.2 %
Engine 14 107.6 2.9 64693 7.1 %
Engine 15 63.9 1.7 38406 4.2 %
Synchronous I/O's
Total Completed I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ----------
Total Completed I/Os 1524.5 40.5 916209
 
 
Device Activity Detail
----------------------
 
Device:
/SYBDEV/LOG1/logdev1
logdev1 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev10
logdev10 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.1 0.0 35 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.1 0.0 35 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev11
logdev11 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.1 0.0 32 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.1 0.0 32 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev12
logdev12 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev13
logdev13 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.3 0.0 207 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.3 0.0 207 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev14
logdev14 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 17 0.2 %
Writes 12.7 0.3 7610 99.8 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 12.7 0.3 7627 0.8 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev15
logdev15 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev16
logdev16 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev17
logdev17 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev18
logdev18 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev19
logdev19 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev2
logdev2 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 27 0.1 %
Writes 46.7 1.2 28059 99.9 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 46.7 1.2 28086 3.1 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev20
logdev20 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev3
logdev3 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev4
logdev4 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.1 0.0 53 0.1 %
Writes 65.2 1.7 39211 99.9 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 65.3 1.7 39264 4.3 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev5
logdev5 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 14 0.1 %
Writes 35.8 1.0 21509 99.9 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 35.8 1.0 21523 2.3 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev6
logdev6 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 5 0.1 %
Writes 5.6 0.1 3384 99.9 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 5.6 0.1 3389 0.4 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev7
logdev7 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev8
logdev8 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 2 0.4 %
Writes 0.8 0.0 474 99.6 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.8 0.0 476 0.1 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/LOG1/logdev9
logdev9 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV01/datadev1
datadev1 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 20 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 20 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV01/datadev10
datadev10 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 10 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 10 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV01/datadev2
datadev2 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV01/datadev3
datadev3 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 6 0.0 %
Writes 112.5 3.0 67639 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 112.6 3.0 67645 7.4 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV01/datadev4
datadev4 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 20 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 20 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV01/datadev5
datadev5 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.1 0.0 72 0.1 %
Writes 86.1 2.3 51756 99.9 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 86.2 2.3 51828 5.7 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV01/datadev6
datadev6 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 20 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 20 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV01/datadev7
datadev7 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV01/datadev8
datadev8 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV01/datadev9
datadev9 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 10 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 10 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV02/datadev11
datadev11 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV02/datadev12
datadev12 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV02/datadev13
datadev13 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV02/datadev14
datadev14 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 20 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 20 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV02/datadev15
datadev15 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV02/datadev16
datadev16 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV02/datadev17
datadev17 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 27 0.0 %
Writes 216.8 5.8 130324 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 216.9 5.8 130351 14.2 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV02/datadev18
datadev18 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.3 0.0 154 0.3 %
Non-APF 0.3 0.0 169 0.3 %
Writes 80.7 2.1 48480 99.3 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 81.2 2.2 48803 5.3 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV02/datadev19
datadev19 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 9 0.0 %
Non-APF 0.1 0.0 38 0.2 %
Writes 34.3 0.9 20623 99.8 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 34.4 0.9 20670 2.3 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV02/datadev20
datadev20 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev21
datadev21 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 10 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 10 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev22
datadev22 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 10 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 10 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev23
datadev23 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.1 0.0 40 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.1 0.0 40 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev24
datadev24 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.1 0.0 40 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.1 0.0 40 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev25
datadev25 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 19 0.0 %
Non-APF 0.1 0.0 66 0.1 %
Writes 92.1 2.4 55356 99.8 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 92.2 2.5 55441 6.1 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev26
datadev26 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.1 0.0 49 0.7 %
Writes 12.3 0.3 7391 99.3 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 12.4 0.3 7440 0.8 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev27
datadev27 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.2 0.0 124 19.7 %
Writes 0.8 0.0 507 80.3 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 1.0 0.0 631 0.1 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev28
datadev28 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 3 0.1 %
Writes 6.4 0.2 3822 99.9 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 6.4 0.2 3825 0.4 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev29
datadev29 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.1 0.0 50 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.1 0.0 50 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev30
datadev30 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.1 0.0 50 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.1 0.0 50 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev31
datadev31 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 20 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 20 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev32
datadev32 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 85.2 2.3 51180 12.7 %
Writes 585.3 15.6 351744 87.3 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 670.4 17.8 402924 44.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev33
datadev33 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 2 1.9 %
Writes 0.2 0.0 102 98.1 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.2 0.0 104 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev34
datadev34 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 1 0.8 %
Writes 0.2 0.0 129 99.2 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.2 0.0 130 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev35
datadev35 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 10 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 10 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev36
datadev36 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 10 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 10 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev37
datadev37 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 10 0.1 %
Writes 12.1 0.3 7287 99.9 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 12.1 0.3 7297 0.8 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev38
datadev38 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 10 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 10 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev39
datadev39 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 1 0.0 %
Non-APF 11.5 0.3 6883 44.5 %
Writes 14.3 0.4 8573 55.5 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 25.7 0.7 15457 1.7 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV03/datadev40
datadev40 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.1 0.0 70 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.1 0.0 70 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV04/datadev41
datadev41 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.1 0.0 70 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.1 0.0 70 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV04/datadev42
datadev42 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.0 0.0 10 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 10 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV04/datadev43
datadev43 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.1 0.0 70 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.1 0.0 70 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV04/datadev44
datadev44 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV04/datadev45
datadev45 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV04/datadev46
datadev46 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV04/datadev47
datadev47 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV04/datadev48
datadev48 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV04/datadev49
datadev49 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.1 0.0 40 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.1 0.0 40 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV04/datadev50
datadev50 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV05/datadev51
datadev51 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV05/datadev52
datadev52 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV05/datadev53
datadev53 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV05/datadev54
datadev54 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV05/datadev55
datadev55 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV05/datadev56
datadev56 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV05/datadev57
datadev57 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV05/datadev58
datadev58 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV05/datadev59
datadev59 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV05/datadev60
datadev60 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV06/datadev61
datadev61 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV06/datadev62
datadev62 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV06/datadev63
datadev63 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV06/datadev64
datadev64 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV06/datadev65
datadev65 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV06/datadev66
datadev66 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV06/datadev67
datadev67 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV06/datadev68
datadev68 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV06/datadev69
datadev69 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYBDEV06/datadev70
datadev70 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYS/masterdev
master per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.1 0.0 52 3.0 %
Non-APF 0.0 0.0 16 0.9 %
Writes 2.8 0.1 1676 96.1 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 2.9 0.1 1744 0.2 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYS/sysdbdev
systemdbdev per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/SYS/sysprocsdev
sysprocsdev per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.1 0.0 32 50.8 %
Non-APF 0.0 0.0 22 34.9 %
Writes 0.0 0.0 9 14.3 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.1 0.0 63 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/TEMP1/tempdev
tempdbdev per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.2 0.0 131 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.2 0.0 131 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/TEMP1/tempdev1
tempdev1 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Reads
APF 0.0 0.0 0 0.0 %
Non-APF 0.0 0.0 0 0.0 %
Writes 0.8 0.0 492 100.0 %
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.8 0.0 492 0.1 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/TEMP1/tempdev2
tempdev2 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/TEMPLOG1/templog1
templog1 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/TEMPLOG1/templog2
templog2 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
Device:
/SYBDEV/TEMPLOG1/templog3
templog3 per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 n/a
------------------------- ------------ ------------ ---------- ----------
Total I/Os 0.0 0.0 0 0.0 %
 
 
-----------------------------------------------------------------------------
 
 
 
===============================================================================
 
Network I/O Management
----------------------
 
Total Network I/O Requests 443.0 11.8 266223 n/a
Network I/Os Delayed 0.0 0.0 0 0.0 %
 
 
Total TDS Packets Received per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Engine 0 50.4 1.3 30313 12.6 %
Engine 1 38.2 1.0 22965 9.6 %
Engine 2 13.1 0.3 7880 3.3 %
Engine 3 3.8 0.1 2272 0.9 %
Engine 4 39.2 1.0 23570 9.8 %
Engine 5 13.8 0.4 8312 3.5 %
Engine 6 24.5 0.7 14705 6.1 %
Engine 7 24.8 0.7 14904 6.2 %
Engine 8 55.3 1.5 33226 13.8 %
Engine 9 9.9 0.3 5975 2.5 %
Engine 10 32.8 0.9 19732 8.2 %
Engine 11 39.8 1.1 23933 10.0 %
Engine 12 23.0 0.6 13794 5.7 %
Engine 13 5.6 0.1 3379 1.4 %
Engine 14 9.4 0.3 5666 2.4 %
Engine 15 15.7 0.4 9408 3.9 %
------------------------- ------------ ------------ ----------
Total TDS Packets Rec'd 399.4 10.6 240034
 
 
Total Bytes Received per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Engine 0 21164.6 562.4 12719945 12.7 %
Engine 1 15464.1 410.9 9293931 9.3 %
Engine 2 6600.3 175.4 3966803 4.0 %
Engine 3 1854.0 49.3 1114238 1.1 %
Engine 4 16477.6 437.8 9903015 9.9 %
Engine 5 6979.4 185.4 4194611 4.2 %
Engine 6 10613.6 282.0 6378757 6.4 %
Engine 7 10822.3 287.6 6504224 6.5 %
Engine 8 20153.2 535.5 12112049 12.1 %
Engine 9 2982.0 79.2 1792194 1.8 %
Engine 10 12676.7 336.8 7618720 7.6 %
Engine 11 18406.7 489.1 11062414 11.0 %
Engine 12 11616.2 308.6 6981321 7.0 %
Engine 13 2817.6 74.9 1693391 1.7 %
Engine 14 2447.9 65.0 1471193 1.5 %
Engine 15 6018.5 159.9 3617119 3.6 %
------------------------- ------------ ------------ ----------
Total Bytes Rec'd 167094.7 4439.8 100423925
 
 
Avg Bytes Rec'd per Packet n/a n/a 418 n/a
 
-----------------------------------------------------------------------------
 
Total TDS Packets Sent per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Engine 0 8.1 0.2 4870 12.8 %
Engine 1 6.8 0.2 4100 10.8 %
Engine 2 0.2 0.0 118 0.3 %
Engine 3 0.4 0.0 218 0.6 %
Engine 4 5.9 0.2 3536 9.3 %
Engine 5 0.2 0.0 134 0.4 %
Engine 6 3.2 0.1 1918 5.0 %
Engine 7 3.1 0.1 1844 4.8 %
Engine 8 14.3 0.4 8579 22.5 %
Engine 9 3.7 0.1 2223 5.8 %
Engine 10 6.7 0.2 4012 10.5 %
Engine 11 3.4 0.1 2038 5.4 %
Engine 12 0.3 0.0 158 0.4 %
Engine 13 0.1 0.0 70 0.2 %
Engine 14 3.7 0.1 2246 5.9 %
Engine 15 3.3 0.1 1982 5.2 %
------------------------- ------------ ------------ ----------
Total TDS Packets Sent 63.3 1.7 38046
 
 
Total Bytes Sent per sec per xact count % of total
------------------------- ------------ ------------ ---------- ----------
Engine 0 147.1 3.9 88434 8.6 %
Engine 1 144.9 3.8 87080 8.5 %
Engine 2 7.0 0.2 4218 0.4 %
Engine 3 120.4 3.2 72374 7.0 %
Engine 4 110.6 2.9 66476 6.5 %
Engine 5 22.9 0.6 13787 1.3 %
Engine 6 96.8 2.6 58158 5.7 %
Engine 7 57.4 1.5 34508 3.4 %
Engine 8 251.3 6.7 151034 14.7 %
Engine 9 239.1 6.4 143710 14.0 %
Engine 10 157.0 4.2 94351 9.2 %
Engine 11 136.7 3.6 82135 8.0 %
Engine 12 16.7 0.4 10027 1.0 %
Engine 13 8.7 0.2 5254 0.5 %
Engine 14 105.5 2.8 63418 6.2 %
Engine 15 86.5 2.3 51978 5.1 %
------------------------- ------------ ------------ ----------
Total Bytes Sent 1708.7 45.4 1026942
 
 
Avg Bytes Sent per Packet n/a n/a 26 n/a
 
=============================== End of Report =================================

Select all Open in new window

by: Joe_WoodhousePosted on 2009-07-16 at 07:42:18ID: 24869597

Top isn't telling us much here. We see all the activity is coming from a single database process at the O/S level. Sure, a lot of that is kernel rather than user, which tends to imply I/O management (disk or network) or some other low-level infrastructure.

Looking inside sysmon we see two interesting things right away:

- the work is being spread over multiple engines as I suggested, and
- Sybase's measure of how busy it is is only 6.7% CPU busy on average!

This says to me you do not have a Sybase issue here... when Sybase is 6.7% busy, the host is 88% busy.

Have a look into patches for KAIO, firmware updates for your HBAs, patches for SAN and volume management, write caches in the controllers (many ships with write caches disabled by default). Basically get your sys admins in on this, because this is an operating system and/or storage issue.

by: PinkSandPosted on 2009-07-16 at 08:07:05ID: 24869889

Thanks Joe,
We will talk to the sysadmin people. By the way, the box is using zfs not nfs. Do you think that may be an issue?

by: Joe_WoodhousePosted on 2009-07-16 at 08:11:40ID: 24869939

I hope you wouldn't be using nfs!! It's terrible for Sybase. You'll get a lot of argument about whether ZFS is better than raw partition or vice versa... but it shouldn't make a lot of difference most of the time. ZFS is pretty decent.

by: PinkSandPosted on 2009-07-16 at 08:18:24ID: 24870010

We are using ZFS.
Thanks!

by: PinkSandPosted on 2009-07-16 at 16:44:16ID: 24874896

Hi Joe,

Thanks for pointing to the right direction. The sysadmin saw lot of time spending on write and checking the disk.

by: alpmoonPosted on 2009-07-16 at 17:11:26ID: 24875010

From the whole output of top command, I think the total cpu utilization is not reported correctly. It looks one dataserver process uses 78.73 per cent of cpu/10 (not 78.73% of all cpu's) and another dataserver process use 3.88% of another cpu. And there is no other processes using cpu heavily. I don't see how total cpu utilization can be higer than 80%. (Btw. I don't mean that Sybase engines uses the same cpu all the time)

At the beginning I thought that the percentages were for all cpu's. But based on the second output, apparently it is for each cpu. And using 78.73% of one cpu to build a clustered index is very reasonable. The wrong output is "CPU states: 12.0% idle, 4.5% user, 83.5% kernel". I am not sure what it means, but it cannot be about usage of all 32 cpu's. You shouldn't worry about any performance issue here.

by: IncisiveOnePosted on 2009-07-17 at 16:14:08ID: 24883723

> Our client applications are also running on the same box.

Not good.

The plan is to use 1 cpu per sybase engine, and 1 per client application.

I think you mean 16 engines for ASE and 16 logical processors remaining for your app.

We checked that the CPU idle is mostly over 70% until we create a clustered index. We repeated the test, and every time the CPU idlle went down to almost 0 when the index is created on a table with more than 1 million rows. We defintely would not create index on a big table during normal operation hours.

Good.

My main concern is why one Sybase engine can have over 40% CPU usage on a 32 CPU machine if max parallel degree is set to 1.

To Sybase, it is a 16 CPU box (16 engines). Given the settings, since only 1 or 2 engines can be active for your single command (index rebuild), I would not be concerned if they hit 100% CPU utilisation (which may be 40% at the unix level for those two; and which is still very idle across the system, either 16 engines or 32 logical CPUs).

Sure, that network and the disk I/O subsystem is THEN under pressure, and sure, it looks slow. For this index rebuild, you are I/O bound, the 2 engines are spinning their wheels waiting for I/O. But that is a separate issue, there is no "problem" with 100% CPU utilisation for such a command; when you fix the I/O problem, you will find you still incur high CPU utilisation for the command, but far less waiting/spinning/yields, and of course, the command executes faster (old elapsed time vs new).

----------

Separately, if the table is partitioned, use at least 16 worker processes.

Cheers

by: Joe_WoodhousePosted on 2009-08-08 at 07:39:36ID: 25050027

The original questions were:

1) Why is Sybase using so much kernel time?
2) Is there a way to limit the CPU used by a Sybase engine?

I believe between us the experts who have contributed here have answered the questions. I recommend the question be closed as answered, with a split between alpmoon, IncisiveOne and myself.

by: PinkSandPosted on 2009-08-08 at 18:03:21ID: 31604061

As Joe suggested, we had the admin people to check the system and may need to bring in people from SUN. We have not found the answer to the problem yet.

Question

Sybase Engine has over 85% kernel usage on rebuilding cluster index

Subscribe now for full access to Experts Exchange and get

Instant Access to this Solution

Sybase kernel usage

Sybase Database

Trusted by hundreds of thousands everyday for fast, accurate and reliable tech support.

See what Experts Exchange can do for you.

Got a question?

We've got the answer.

Need individual assistance?

Our experts are ready to help.

Want to learn from the best?

Read articles from industry experts.

Working on a long term project?

Store your work and research.

What Makes Experts Exchange Unique?

Trusted by the world's most respected brands.

Faithfully serving IT professionals since 1996.

Related Solutions

Free Tech Articles

Cloud Class Webinars

Join the Community

Give a Little. Get a Lot.

Answers

3 Ways to Join

The Experts

The Experts

Testimonials

Testimonials

Testimonials

Business Clients

Business Clients

In the Press

In the Press

In the Press