Solved

_SC_THREAD_KEY_MAX

Posted on 2011-09-23
27
380 Views
Last Modified: 2012-05-12
If i run the below command on hp ia64 box,

# tusc -aeflh -o <sham.txt> <executable> <args>

my executable is failing with pthread_create() rc 11 error after creating 86 threads.


I see that sham .txt is showing this values at some point.

    {1543737} sysconf(_SC_THREAD_THREADS_MAX) ....................... = 8192
    {1543737} sysconf(_SC_THREAD_KEYS_MAX) .............................. = 256

What command or configuration has to be applied to make pthread_create() successful?

Sham

0
Comment
Question by:mohet01
  • 16
  • 11
27 Comments
 
LVL 62

Expert Comment

by:gheist
ID: 36713592
You should create code that does not create bigger thread pool than possible.
Threads use heap and/or stack space too.
0
 

Author Comment

by:mohet01
ID: 36714810
Hello ghiest

"You should create code that does not create bigger thread pool than possible"
On hp-ux / linux when a forked  process creates n limit threads,
Will OS SCHEDULER recognize each thread as individual unit of work in the run queue and gives separate time slice for each thread in addition to processes?
If my point is correct, That is why high thread scenario has less throughput?

Sham
0
 
LVL 62

Expert Comment

by:gheist
ID: 36714909
Involved task switching method is extremely scalable, you should not worry about that. (it is much more complicated than just queuing round-robin you mentioned, even "classic" unix scheduler includes priority aging for example)
HP-UX is notorious about having HUGE thread structure (exceeding in memory consumption that of a process)leading to memory exhaustion way too often thus suggestion of implementing thread pool. On the other hand on Linux you will have 16000 threads allowed and working by default.
0
Free learning courses: Active Directory Deep Dive

Get a firm grasp on your IT environment when you learn Active Directory best practices with Veeam! Watch all, or choose any amount, of this three-part webinar series to improve your skills. From the basics to virtualization and backup, we got you covered.

 

Author Comment

by:mohet01
ID: 36717773
Ok i got your answer.
Can u please confirm the following?
Generally speaking, Will OS SCHEDULER recognize each thread as individual unit of work in the run queue and gives separate time slice for each thread in addition to processes irrespective of scheduling technique that it uses?
0
 
LVL 62

Expert Comment

by:gheist
ID: 36718049
It depends.

Multiple threading implementations exist
1:n or software threads
1:1 or all threads are handled as processes
m:n where threads which have work are assigned to kernel-handled processes.

To me it seems that behaviour you describe is No 2 but linux has 2 or 3, but some threading libraries in more exotic configurations default ot No 1
0
 

Author Comment

by:mohet01
ID: 36718104
OK
So,
1:n, mean scheduler does not know about these n threads and the process itself has scheduler code which divides the time slice given to this process, if am correct,
Can u give an example OS or thread library which behaves like this?

1:1 is very clear

m:n, am really not clear about this,
Which thread  library in linux provides htis facility?

0
 
LVL 62

Expert Comment

by:gheist
ID: 36718159
http://linux.die.net/man/7/pthreads

timeline of pthreads starts at 15% of manual page.
0
 

Author Comment

by:mohet01
ID: 36719217
this link talks about m:n
http://linux.die.net/man/7/pthreads

what about this query,
1:n, mean scheduler does not know about these n threads and the process itself has scheduler code which divides the time slice given to this process, if am correct,
Can u give an example OS or thread library which behaves like this?

Sham


0
 

Author Comment

by:mohet01
ID: 36947095
Hello gheist
1:n, mean scheduler does not know about these n threads and the process itself has scheduler code which divides the time slice given to this process, if am correct,
Can u give an example OS or thread library which behaves like this?
Sham

0
 
LVL 62

Expert Comment

by:gheist
ID: 36947644
OpenBSD v3.1 (<year 2002)
0
 

Author Comment

by:mohet01
ID: 36947786
Hello gheist
If i take machine
HP-UX lodlchp3 B.11.23 U 9000/800 2704064441 unlimited-user license
I am able to create 1000 threads

If i take machine
HP-UX lodii018 B.11.31 U ia64 3276545932 unlimited-user license
I am ableto create not more than 85 threads

Sham
0
 
LVL 62

Expert Comment

by:gheist
ID: 36948064
what happens if you compile with gcc against gnu pthread headers?
0
 

Author Comment

by:mohet01
ID: 36948466
i dont know that there is something called gnu pthread headers, may be if you can say more on that then i can understand your updates.
currently am using pthread.h
in mak file, it looks like lthis
ifeq ($(SYSTYPE), HP-UX)
CLIBES := -ldld -lm -lnet -lV3 -lsec -lpthread -lrt -lnsl

Sham
0
 

Author Comment

by:mohet01
ID: 36948558
content of pthread.h in /usr/include on problem machine is:
/*
 * @(#)B.11.31_LR
 */

#ifndef _PTHREAD_INCLUDED /* allows multiple inclusion */
#define _PTHREAD_INCLUDED

#ifdef _PTHREADS_DRAFT4
#    include <dce/pthread.h>
#else
#  ifdef _INCLUDE_XOPEN_SOURCE_500
#    ifndef _TIME_INCLUDED
#      include <time.h>
#    endif /* _TIME_INCLUDED */
#  endif /* _INCLUDE_XOPEN_SOURCE_500 */
#    include <sys/pthread.h>
#endif

#endif /* _PTHREAD_INCLUDED */


content of pthread.h in /usr/include on working machine is:
/*
 * @(#)B11.23_LR
 */

#ifndef _PTHREAD_INCLUDED /* allows multiple inclusion */
#define _PTHREAD_INCLUDED

#ifdef _PTHREADS_DRAFT4
#    include <dce/pthread.h>
#else
#    include <sys/pthread.h>
#endif

#endif /* _PTHREAD_INCLUDED */


Sham


0
 
LVL 62

Expert Comment

by:gheist
ID: 36950639
try including or not including time.h
0
 

Author Comment

by:mohet01
ID: 36951280
hello gheist
what is the idea behind your point , please let me know
Sham
0
 

Author Comment

by:mohet01
ID: 36951337
hello
When there are changes at system header files, there would be more places changes done by HP.
sham
0
 

Author Comment

by:mohet01
ID: 36951367
I also see that sys/pthread.h is also different on working and non-working machines.
Sham
0
 
LVL 62

Expert Comment

by:gheist
ID: 36955293
ifdef in v31 examle includes sys/time.h in some conditions.
it could happen that now it is included twice or not at all and that leads to less efficient thread usage.

0
 

Author Comment

by:mohet01
ID: 36961831
Hello gheist
one interesting update
problem machine is itanium machine

hp support says that perform the following, What do you understand from this?

==========================================
This is a PA-RISC program which gets emulated by aries(5) on Itanium systems. The default aries settings allow only 85 threads to be created. If you need more, then the aries heap must be increased. Calculate the necessary heapsize as

           Required ARIES Heap Size =
               4 MB + (number of PA-RISC application threads) * 215KB

For 1000 threads you need a heapsize of 219 MB. Create an .ariesrc file e.g. in your home directory with a line like this:

  /opt/CA/SharedComponents/ccs/cci/bin/hammer -heap_ssz 219000
==============================================

Sham
0
 

Author Comment

by:mohet01
ID: 36961847
BTW
hammer is the application which actually calls pthread_create() 1000 times
0
 
LVL 62

Expert Comment

by:gheist
ID: 36963477
you can check with ulimit -a if you have enough heap for threads + all other allocations.
0
 

Author Comment

by:mohet01
ID: 37055226
Hello gheist
increasing the heap size for aries emulator resolved thread creation problem.
thanx for the help
Sham
0
 
LVL 62

Expert Comment

by:gheist
ID: 37055240
Linux is probably more comfortable since it has no limits (or huge) of any resource, thus "everything works" even if nobody cares about tuning stuff.
0
 

Author Comment

by:mohet01
ID: 37236627
hello gheist
root cause is
running hp pa-risc binaries on ia64 thru aries
Sham
0
 
LVL 62

Accepted Solution

by:
gheist earned 500 total points
ID: 37236759
solution is recompilation or making a powerful PA-RISC server from parts of two old servers.

Otherwise "man aries" ->
http://h21007.www2.hp.com/portal/download/files/unprot/hpux/aries/aries_manpage_11.31_PHCO_36447.txt

Namely
    Relation Between pa_maxssiz_32|64bit and ARIES Parameters
      If the value of kernel tunable parameter maxssiz is much higher than
      its default value, any additional memory obtained by increasing the
      kernel tunable parameter pa_maxssiz_32/64bit will be used by ARIES to
      increase the size of PA-RISC application stack and it may not be
      possible to increase the size of ARIES parameter -heap_ssz.  To
      achieve optimal memory utilization in such cases, HP recommends that
      you limit the size of PA-RISC application stack by setting ARIES
      option -ssz value to a reasonable value e.g.  8192 kbytes and use
      ARIES option -heap_ssz to increase the size of ARIES heap to be able
      to create more number of threads than possible with default ARIES
      option values.
0
 

Author Closing Comment

by:mohet01
ID: 37236763
Thanx
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
awk file 6 79
maybe no no httpd.conf 6 63
Trying to install php56 on CentOS 7 get GPG error 5 30
Secondary drive is failing... D Volume 11 17
Why Shell Scripting? Shell scripting is a powerful method of accessing UNIX systems and it is very flexible. Shell scripts are required when we want to execute a sequence of commands in Unix flavored operating systems. “Shell” is the command line i…
It’s 2016. Password authentication should be dead — or at least close to dying. But, unfortunately, it has not traversed Quagga stage yet. Using password authentication is like laundering hotel guest linens with a washboard — it’s Passé.
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use nested-loops in the C programming language.
The goal of this video is to provide viewers with basic examples to understand opening and reading files in the C programming language.

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question