Solved

_SC_THREAD_KEY_MAX

Posted on 2011-09-23
27
373 Views
Last Modified: 2012-05-12
If i run the below command on hp ia64 box,

# tusc -aeflh -o <sham.txt> <executable> <args>

my executable is failing with pthread_create() rc 11 error after creating 86 threads.


I see that sham .txt is showing this values at some point.

    {1543737} sysconf(_SC_THREAD_THREADS_MAX) ....................... = 8192
    {1543737} sysconf(_SC_THREAD_KEYS_MAX) .............................. = 256

What command or configuration has to be applied to make pthread_create() successful?

Sham

0
Comment
Question by:mohet01
  • 16
  • 11
27 Comments
 
LVL 61

Expert Comment

by:gheist
ID: 36713592
You should create code that does not create bigger thread pool than possible.
Threads use heap and/or stack space too.
0
 

Author Comment

by:mohet01
ID: 36714810
Hello ghiest

"You should create code that does not create bigger thread pool than possible"
On hp-ux / linux when a forked  process creates n limit threads,
Will OS SCHEDULER recognize each thread as individual unit of work in the run queue and gives separate time slice for each thread in addition to processes?
If my point is correct, That is why high thread scenario has less throughput?

Sham
0
 
LVL 61

Expert Comment

by:gheist
ID: 36714909
Involved task switching method is extremely scalable, you should not worry about that. (it is much more complicated than just queuing round-robin you mentioned, even "classic" unix scheduler includes priority aging for example)
HP-UX is notorious about having HUGE thread structure (exceeding in memory consumption that of a process)leading to memory exhaustion way too often thus suggestion of implementing thread pool. On the other hand on Linux you will have 16000 threads allowed and working by default.
0
 

Author Comment

by:mohet01
ID: 36717773
Ok i got your answer.
Can u please confirm the following?
Generally speaking, Will OS SCHEDULER recognize each thread as individual unit of work in the run queue and gives separate time slice for each thread in addition to processes irrespective of scheduling technique that it uses?
0
 
LVL 61

Expert Comment

by:gheist
ID: 36718049
It depends.

Multiple threading implementations exist
1:n or software threads
1:1 or all threads are handled as processes
m:n where threads which have work are assigned to kernel-handled processes.

To me it seems that behaviour you describe is No 2 but linux has 2 or 3, but some threading libraries in more exotic configurations default ot No 1
0
 

Author Comment

by:mohet01
ID: 36718104
OK
So,
1:n, mean scheduler does not know about these n threads and the process itself has scheduler code which divides the time slice given to this process, if am correct,
Can u give an example OS or thread library which behaves like this?

1:1 is very clear

m:n, am really not clear about this,
Which thread  library in linux provides htis facility?

0
 
LVL 61

Expert Comment

by:gheist
ID: 36718159
http://linux.die.net/man/7/pthreads

timeline of pthreads starts at 15% of manual page.
0
 

Author Comment

by:mohet01
ID: 36719217
this link talks about m:n
http://linux.die.net/man/7/pthreads

what about this query,
1:n, mean scheduler does not know about these n threads and the process itself has scheduler code which divides the time slice given to this process, if am correct,
Can u give an example OS or thread library which behaves like this?

Sham


0
 

Author Comment

by:mohet01
ID: 36947095
Hello gheist
1:n, mean scheduler does not know about these n threads and the process itself has scheduler code which divides the time slice given to this process, if am correct,
Can u give an example OS or thread library which behaves like this?
Sham

0
 
LVL 61

Expert Comment

by:gheist
ID: 36947644
OpenBSD v3.1 (<year 2002)
0
 

Author Comment

by:mohet01
ID: 36947786
Hello gheist
If i take machine
HP-UX lodlchp3 B.11.23 U 9000/800 2704064441 unlimited-user license
I am able to create 1000 threads

If i take machine
HP-UX lodii018 B.11.31 U ia64 3276545932 unlimited-user license
I am ableto create not more than 85 threads

Sham
0
 
LVL 61

Expert Comment

by:gheist
ID: 36948064
what happens if you compile with gcc against gnu pthread headers?
0
 

Author Comment

by:mohet01
ID: 36948466
i dont know that there is something called gnu pthread headers, may be if you can say more on that then i can understand your updates.
currently am using pthread.h
in mak file, it looks like lthis
ifeq ($(SYSTYPE), HP-UX)
CLIBES := -ldld -lm -lnet -lV3 -lsec -lpthread -lrt -lnsl

Sham
0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 

Author Comment

by:mohet01
ID: 36948558
content of pthread.h in /usr/include on problem machine is:
/*
 * @(#)B.11.31_LR
 */

#ifndef _PTHREAD_INCLUDED /* allows multiple inclusion */
#define _PTHREAD_INCLUDED

#ifdef _PTHREADS_DRAFT4
#    include <dce/pthread.h>
#else
#  ifdef _INCLUDE_XOPEN_SOURCE_500
#    ifndef _TIME_INCLUDED
#      include <time.h>
#    endif /* _TIME_INCLUDED */
#  endif /* _INCLUDE_XOPEN_SOURCE_500 */
#    include <sys/pthread.h>
#endif

#endif /* _PTHREAD_INCLUDED */


content of pthread.h in /usr/include on working machine is:
/*
 * @(#)B11.23_LR
 */

#ifndef _PTHREAD_INCLUDED /* allows multiple inclusion */
#define _PTHREAD_INCLUDED

#ifdef _PTHREADS_DRAFT4
#    include <dce/pthread.h>
#else
#    include <sys/pthread.h>
#endif

#endif /* _PTHREAD_INCLUDED */


Sham


0
 
LVL 61

Expert Comment

by:gheist
ID: 36950639
try including or not including time.h
0
 

Author Comment

by:mohet01
ID: 36951280
hello gheist
what is the idea behind your point , please let me know
Sham
0
 

Author Comment

by:mohet01
ID: 36951337
hello
When there are changes at system header files, there would be more places changes done by HP.
sham
0
 

Author Comment

by:mohet01
ID: 36951367
I also see that sys/pthread.h is also different on working and non-working machines.
Sham
0
 
LVL 61

Expert Comment

by:gheist
ID: 36955293
ifdef in v31 examle includes sys/time.h in some conditions.
it could happen that now it is included twice or not at all and that leads to less efficient thread usage.

0
 

Author Comment

by:mohet01
ID: 36961831
Hello gheist
one interesting update
problem machine is itanium machine

hp support says that perform the following, What do you understand from this?

==========================================
This is a PA-RISC program which gets emulated by aries(5) on Itanium systems. The default aries settings allow only 85 threads to be created. If you need more, then the aries heap must be increased. Calculate the necessary heapsize as

           Required ARIES Heap Size =
               4 MB + (number of PA-RISC application threads) * 215KB

For 1000 threads you need a heapsize of 219 MB. Create an .ariesrc file e.g. in your home directory with a line like this:

  /opt/CA/SharedComponents/ccs/cci/bin/hammer -heap_ssz 219000
==============================================

Sham
0
 

Author Comment

by:mohet01
ID: 36961847
BTW
hammer is the application which actually calls pthread_create() 1000 times
0
 
LVL 61

Expert Comment

by:gheist
ID: 36963477
you can check with ulimit -a if you have enough heap for threads + all other allocations.
0
 

Author Comment

by:mohet01
ID: 37055226
Hello gheist
increasing the heap size for aries emulator resolved thread creation problem.
thanx for the help
Sham
0
 
LVL 61

Expert Comment

by:gheist
ID: 37055240
Linux is probably more comfortable since it has no limits (or huge) of any resource, thus "everything works" even if nobody cares about tuning stuff.
0
 

Author Comment

by:mohet01
ID: 37236627
hello gheist
root cause is
running hp pa-risc binaries on ia64 thru aries
Sham
0
 
LVL 61

Accepted Solution

by:
gheist earned 500 total points
ID: 37236759
solution is recompilation or making a powerful PA-RISC server from parts of two old servers.

Otherwise "man aries" ->
http://h21007.www2.hp.com/portal/download/files/unprot/hpux/aries/aries_manpage_11.31_PHCO_36447.txt

Namely
    Relation Between pa_maxssiz_32|64bit and ARIES Parameters
      If the value of kernel tunable parameter maxssiz is much higher than
      its default value, any additional memory obtained by increasing the
      kernel tunable parameter pa_maxssiz_32/64bit will be used by ARIES to
      increase the size of PA-RISC application stack and it may not be
      possible to increase the size of ARIES parameter -heap_ssz.  To
      achieve optimal memory utilization in such cases, HP recommends that
      you limit the size of PA-RISC application stack by setting ARIES
      option -ssz value to a reasonable value e.g.  8192 kbytes and use
      ARIES option -heap_ssz to increase the size of ARIES heap to be able
      to create more number of threads than possible with default ARIES
      option values.
0
 

Author Closing Comment

by:mohet01
ID: 37236763
Thanx
0

Featured Post

Get up to 2TB FREE CLOUD per backup license!

An exclusive Black Friday offer just for Expert Exchange audience! Buy any of our top-rated backup solutions & get up to 2TB free cloud per system! Perform local & cloud backup in the same step, and restore instantly—anytime, anywhere. Grab this deal now before it disappears!

Join & Write a Comment

SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
This is a short and sweet, but (hopefully) to the point article. There seems to be some fundamental misunderstanding about the function prototype for the "main" function in C and C++, more specifically what type this function should return. I see so…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
The goal of this video is to provide viewers with basic examples to understand how to create, access, and change arrays in the C programming language.

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now