[Last Call] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 411
  • Last Modified:

_SC_THREAD_KEY_MAX

If i run the below command on hp ia64 box,

# tusc -aeflh -o <sham.txt> <executable> <args>

my executable is failing with pthread_create() rc 11 error after creating 86 threads.


I see that sham .txt is showing this values at some point.

    {1543737} sysconf(_SC_THREAD_THREADS_MAX) ....................... = 8192
    {1543737} sysconf(_SC_THREAD_KEYS_MAX) .............................. = 256

What command or configuration has to be applied to make pthread_create() successful?

Sham

0
mohet01
Asked:
mohet01
  • 16
  • 11
1 Solution
 
gheistCommented:
You should create code that does not create bigger thread pool than possible.
Threads use heap and/or stack space too.
0
 
mohet01Author Commented:
Hello ghiest

"You should create code that does not create bigger thread pool than possible"
On hp-ux / linux when a forked  process creates n limit threads,
Will OS SCHEDULER recognize each thread as individual unit of work in the run queue and gives separate time slice for each thread in addition to processes?
If my point is correct, That is why high thread scenario has less throughput?

Sham
0
 
gheistCommented:
Involved task switching method is extremely scalable, you should not worry about that. (it is much more complicated than just queuing round-robin you mentioned, even "classic" unix scheduler includes priority aging for example)
HP-UX is notorious about having HUGE thread structure (exceeding in memory consumption that of a process)leading to memory exhaustion way too often thus suggestion of implementing thread pool. On the other hand on Linux you will have 16000 threads allowed and working by default.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
mohet01Author Commented:
Ok i got your answer.
Can u please confirm the following?
Generally speaking, Will OS SCHEDULER recognize each thread as individual unit of work in the run queue and gives separate time slice for each thread in addition to processes irrespective of scheduling technique that it uses?
0
 
gheistCommented:
It depends.

Multiple threading implementations exist
1:n or software threads
1:1 or all threads are handled as processes
m:n where threads which have work are assigned to kernel-handled processes.

To me it seems that behaviour you describe is No 2 but linux has 2 or 3, but some threading libraries in more exotic configurations default ot No 1
0
 
mohet01Author Commented:
OK
So,
1:n, mean scheduler does not know about these n threads and the process itself has scheduler code which divides the time slice given to this process, if am correct,
Can u give an example OS or thread library which behaves like this?

1:1 is very clear

m:n, am really not clear about this,
Which thread  library in linux provides htis facility?

0
 
gheistCommented:
http://linux.die.net/man/7/pthreads

timeline of pthreads starts at 15% of manual page.
0
 
mohet01Author Commented:
this link talks about m:n
http://linux.die.net/man/7/pthreads

what about this query,
1:n, mean scheduler does not know about these n threads and the process itself has scheduler code which divides the time slice given to this process, if am correct,
Can u give an example OS or thread library which behaves like this?

Sham


0
 
mohet01Author Commented:
Hello gheist
1:n, mean scheduler does not know about these n threads and the process itself has scheduler code which divides the time slice given to this process, if am correct,
Can u give an example OS or thread library which behaves like this?
Sham

0
 
gheistCommented:
OpenBSD v3.1 (<year 2002)
0
 
mohet01Author Commented:
Hello gheist
If i take machine
HP-UX lodlchp3 B.11.23 U 9000/800 2704064441 unlimited-user license
I am able to create 1000 threads

If i take machine
HP-UX lodii018 B.11.31 U ia64 3276545932 unlimited-user license
I am ableto create not more than 85 threads

Sham
0
 
gheistCommented:
what happens if you compile with gcc against gnu pthread headers?
0
 
mohet01Author Commented:
i dont know that there is something called gnu pthread headers, may be if you can say more on that then i can understand your updates.
currently am using pthread.h
in mak file, it looks like lthis
ifeq ($(SYSTYPE), HP-UX)
CLIBES := -ldld -lm -lnet -lV3 -lsec -lpthread -lrt -lnsl

Sham
0
 
mohet01Author Commented:
content of pthread.h in /usr/include on problem machine is:
/*
 * @(#)B.11.31_LR
 */

#ifndef _PTHREAD_INCLUDED /* allows multiple inclusion */
#define _PTHREAD_INCLUDED

#ifdef _PTHREADS_DRAFT4
#    include <dce/pthread.h>
#else
#  ifdef _INCLUDE_XOPEN_SOURCE_500
#    ifndef _TIME_INCLUDED
#      include <time.h>
#    endif /* _TIME_INCLUDED */
#  endif /* _INCLUDE_XOPEN_SOURCE_500 */
#    include <sys/pthread.h>
#endif

#endif /* _PTHREAD_INCLUDED */


content of pthread.h in /usr/include on working machine is:
/*
 * @(#)B11.23_LR
 */

#ifndef _PTHREAD_INCLUDED /* allows multiple inclusion */
#define _PTHREAD_INCLUDED

#ifdef _PTHREADS_DRAFT4
#    include <dce/pthread.h>
#else
#    include <sys/pthread.h>
#endif

#endif /* _PTHREAD_INCLUDED */


Sham


0
 
gheistCommented:
try including or not including time.h
0
 
mohet01Author Commented:
hello gheist
what is the idea behind your point , please let me know
Sham
0
 
mohet01Author Commented:
hello
When there are changes at system header files, there would be more places changes done by HP.
sham
0
 
mohet01Author Commented:
I also see that sys/pthread.h is also different on working and non-working machines.
Sham
0
 
gheistCommented:
ifdef in v31 examle includes sys/time.h in some conditions.
it could happen that now it is included twice or not at all and that leads to less efficient thread usage.

0
 
mohet01Author Commented:
Hello gheist
one interesting update
problem machine is itanium machine

hp support says that perform the following, What do you understand from this?

==========================================
This is a PA-RISC program which gets emulated by aries(5) on Itanium systems. The default aries settings allow only 85 threads to be created. If you need more, then the aries heap must be increased. Calculate the necessary heapsize as

           Required ARIES Heap Size =
               4 MB + (number of PA-RISC application threads) * 215KB

For 1000 threads you need a heapsize of 219 MB. Create an .ariesrc file e.g. in your home directory with a line like this:

  /opt/CA/SharedComponents/ccs/cci/bin/hammer -heap_ssz 219000
==============================================

Sham
0
 
mohet01Author Commented:
BTW
hammer is the application which actually calls pthread_create() 1000 times
0
 
gheistCommented:
you can check with ulimit -a if you have enough heap for threads + all other allocations.
0
 
mohet01Author Commented:
Hello gheist
increasing the heap size for aries emulator resolved thread creation problem.
thanx for the help
Sham
0
 
gheistCommented:
Linux is probably more comfortable since it has no limits (or huge) of any resource, thus "everything works" even if nobody cares about tuning stuff.
0
 
mohet01Author Commented:
hello gheist
root cause is
running hp pa-risc binaries on ia64 thru aries
Sham
0
 
gheistCommented:
solution is recompilation or making a powerful PA-RISC server from parts of two old servers.

Otherwise "man aries" ->
http://h21007.www2.hp.com/portal/download/files/unprot/hpux/aries/aries_manpage_11.31_PHCO_36447.txt

Namely
    Relation Between pa_maxssiz_32|64bit and ARIES Parameters
      If the value of kernel tunable parameter maxssiz is much higher than
      its default value, any additional memory obtained by increasing the
      kernel tunable parameter pa_maxssiz_32/64bit will be used by ARIES to
      increase the size of PA-RISC application stack and it may not be
      possible to increase the size of ARIES parameter -heap_ssz.  To
      achieve optimal memory utilization in such cases, HP recommends that
      you limit the size of PA-RISC application stack by setting ARIES
      option -ssz value to a reasonable value e.g.  8192 kbytes and use
      ARIES option -heap_ssz to increase the size of ARIES heap to be able
      to create more number of threads than possible with default ARIES
      option values.
0
 
mohet01Author Commented:
Thanx
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

  • 16
  • 11
Tackle projects and never again get stuck behind a technical roadblock.
Join Now