Conference hydra::axp-developer

Greetings!

  I'm just going to send Hu a quick "ACK" back, I'll research this later...



						Tony Swierkowski
						Digital Equipment Corporation
						Software Partner Engineering
						Palo Alto, California
						(415) 617-3601
						"[email protected]"

P.S.    Message I sent to developer is below for reference...

From:	HYDRA::AXPDEVELOPER "[email protected]" 21-APR-1997 17:58:17.35
To:	SMTP%"[email protected]"
CC:	AXPDEVELOPER
Subj:	RE: threads

Greetings!

  This message is to acknowledge the email request you sent to the ASAP
developer's support address re: threads on Digital UNIX.  I'll research 
this and get back to you as soon as possible, cheers...


                                                Tony Swierkowski
                                                Digital Equipment Corporation
                                                Software Partner Engineering
                                                Palo Alto, California
                                                (415) 617-3601
                                                "[email protected]"

P.S.    Your original message re: "threads (on Digital UNIX)" is below
        for your reference...

From:	SMTP%"[email protected]" 21-APR-1997 09:46:32.00
To:	"[email protected]" <[email protected]>
CC:	
Subj:	threads

Return-Path: [email protected]
Received: by vaxsim.mro.dec.com (UCX V4.1-12, OpenVMS V6.2 VAX);
	Mon, 21 Apr 1997 09:46:29 -0400
Received: from smtp.inet.fi by mail13.digital.com (8.7.5/UNX 1.5/1.0/WV)
	id JAA24324; Mon, 21 Apr 1997 09:38:50 -0400 (EDT)
Received: from [email protected] (port 1580 [194.197.150.5]) by smtp.inet.fi with SMTP id <17113-1541>; Mon, 21 Apr 1997 16:00:55 +0300
Received: from localhost by armi.softline.fi; (5.65/1.1.8.2/29Apr96-0950AM)
	id AA02354; Mon, 21 Apr 1997 15:08:02 +0200
Date: Mon, 21 Apr 1997 16:08:02 +0300 (EET)
From: Hu Rui <[email protected]>
To: "[email protected]" <[email protected]>
Subject: threads
In-Reply-To: <[email protected]>
Message-Id: <[email protected]>
Organization: Scandinavian Softline Technology
Mime-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII


Digital Unix 4.0B on Alpha Station 255
How can I choose for user-level and kernel-level threads implementation? I
want to make a performance test.
What is the default implementation. I just installed every thing with
default options.

Regards.

_________________________________________________
Hu Rui		 (Oracle Support No.        14428)
R&D, SMS Unit    (Digital ASAP Support No. A60205)
Scandinavian Softline Technology Oy
Tulkinkuja 3   02600 ESPOO   Finland
Tel +358-9-5495 6202  Fax +358-9-512 4629
Home Tel +358-9-2789426
Internet [email protected]  http://www.softline.fi/
_________________________________________________

Greetings!

  I had a chance to do a little research on this and crafted the message below
for Hu.  I'm no threads guru, so if anyone know more details on the use & abuse
of what he refers to as "user-level" vs "kernel-level", please feel free to jump
in, cheers...


						Tony Swierkowski
						Digital Equipment Corporation
						Software Partner Engineering
						Palo Alto, California
						(415) 617-3601
						"[email protected]"

P.S.    Message I sent to developer is below for reference...

From:	HYDRA::AXPDEVELOPER "[email protected]" 21-APR-1997 18:34:52.02
To:	SMTP%"[email protected]"
CC:	AXPDEVELOPER
Subj:	RE: threads

Greetings!

  Basic information on system parameters (like maxuthread and threadmax) that
govern thread usage is documented in Chapter 4 of the "System Administration"
manual (see section 4.6.1.9 and 4.6.1.10).  In addition, related parameters
(like task_max, maxusers and maxuprc) are in sections 4.6.1.11 and 4.6.1.12
of the "System Administration" manual.

  For details on kernel threads see Part 3 in the "Writing Device Drivers: 
Advanced Topics" manual.  From section 6.1 in that manual:

<--->

The term kernel thread distinguishes between the threads that application 
programs use. A kernel thread is a single sequential flow of control within 
a device driver or other systems-based program.  The device driver or other 
systems-based program makes use of the kernel interfaces (instead of a
threads library package such as DECthreads) to start, terminate, delete, 
and perform other kernel threads-related operations. 

<--->

  Is this what you are looking for?.  In a nutshell, in order to use kernel
threads, you do *not* use the standard DECthreads library (see "Guide to 
DECthreads" manual if interested).  Instead, you use the Kernel Threads-related
Interfaces documented in Table 6-1 of the "Writing Device Drivers: Advanced
Topics" manual.  Let me know if this is the information you needed, cheers...



						Tony Swierkowski
						Digital Equipment Corporation
						Software Partner Engineering
						Palo Alto, California
						(415) 617-3601
						"[email protected]"

Tony --

Here's a note from another conference.  The context is different, but there is a
lot of discussion of kernel vs. user threads in this note stream.  Hope this
helps more than confuses!

								-- Chuck Newman
--------------------------------------------------------------------------------
       <<< TURRIS::DISK$NOTES_PACK2:[NOTES$LIBRARY]DIGITAL_UNIX.NOTE;1 >>>
                               -< DIGITAL UNIX >-
================================================================================
Note 8467.1           How to determine cpu time per thread               1 of 16
SMURF::DENHAM "USG"                                  25 lines  15-JAN-1997 10:27
--------------------------------------------------------------------------------
    The only documented means to gather per-thread (note: kernel or Mach
    thread) info is getrusage(RUSAGE_THREAD). The one gothca for this
    is that call operate on the *current* thread, so threads would have
    to poll their own usage. There are some undocumented and officially
    unsupported Mach interfaces to get info on arbitrary threads in
    a process. I wouldn't encourage using this except as a last resort.
    
    There are other hitches as well. On V4.0, there is no longer a
    one-to-one binding between the program's POSIX threads and the
    kernel/Mach threads they run "on." The DECthreads library can
    use a variety of kernel threads to "host" the applcation's
    user threads. So it becomes virtually impossible to gather
    per-pthread usage statistics.
    
    On the release that looks like V4.0D (due out this summer), DECthreads
    will provide support for "system contention scope." This means that
    an application thread can bind itself permanently to a kernel thread.
    That can get you a more consistent usage picture, but may not
    necessarily be the best model for the application.
    
    I've got a request to implement the PIOCUSAGE proc ioctl in a future
    release. A PIOCTUSAGE (per kerenl thead) call might make sense as well
    to replace the Mach calls for thread info....
    
    Sorry the picutre here isn't prettier.

Greetings!

  Re: .3, Thanks!  I had read that (along with dozens of other "thread" related
notes in the CMA and DIGITAL_UNIX conferences).  I think I understand some of 
the issues a little better but by no means am I up to speed on this (nobody
else is either locally that I've found so far)...

  Hu sent more mail to the "[email protected]" account that was 
forwarded to me this morning (appended below).  I did craft up a response 
(also appended below) in response to that email from Hu.  I hope I didn't 
lead this guy too far down the wrong path with my response.  Anyone out there
feeling threads-literate, feel free to answer any of Hu's questions and/or
critique/comment on the reply I sent him, cheers...



						Tony Swierkowski
						Digital Equipment Corporation
						Software Partner Engineering
						Palo Alto, California
						(415) 617-3601
						"[email protected]"

P.S.	Hu's latest (and my response) is below - please correct me if I'm
	really hosed up here...

From:	DECPA::HYDRA::AXPDEVELOPER "[email protected]" 24-APR-1997 02:24:53.83
To:	[email protected]
CC:	hydra::axpdeveloper
Subj:	FWD: Performace Turning for User-level Threads Program

From:	SMTP%"[email protected]"   24-APR-1997 05:01:23.03
To:	"[email protected]" <[email protected]>
CC:	Kari Kailamaki <[email protected]>
Subj:	Performace Turning for User-level Threads Program

Return-Path: [email protected]
Received: by vaxsim.mro.dec.com (UCX V4.1-12, OpenVMS V6.2 VAX);
	Thu, 24 Apr 1997 05:01:18 -0400
Received: from smtp.inet.fi by mail13.digital.com (8.7.5/UNX 1.5/1.0/WV)
	id EAA27970; Thu, 24 Apr 1997 04:58:04 -0400 (EDT)
Received: from [email protected] (port 4335 [194.197.150.5]) by 
smtp.inet.fi with SMTP id <17678-1570>; Thu, 24 Apr 1997 11:21:35 +0300
Received: from localhost by armi.softline.fi; (5.65/1.1.8.2/29Apr96-0950AM)
	id AA07294; Thu, 24 Apr 1997 10:28:31 +0200
Date: Thu, 24 Apr 1997 11:28:30 +0300 (EET)
From: Hu Rui <[email protected]>
To: "[email protected]" <[email protected]>
Cc: Kari Kailamaki <[email protected]>
Subject: Performace Turning for User-level Threads Program
In-Reply-To: <[email protected]>
Message-Id: <[email protected]>
Organization: Scandinavian Softline Technology
Mime-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

To;
> 
> 
> 						Tony Swierkowski
> 						Digital Equipment Corporation
> 						Software Partner Engineering
> 						Palo Alto, California
> 						(415) 617-3601
> 						"[email protected]"

Alpha Station 255, Unix 4.0B

Thank you very much for your previous answer. What I am using is
user-level threads. I choose user-level threads instead of multi-process
implementation is to avoid unnecessary switching between processes.

One problem with user-level threads, it can not dynamiclly get more CPU
time when more threads are created in the process. In my program I can
have 1 -- maybe 50 threads. I need more CPU time when I have 50 threads
in the process.

- I am superuser and my program running as root process.
- How I can set schedule policy and change the priority dynamically?
- What is the structure that kernel keep for each process, how to modify
it. (In SCO unix it is the struct proc keep all the information about
process), what is in the Digital?
- I need to write some program similar to "TOP" to monitor the running of
the whole machine, it read the process table in predefined order, and make
stastics.

- I need some material talking about the kernel data structure, and where
those structures are and how to read and modify them.

Regards.


_________________________________________________
Hu Rui		 (Oracle Support No.        14428)
R&D, SMS Unit    (Digital ASAP Support No. A60205)
Scandinavian Softline Technology Oy
Tulkinkuja 3   02600 ESPOO   Finland
Tel +358-9-5495 6202  Fax +358-9-512 4629
Home Tel +358-9-2789426
Internet [email protected]  http://www.softline.fi/
_________________________________________________



% ====== Internet headers and postmarks (see DECWRL::GATEWAY.DOC) ======
% Received: from pobox1.pa.dec.com by mts-gw.pa.dec.com (5.65/09May94) id AA26367; Thu, 24 Apr 97 02:19:27 -0700
% Received: by pobox1.pa.dec.com; id AA10886; Thu, 24 Apr 97 02:19:26 -0700
% Received: from hydra.enet by us4rmc.pko.dec.com (5.65/rmc-22feb94) id AA23892; Thu, 24 Apr 97 05:22:22 -0400
% Message-Id: <[email protected]>
% Received: from hydra.enet; by us4rmc.enet; Thu, 24 Apr 97 05:22:22 EDT
% Date: Thu, 24 Apr 97 05:22:22 EDT
% From: [email protected] <hydra::axpdeveloper>
% To: [email protected]
% Cc: hydra::axpdeveloper
% Apparently-To: [email protected]
% Subject: FWD: Performace Turning for User-level Threads Program

From:	AMCUCS::SWIERKOWSKI  "Tony Swierkowski" 24-APR-1997 16:47:39.84
To:	SMTP%"[email protected]"
CC:	SWIERKOWSKI
Subj:	Re: Performance Tuning for User-level Threads Program...

Greetings!

  Tony Swierkowski @ Digital here with some answers (?) to the questions you 
posed in your last email re: "Performance Tuning for User-level Threads 
Program".  First a couple of basic questions:

1) What is the real problem you're trying to solve? (i.e. low perceived CPU 
   utilization, a process with one or more threads "stalled", a possible
   deadlock between threads that hangs the process at some point or perhaps
   behaves strangely on an SMP machine?).
   NOTE: You indicated that your are using an "AlphaStation" (a uni-processor),
   so SMP-specific problems can't be tested for unless you also have access to
   an SMP "AlphaServer" in addition to your "AlphaStation".

2) You stated: "...One problem with user-level threads, it can not dynamically 
   get more CPU time when more threads are created in the process...".  What do
   you mean by "get more CPU time"?  Do you see a thread not running that you
   believe should be?  When you use the word "dynamically", are you referring
   to adjusting the thread prioroty level under program control once the thread
   has been created?

3) As you know, the kernel takes care of scheduling processes based on a
   pre-emptive, priority-driven algorithm and can dynamically adjust process
   priority levels, but gives you no help in adjusting priority levels on
   individual threads within that process.  Do you need a mechanism to do
   this automagically as opposed to managing the threads yourself via the
   thread attributes you can control?

>Alpha Station 255, Unix 4.0B
>
>Thank you very much for your previous answer. What I am using is
>user-level threads. I choose user-level threads instead of multi-process
>implementation is to avoid unnecessary switching between processes.

  No problem so far.  As you know all threads in a process share things like
the address space, etc. and a thread context switch is definitely designed to
incur less overhead than a process context switch.  FWIW, user threads are also
known as "lightweight" threads and kernel threads are known as "heavyweight"
threads and while they incur less overhead than a process context switch, they
incur more overhead than a user thread context switch and in general are a lot
more trouble to program with.  The advantage of using kernel threads is the
operating system takes care of distributing them amongst all availble CPU's in
an SMP system, with user threads you decide on the thread attributes that per-
tain to scheduling.

>One problem with user-level threads, it can not dynamiclly get more CPU
>time when more threads are created in the process. In my program I can
>have 1 -- maybe 50 threads. I need more CPU time when I have 50 threads
>in the process.

  The above is what drove my basic question #2.  I'm not sure what you mean
here.  If the CPU is 100% utilized to begin with, the only way a given thread
is going to be scheduled more often (and thus consume more CPU time) is if
some other thread relinquishes control (voluntarily or otherwise).  To better
understand thread scheduling in general, your best resource is the "Guide to 
DECthreads" manual in the Digital UNIX documentation set.  Chapter 2 goes into 
a lot of detail re: thread attributes, including scheduling inheritance, 
scheduling policy and scheduling parameters.  I did spot an interesting comment
in an internal support conference that may shed some light on your starement 
re: "...can not dynamically..." 

<--- begin partial quote --->

Setting scheduling policy and priority in an attributes object does NOT affect
threads created with that attributes object UNLESS you also change the "inherit
scheduling" attribute from the default value of cma_c_sched_inherit to
cma_c_sched_use_default. The default behavior is that newly created threads
inherit the scheduling policy and priority of the creating thread; the policy
and priority values in the attributes object are IGNORED! When you set the
inherit sched attribute to cma_c_sched_use_default, however, a new thread will
inherit the default policy and priority attributes.

<--- end partial quote --->

  Does the above quote help you in understanding your needs as far as thread
attributes like the scheduling policy and scheduling parameters for a new
thread as you create it?


>- I am superuser and my program running as root process.
>- How I can set schedule policy and change the priority dynamically?

  From my understanding of thread attributes in general, once a scheduling 
"policy" or some other thread attribute is chosen, you can then use the 
"pthread_attr_setXXX" routines to then alter thread attributes.  Per Section 
2.2.3.3: "...Change the scheduling policy and parameters of an existing thread
by calling the pthread_setschedparam routine. This allows a thread to change 
its own execution policy and/or priority...".

  Refer also to Section 1.6 (i.e. "DECthreads POSIX 1003.1c Routines Summary")
and also Table 1-1 for details.  Section 2.7 also has a discussion of thread 
scheduling that may help you out.

>- What is the structure that kernel keep for each process, how to modify
>it. (In SCO unix it is the struct proc keep all the information about
>process), what is in the Digital?

  See man pages for 'proc' - For each active or zombie process, there is an 
entry in the system process table, which appears as a file name in the /proc 
directory.  I think the details on the actual 'proc' structure are in
/sys/include/procfs/procfs.h.

>- I need to write some program similar to "TOP" to monitor the running of
>the whole machine, it read the process table in predefined order, and make
>stastics.

  I'm not familar with "top", but I suspect this is on the Digital UNIX
Freeware CD-ROM that shipped with the V4.0 distribution kit.  You can find it
online at the following URL:

- http://www.unix.digital.com/demos/index.html

Check out the following URL:

- http://www.unix.digital.com/demos/freekit/docs/top/README 

for the "readme" file on that Freeware CD-ROM at the site above.  The kit 
itself is at:

- http://www.unix.digital.com/demos/freesrc/Freeware_Digital_src/top-3.3.tar.gz

  FWIW, another very useful tool we use is 'vmubc', you can pull this from the
ftp archives on gatekeeper.dec.com (it's also listed on the Freeware CD-ROM 
index above).

>- I need some material talking about the kernel data structure, and where
>those structures are and how to read and modify them.

  The only place I'm aware of (outside of the Digital UNIX documentation set)
that discusses kernel data structures in general is the Student Guide provided
in the Digital UNIX Internals course(s), but you have to take the course (taught
by Global Knowledge Network) to get the course materials.  There is/was supposed
to be a "UNIX Internals & Data Structures" book similar to the "OpenVMS Inter-
nals & Data Structures" book that has been around for years.  I believe the
"UNIX IDSM" was to be published by Digital Press, but I have no idea whether
the book actually was completed nor do I have an ISBN number for it.  One alter-
native I did found out about is the "UNIX Internals, the New Frontiers", ISBN
0-13-101908-2, Prentice-Hall, Uresh Vahalia.  I think the precis of that book 
used be at " http://www.prenhall.com/ ", you might contact Prentiss-Hall for
details.  Shy of poking around the source listings themselves, I don't know of
other good general resources to better understand the internals and data
structures.  Hope this helps, cheers...



						Tony Swierkowski
						Digital Equipment Corporation
						Software Partner Engineering
						Palo Alto, California
						(415) 617-3601
						"[email protected]"

Greetings!

  Just a quick update.  I've exchanged more mail with Hu re: threads and this
is really boiling down to his need for some training I think.  I'll craft 
another message with some basics and some pointers and see what happens.  Most
of his questions indicate he doesn't really understand OSF/1 internals and the
process vs task vs thread stuff.  I've seen some pretty nice diagrams from an
ancient (pre V3.0?) "Internals" course student guide I have accesss to, but this
is copyrighted so I can't just send him a copy, cheers...


						Tony Swierkowski
						Digital Equipment Corporation
						Software Partner Engineering
						Palo Alto, California
						(415) 617-3601
						"[email protected]"

Greetings!

  Just another quick update.  I've crafted another message for Hu in response to
his latest message re: "threads" and suggested that if it (plus the reading I've
suggested) doesn't click, he needs to pursue outside training.  His last message
indicated he is not very comfortable with the while process vs task vs thread
business as implemented in Digital UNIX.  I'll leave this open a little while
longer until I hear back from him, cheers...


						Tony Swierkowski
						Digital Equipment Corporation
						Software Partner Engineering
						Palo Alto, California
						(415) 617-3601
						"[email protected]"

Greetings!

  I haven't heard back from Hu re: the latest (verbose) message I sent him re:
threads, so I assume I either answered his question(s) or he's figured out he
needs to do some of the reading I suggested all along.  I'm closing this call
for now...


						Tony Swierkowski
						Digital Equipment Corporation
						Software Partner Engineering
						Palo Alto, California
						(415) 617-3601
						"[email protected]"

Greetings!

  Noticed that Hu logged another (#3616) call re: threads with MRO.  I've
exchanged a lot of email with him and per my .1 to his new call, here is the
last mail I got with my response back I sent Hu in early May a week or so
before I closed this call, cheers...


						Tony Swierkowski
						Digital Equipment Corporation
						Software Partner Engineering
						Palo Alto, California
						(415) 617-3601
						"[email protected]"



P.S.	The previous replies to this note have all?/most of the email exchanged
	with Hu re: threads except for the very last pair of messages I now
	include below for your reference...

From:	DECPA::"[email protected]" "Hu Rui" 27-APR-1997 12:36:04.93
To:	Tony Swierkowski <amcucs::swierkowski>
CC:	
Subj:	User-level Threads Implementation

 
How user-level threads are implemented? Does user-level thread occupies
one process slot in the proc table? What I mean is if my OS can have at
the most 200 process, does it become less if I create threads inside
process. How about Kernel-level thread in this case? What is difference
between kernel-level thread and process?

We are planning the new architecture of our products. We have to evalute
different design. I can make the new software in all the three ways.
- multi-processes
- multi-threads (user-level & kerne-level)
What is the best? It is hard to make test, but if I know exactly the
kernel implementation than it will be very clear.


Regards!

_________________________________________________
Hu Rui		 (Oracle Support No.        14428)
R&D, SMS Unit    (Digital ASAP Support No. A60205)
Scandinavian Softline Technology Oy
Tulkinkuja 3   02600 ESPOO   Finland
Tel +358-9-5495 6202  Fax +358-9-512 4629
Home Tel +358-9-2789426
Internet [email protected]  http://www.softline.fi/
_________________________________________________


% ====== Internet headers and postmarks (see DECWRL::GATEWAY.DOC) ======
% Received: from mail1.digital.com by mts-gw.pa.dec.com (5.65/09May94) id AA17999; Sun, 27 Apr 97 12:34:32 -0700
% Received: from smtp.inet.fi by mail1.digital.com (5.65 EXP 4/12/95 for V3.2/1.0/WV) id AA12539; Sun, 27 Apr 1997 12:27:54 -0700
% Received: from [email protected] (port 1368 [194.197.150.5]) by smtp.inet.fi with SMTP id <18196-19673>; Sun, 27 Apr 1997 22:21:46 +0300
% Received: from localhost by armi.softline.fi; (5.65/1.1.8.2/29Apr96-0950AM) id AA04172; Sun, 27 Apr 1997 21:22:04 +0200
% Date: Sun, 27 Apr 1997 22:22:04 +0300 (EET)
% From: Hu Rui <[email protected]>
% To: Tony Swierkowski <amcucs::swierkowski>
% Subject: User-level Threads Implementation
% In-Reply-To: <[email protected]>
% Message-Id: <[email protected]>
% Organization: Scandinavian Softline Technology
% Mime-Version: 1.0
% Content-Type: TEXT/PLAIN; charset=US-ASCII

From:	AMCUCS::SWIERKOWSKI  "Tony Swierkowski"  5-MAY-1997 17:04:11.37
To:	DECPA::"[email protected]"
CC:	SWIERKOWSKI
Subj:	RE: User-level Threads Implementation

Greetings!

  Tony Swierkowski @ Digital here with some answers to your questions re:
"User-level Threads Implementation"

  You said:

>How user-level threads are implemented?...

  Per the "Guide to DECthreads" manual - section 1.2:

<--- begin paraphrase --->

A thread is a single, sequential flow of control within a program. Within 
each thread, there is a single point of execution. 

<--- end paraphrase --->

  In Digital UNIX, per the V4.0A Software Product Description:

<--- begin paraphrase --->

 Threads

 Digital UNIX provides software developers the ability to write mul-
 tithreaded programs using DECthreads. DECthreads provides a pthreads
 interface that complies with the POSIX 1003.1c semantics and DCE se-
 mantics.

<--- end paraphrase --->

  A thread itself in the generic OSF model of "processes", "tasks" and "threads"
shares it's address space, (heap, static & code) with all other threads in 
that process.  Each thread maintains it's own thread-specific context (i.e.
registers & stack) however.  While there is a one-to-one mapping of "tasks"
to "processes" and all "tasks" consist of at least one "thread", you can have
more than one "thread" per "task" (which is sort of a round-a-bout way of
saying you can have more than one "thread" per "process".  You might get
some general books (O'Reily & Associates has a good one) on the basic OSF/1
architecture that Digital UNIX is based on.  Another good reference is: "The
Design of the OSF/1 Operating System" from Prentiss-Hall.

  Is there something specific you are looking for?  I'm afraid I just don't
understand your first question and if the "Overview of Threads" in the "Guide
to DECthreads" manual doesn't make sense, please elaborate on what it is you
need.


  You then said:

>Does user-level thread occupies one process slot in the proc table?...

  See the man page for pthread, pthread_intro and proc.  This is from the man
page specifically for proc:

<--- begin paraphrase --->

  The only files that appear to exist in the /proc directory correspond to
  valid (active or zombie) processes in the system proc table. 

<--- begin paraphrase --->

  Given the above statement re: 'proc', I believe you are safe in assuming that 
a single process (with 1 -> n threads) occupies a single slot in the proc table.
Please note that a "process" has one "task" and a "task" has (at least) one 
"thread".  Please also note that the maximum number of threads per task is 
governed by "maxuthreads" (for non-priveleged "tasks").  Please also note that 
an overall system-wide limit that potentially impacts the number of threads in 
a process is defined by "threadmax".  If you try to fork a process (and there-
fore implicitly create another thread) and this system-wide limit is reached, 
you'll see:

	fork/procdup: thread_create failed. Code: 6

  Note that 'ps -m' prints all threads in a task, if the task has more than one.
Another way to look at all threads in a process is with 'dbx'.  For instance
to look at all the threads in PID 0 (i.e. the [kernel idle] process), try:

        # dbx -k /vmunix /dev/mem
        (dbx) set $pid=0
        (dbx) tlist
        (dbx) tstack

  I hesitate to make a blanket statement to definitively answer your second
question, because after much research the thread model underwent a major over-
haul in Digital UNIX V4.0 and more changes are on the way.  Don't forget that
the "task" (and not a traditional BSD-style "process") is the basic entity used
to manage system resources and a "thread" is the basic entity that is executed
(and scheduled).

  I know the details of the proc structure, task structure and thread structure
were nicely diagrammed in an old (pre V3.0 I think?) "DEC OSF/1 Internals"
course.  You can find the structure definitions for a "process". a "task" and a
"thread" at:

/usr/include/sys/proc.h
/sys/include/kern/task.h
/sys/include/kern/thread.h

  Unfortunately the diagrams showing the relationships between the process data
structures in my old "Internals" student guide are slightly obsolete and they
are copyrighted so I can't FAX them to you.  To get up-to-date "internals" type
training (using Digital UNIX V4.0[x]), you'd have to take the course offered by
Global Knowledge Network ( http://www.globalknowledge.com/ ).


  You then said:

>What I mean is if my OS can have at the most 200 process, does it become less 
>if I create threads inside process.

  I'm not sure I understand your question here.  The traditional BSD definition
of a "process" (with a single thread of execution and system resourses assigned
to it) doesn't really apply to the Carnegie-Mellon University's Mach V2.5 kernel
upon which Digital UNIX is based.  There are limits on both the maximum number
of processes that can exist on a given system configuration as well as the 
maximum number of threads (see Section 4.6.1.9 to 4.6.1.12 in the "System 
Administration" manual for details).


  You then said:

>How about Kernel-level thread in this case? What is difference
>between kernel-level thread and process?

  It's safe to assume that a "User" thread is a "lightweight" thread and a 
"Kernel" thread is a "heavyweight" thread.  The context that defines a "User"
thread is very minimal so actions like thread creation/deletion and context 
switching of "User" threads incurs the least overhead possible.  At the opposite
end of the scale is the (relatively high) overhead in maintaining/managing a 
traditional BSD "process".  Somewhere in between (in terms of overhead) lies
the overhead associated with "Kernel" threads.

  It's also worth noting that "User" threads (properly called "DECthreads" as
implemented in Digital UNIX) provides a pthreads interface that complies with 
the POSIX 1003.1c semantics and DCE semantics.  As such a set of threadsafe
libraries is an integral part of the base operationg system.  An implicit
benefit to using DECthreads (on SMP machines) is that multiple threads from 
the same or different tasks can run concurrently on different processors.  In
addition, the DECthreads library package provides threadsafe routines to help
you avoid many common pitfalls when designing multi-threaded applications, (like
"deadlocks", "priority inversion", etc.).

  In contrast, "Kernel" threads have more overhead associated with them than a 
"User" thread (but still less overhead than a traditional BSD "process").  For
the most part, "Kernel" threads are most often used in device drivers or other
systems-based programs (vs "DECthreads" used by application programmers).  It's
worth noting that "Kernel" threads do *NOT* use the threads library package 
provided by "DECthreads", but instead use their own kernel interfaces.  Like 
"User" threads, "Kernel" threads share an address space with other "Kernel" 
threads and as such can improve throughput in driver code on SMP (and on Uni-
processor) systems by overlapping input, ouput, or other (slow) device opera-
tions with computational operations.  Also, "Kernel" threads are more work 
(from the programmer's point of view) to use and manage as well.  Since you 
are not relying on a canned set of threadsafe library routines (like you do 
when using "DECthreads"), you can get into a lot of trouble quickly by not 
designing your code to avoid race conditions, deadlocks and interplay among 
"Kernel" threads that can induce a phenonmena known as "priority inversion".  
Make sure you read (and thoroughly understand!) the information on locks (both
simple and complex), hardware issues (atomicity and granualarity), etc. in the 
"Writing Device Drivers: Advanced Topics" manual as well as the details on 
"Kernel" threads in Chapter 6 & 7 of that manual.

  You then said:

>We are planning the new architecture of our products. We have to evalute
>different design. I can make the new software in all the three ways.
>- multi-processes
>- multi-threads (user-level & kerne-level)
>What is the best? It is hard to make test, but if I know exactly the
>kernel implementation than it will be very clear.

  No quick answer here either.  Multi-process vs Multi-thread (user or kernel)
schemes all have their advantages and disadvantages not the least of which is
time spent ramping up on new concepts like threads if you're used to the
traditional BSD multi-process approach.  On the otherhand, using DECthreads
(with the correspondingly smaller overhead) could have significant performance
advantages especially if hundreds of processes/threads have to be managed.

  In summary, you really need to spend some time in the "Guide to DECthreads"
manual so you feel comfortable with thread concepts in general (and "DECthreads"
specifically) as well as the "Writing Device Drivers: Advanced Topics" manual
before you can make an intelligent decision about which programming model you
want to implement.  Let me know if this helps, cheers...



						Tony Swierkowski
						Digital Equipment Corporation
						Software Partner Engineering
						Palo Alto, California
						(415) 617-3601
						"[email protected]"