|
VMS V5.2 DECwindows
Remote Application Performance
John Buford
VMS Performance and Test Systems Engineering
Report 89/11
August 1989
This report quantifies the costs and benefits of running DECwin-
dows applications on one system (the remote system) and display-
ing the user interface on another system (the workstation).
The information presented in this report is For Digital Internal
Use Only.
VMS V5.2 DECwindows
Remote Application Performance
1 Introduction
In an effort to characterize the performance of the DECwindows
user interface when applications execute on a remote system
(that is, a system other than the workstation), we measured
the user perceived response times of a number of DECwindows
applications as they executed both locally and remotely. We also
measured system resource utilization on the remote system as we
increased the number of users.
The purpose of this report is to summarize our findings. Specif-
ically, this report addresses topics such as:
o DECwindows applications and the server are CPU intensive, so
the user perceived performance improves when they execute on
a faster CPU.
o DECwindows applications are also memory intensive. Some of
this memory is sharable, so the per user memory requirements
are reduced when several users share the same global pages.
o Using the DECnet transport instead of the faster local shared
memory is not readily apparent to the typical user when it
is offset by additional system resources available to the
applications on the remote system.
In addition, this report quantifies the resources utilized by
each user on the remote system observed during this experiment.
Note that these values can and will vary from system to system,
from user to user, and from day to day. These values are meant
to provide guidelines, not rules.
2 Summary
Running VMS V5.2 DECwindows applications on a remote system
instead of on the workstation can improve user perceived perfor-
mance provided that the remote system has sufficient resources.
This is in spite of using the DECnet transport instead of the
faster local transport.
FOR DIGITAL INTERNAL USE ONLY 1
VMS V5.2 DECwindows
Remote Application Performance
Rule-of-thumb: run DECwindows applications where there are
system resources to support them.
For this experiment, each user running remote DECwindows appli-
cations used the following incremental resources on the remote
system:
0.3 VUPs
1.8 MB
11.2 page faults per second (98.3% soft)
0.3 direct I/Os per second
10.7 buffered I/Os per second
A substantial amount of memory was saved since global pages were
shared among many users on the remote system.
Even when running clients remotely, the workstation's CPU speed
and memory configuration had an effect on user perceived perfor-
mance:
o Average response time was 36% better on a VAXstation 3600
compared to a VAXstation 2000.
o The workstation used up to 6 MB of memory. When only 4 MB was
available, hard page faulting occurred and average response
time degraded 35%.
2 FOR DIGITAL INTERNAL USE ONLY
VMS V5.2 DECwindows
Remote Application Performance
3 Local Execution versus Remote Execution Performance
When DECwindows applications execute remotely, they must com-
municate with the server on the workstation via the network
transport (DECnet in this case) which is slower than the local
shared memory transport. Offsetting this cost are the additional
resources that may be available on the remote system such as
more memory, a faster CPU, bigger and faster disks, and so on.
Table 1 lists the average user perceived response time and
selected resource utilization figures observed when a workload
(described in Appendix A) was applied to a VAXstation 2000
configured with 6 MB of memory. Two test were run: one with
the applications executing locally on the workstation and the
other with the applications executing remotely on a VAX 6000-220
configured with 64 MB.
________________________________________________________________
Table_1:__Local_and_Remote_Application_Performance_Compared_____
Local Applica- Remote Ap-
Workstation_Resource______________tions______________plications_
Average User Perceived Re- 4.79 1.43
sponse Time (seconds)
CPU Utilization (%) 42.2 20.9
Total Memory Utilization (MB) 5.7 5.8
Page Fault Rate (per second) 22.4 0.3
System Disk I/O Rate (per 2.0 0.1
second)
Paging Disk I/O Rate (per 2.5 0.0
second)_________________________________________________________
FOR DIGITAL INTERNAL USE ONLY 3
VMS V5.2 DECwindows
Remote Application Performance
In this experiment, executing the applications locally over-
committed the 6 MB of memory that was available and forced the
applications and the server to share a relatively slow CPU.
Both the server and the applications executed in a constrained
environment.
When the applications executed remotely, system resource re-
quirements shifted from the workstation to the remote system, so
workstation page faulting and resultant disk I/Os dropped almost
to zero. (However, note that the workstation memory was still
fully utilized.) Both the server and the applications executed
in relatively unconstrained environments.
When comparing the average user perceived response times of
DECwindows applications executing both locally and remotely,
note that the remote applications were able to utilize the
additional resources present on the remote system and that
performance improved by a factor of 3.4. The cost of using the
slower transport was well offset by the faster CPU and reduced
page faulting due to the additional memory available.
The performance improvements observed when executing DECwin-
dows applications remotely were largely due to the fact that
more system resources were available in this experiment. It
seems obvious that the applications should execute faster on an
under-utilized VAX 6000-220 than on an over-utilized VAXstation
2000. But the point needs to be stressed that if the reverse
is true, if the remote system resources are over-utilized and
the workstation resources are not, one cannot expect to improve
performance by running the applications remotely.
RULE-OF-THUMB
Run DECwindows applications where there are system re-
sources to support them.
4 FOR DIGITAL INTERNAL USE ONLY
VMS V5.2 DECwindows
Remote Application Performance
4 Multi-user Remote Performance
In the previous section we noted that given an unconstrained
environment in which system resources are freely available,
applications which execute remotely can and do utilize those
additional resources. Yet this does not say how quickly those
resources may become exhausted as more and more users try to run
their applications remotely.
4.1 Remote System Results
Table 2 lists the average user perceived response time and
selected resource utilization figures observed when various
numbers of workstation users ran their applications remotely on
a VAX 6000-220 configured with 64 MB of memory. The average user
perceived response times listed were observed on a VAXstation
2000 configured with 6 MB.
________________________________________________________________
Table_2:__Remote_System_Performance_at_Various_User_Loads_______
2 4 6 8 10
Remote_System_Resource_________users__users__users__users__users
Average User Perceived Re- 1.45 1.49 1.53 1.55 1.71
sponse Time (seconds)
CPU Utilization (%) 23.4 46.5 70.8 90.9 112.5
Total Memory Utilization (MB) 14.1 18.0 21.6 25.3 28.7
Page Fault Rate (per second)+ 24.3 47.5 72.4 92.0 114.0
________________________________________________________________
+98.3% of the page faults observed in each experiment were soft
and did not require disk I/O. The large majority of these page
faults were global valid or demand zero, indicating that they
probably resulted from image activation.
FOR DIGITAL INTERNAL USE ONLY 5
VMS V5.2 DECwindows
Remote Application Performance
________________________________________________________________
Table 2 (Cont.): Remote System Performance at Various User
__________________Loads_________________________________________
2 4 6 8 10
Remote_System_Resource_________users__users__users__users__users
Direct I/O rate (per second) 0.8 1.4 2.0 2.5 3.1
Buffered I/O rate (per sec- 23.6 46.2 69.8 88.7 109.3
ond)____________________________________________________________
As expected, when the number of users executing applications on
a single remote system increased, the resource utilization on
that system increased. As long as none of the remote system's
resources were over-utilized, resource utilization progressed
linearly and user perceived response times remained fairly
stable.
Since resource utilization progressed linearly as long as no
resource became exhausted, it is possible to compute a per user
utilization for this specific workload. These are listed in
Table 3.
________________________________________________________________
Table_3:__Remote_System_Resource_Utilization_Per_User___________
Remote System Re-
source__________________Base______Per_User_Increment____________
CPU Utilization 0.1 0.3
(VUPs)
Total Memory Uti- 10.6 1.8
lization (MB)
Page Fault Rate (per 2.9 11.2
second)
6 FOR DIGITAL INTERNAL USE ONLY
VMS V5.2 DECwindows
Remote Application Performance
________________________________________________________________
Table_3_(Cont.):__Remote_System_Resource_Utilization_Per_User___
Remote System Re-
source__________________Base______Per_User_Increment____________
Direct I/O rate (per 0.3 0.3
second)
Buffered I/O rate 3.4 10.7
(per_second)____________________________________________________
NOTE
The figures that appear in Table 3 represents the load ap-
plied to the remote system per user, not per application.
In this experiment, each user executed up to 4 remote ap-
plications concurrently. (For more information regarding
the experiment workload, see Appendix A).
Per user memory utilization is particularly interesting since it
is possible for multiple processes to share global pages, and
so reduce the total system memory needs. The amount of memory
needed by one user to execute in an unconstrained environment
was 11.4 MB. When the memory required to add the second user is
calculated (14.1 MB - 11.4 MB = 2.7 MB) we see that the second
user added 2.7 MB to the system's total memory requirement in
order to execute in an unconstrained environment. Note that each
additional user above 2 users added only 1.8 MB. This indicates
that a substantial number of pages were being shared when there
were 3 or more users executing remote DECwindows applications on
the same system.
FOR DIGITAL INTERNAL USE ONLY 7
VMS V5.2 DECwindows
Remote Application Performance
4.2 Workstation Results
Given that shifting the application load from the workstation
to a remote system improves user perceived response time when
the remote system has additional resources that the applications
can use, what effect does the workstation now have on DECwindows
performance?
To answer this question, we compared the response times observed
on different workstations that participated in the 10 user
experiment described in the previous section.
4.2.1 Workstation CPU
Comparing the response times observed on a VAXstation 2000 and a
VAXstation 3600, Table 4 shows that workstation CPU speed is im-
portant to user perceived response times even when applications
execute remotely.
________________________________________________________________
Table 4: VAXstation 2000 and 3600 Compared While Executing
__________Remote_Applications___________________________________
VAXstation
Workstation_Resource______________VAXstation_2000____3600_______
Average User Perceived Re- 1.71 1.09
sponse Time (seconds)
CPU Utilization (%) 21.0 8.7
Total Memory Utilization (%) 94.8 42.9
Total Memory Utilization (MB) 5.7 6.9
Page Fault Rate (per second) 0.1 0.2
8 FOR DIGITAL INTERNAL USE ONLY
VMS V5.2 DECwindows
Remote Application Performance
________________________________________________________________
Table 4 (Cont.): VAXstation 2000 and 3600 Compared While Exe-
__________________cuting_Remote_Applications____________________
VAXstation
Workstation_Resource______________VAXstation_2000____3600_______
System Disk I/O Rate (per 0.1 0.1
second)
Paging Disk I/O Rate (per 0.0 0.0
second)_________________________________________________________
These figures show that the VAXstation 3600 response time was
36% better than that observed on the VAXstation 2000. The VAXs-
tation 3600 CPU is approximately 3 times faster than that of the
VAXstation 2000, yet only 21% of the VAXstation 2000 CPU time
was being utilized. So while the faster CPU improved user per-
ceived response times, the improvement was scaled to the amount
of CPU time being utilized, not total capacity.
A 36% improvement in average user perceived response time means
that the user does not have to wait as much for system response
to user input. For example, applications start up faster, dialog
boxes appear and disappear quicker, and menu pulldown appears
crisp. Bottom line: users can do their work with fewer and
shorter interruptions due to user interface processing.
4.2.2 Workstation Memory
Comparing the response times observed on VAXstation IIs con-
figured with 4, 5, 6, and 9 MB of memory, Table 5 shows that
the amount of workstation memory is important to user perceived
response times even when applications execute remotely.
FOR DIGITAL INTERNAL USE ONLY 9
VMS V5.2 DECwindows
Remote Application Performance
________________________________________________________________
Table 5: Workstation Performance with Various Amounts of Memory
__________While_Executing_Remote_Applications___________________
VAXstation_II_Resource___________4_MB_____5_MB_____6_MB_____9 MB
Average User Perceived Re- 2.53 1.91 1.90 1.87
sponse Time (seconds)
CPU Utilization (%) 27.0 27.0 28.6 27.0
Total Memory Utilization (%) 96.2 96.7 92.8 73.2
Total Memory Utilization (MB) 3.8 4.8 5.6 6.6
Page Fault Rate (per second) 13.1 2.9 0.1 0.0
System Disk I/O Rate (per 0.5 0.1 0.1 0.1
second)
Paging Disk I/O Rate (per 1.4 0.2 0.0 0.0
second)_________________________________________________________
These figures show that hard page faulting (as indicated by
the increased system and paging disk I/O) was absent on the 9
and 6 MB workstations, may have been occurring at a low level
on the 5 MB workstation, and was readily apparent in the 4 MB
workstation. Likewise, the response times on the 9, 6, and even
the 5 MB workstations were comparable, yet the 4 MB workstation
showed a 35% degradation due to contention for memory.
10 FOR DIGITAL INTERNAL USE ONLY
APPENDIX A
EXPERIMENT CONFIGURATION AND METHODS
Table 6 describes the nodes and disks which participated in the
VMS V5.2 Mixed Interconnect VAXcluster used in this experiment.
________________________________________________________________
Table_6:__VAXcluster_Configuration______________________________
Count___CPU_____________Memory______Connection__Purpose_________
1 VAX 6000-240 128 MB CI Boot and disk
server
1 VAX 6000-220 64 MB CI Execute remote
applications
4 VAXstation 6 MB NI Workstation
2000
1 VAXstation 16 MB NI Workstation
3600
1 VAXstation 9 MB NI Workstation
II/GPX
1 VAXstation 9 MB NI Workstation
II
Experiment Configuration And Methods 11
VMS V5.2 DECwindows
Remote Application Performance
________________________________________________________________
Table_6_(Cont.):__VAXcluster_Configuration______________________
Count___CPU_____________Memory______Connection__Purpose_________
1 VAXstation 6 MB NI Workstation
II
1 VAXstation 5 MB NI Workstation
II
1 VAXstation 4 MB NI Workstation
II
1 HSC70 Disk controller
2 RA82 System and user
disks
1_______RA60____________________________________Data_disk_______
All workstations were configured with a local paging disk.
Normally both 62xx nodes would be configured as boot and disk
servers, but they were not for this experiment so that it would
be possible to attribute resource utilization specifically to
DECwindows or cluster operations.
In order to achieve the maximum amount of memory sharing pos-
sible on the remote system, the following files were installed
/OPEN/HEADER_RESIDENT/SHARED in addition to those which are
installed automatically:
SYS$SYSTEM:DECw$BOOKREADER.EXE
SYS$SYSTEM:DECw$CALENDAR.EXE
SYS$SYSTEM:DECw$MAIL.EXE
SYS$SYSTEM:VUE$MASTER.EXE
SYS$LIBRARY:DECw$MAILSHR.EXE
SYS$LIBRARY:LIBRTL2.EXE
12 Experiment Configuration And Methods
VMS V5.2 DECwindows
Remote Application Performance
SYS$LIBRARY:SORTSHR.EXE
SYS$MESSAGE:DECw$DWTMSG.EXE
SYS$MESSAGE:DECw$MAIL_MESSAGES.EXE
SYS$MESSAGE:DECw$TRANSPORTMSG.EXE
SYS$MESSAGE:DECw$XLIBMSG.EXE
SYS$MESSAGE:VAXCMSG.EXE
It was necessary to increase the NCP executor characteristic MAX
LINKS on the remote system. Each remote DECwindows application
established a DECnet link with the server. The default maximum
number of links was 32. Since each user ran as many as 4 remote
applications concurrently (FileView, Calendar, Mail plus one
other), it was not possible to accommodate 10 users (40 links)
until MAX LINKS was increased. An arbitrary value of 100 was
chosen for the new MAX LINKS.
SPM V3.2 was used to collect system resource utilization statis-
tics on each node. Samples were taken every thirty (30) seconds
from the time that the last workstation began steady state op-
erations until the time that the first workstation completed
steady state operations (approximately 200 minutes).
An APEX workload was used to measure the response time that a
typical user would observe while running a variety of DECwindows
applications including FileView, Mail, Calendar, BookReader,
TPU, and DECTerm. APEX accomplishes this by simulating the
user's keyboard and pointer device input and then measuring
the elapsed time until the system's response is displayed.
The DECwindows server, Window Manager, and Session Manager al-
ways executed on the workstation. FileView was submitted to a
batch queue executing at priority 4 on either the workstation
(local case) or the VAX 6000-220 (remote case). FileView was
used to start all other applications. Mail and Calendar were
started once and ran continuously for the duration of the exper-
iment. The other applications were started and terminated on an
as needed basis.
Experiment Configuration And Methods 13
VMS V5.2 DECwindows
Remote Application Performance
Average User Perceived Response Time is the arithmetic mean of
the elapsed time that the user was waiting for the system to
respond to input. The specific operations measured were:
o BookReader operations:
BOOK_CONTENTS_TO_TOPIC
BOOK_INDEX
BOOK_INDEX_TO_TOPIC
BOOK_NEXT_TOPIC
BOOK_OPENBOOK
BOOK_START
BOOK_STOP
o Calendar operations:
CAL_DEICONIFY
CAL_ICONIFY
CAL_DISPLAY_DAY
CAL_DISPLAY_MONTH
o Mail operations:
MAIL_CLOSE
MAIL_DEICONIFY
MAIL_DIALOG_DOWN
MAIL_DIALOG_UP
MAIL_DELIVER
MAIL_ICONIFY
MAIL_READ_UP
MAIL_SEND
MAIL_SEND_QUIT
MAIL_SEND_UP
o PTF operations:
PTF_DEICONIFY
PTF_DIALOG_DOWN_1
PTF_DIALOG_DOWN_2
PTF_DIALOG_UP_1
PTF_DIALOG_UP_2
PTF_ICONIFY
14 Experiment Configuration And Methods
VMS V5.2 DECwindows
Remote Application Performance
PTF_MENU_PULLDOWN_1
PTF_MENU_PULLDOWN_2
PTF_MENU_SWEEP_1
PTF_MENU_SWEEP_2
PTF_START
PTF_STOP
PTF_SUBMENU_1
PTF_SUBMENU_2
o Session Manager operations:
SM_PAUSE
SM_RAISE
SM_UNPAUSE
o DECTerm operations:
TERM1_INFO_CTRLZ
o TPU operations:
TPU_QUIT_TO_CONFIRM
TPU_START
TPU_STOP
o FileView operations:
VUE_DCL_START
VUE_DCL_STOP
VUE_RAISE
VUE_SETDIR
o Operations common to all applications:
BUTTON_PRESS
(Note that PTF is a special purpose application designed to ex-
ercise the DECwindows user interface features. Its purpose is to
demonstrate the performance of just the user interface without
any convolution from other application-specific functions.)
Experiment Configuration And Methods 15
|
| Here is an example of how to calculate roughly the number of users that can run
remote applications on a specific system.
You need to consider what else is running on the machine and how much of the
system is left idle. To state the obvious, this varies from site-to-site and
from machine-to-machine. (See the Guide to Performance Management for the
standard VMS tools that help you see what is using resources on your system.)
You need to leave some capacity in reserve. I like to leave 20%
Not all users are active at the same time. For the purpose of example, I'll
call all the people who could legitamately login and use the system the
"subscribers" and only those people who are actually using the system at this
point in time the "users". Some subscribers are out on vacation, some are out
to lunch, some are talking on the phone, some are talking to themselves, etc.
You need to define a subscribers-to-users ratio, sometimes called the
"stretch" factor. I generally use a stretch factor of around 2.5. I've seen
factors ranging from 1.5 to 4. The number you use will depend on how "nose to
the grind stone" your the shop is.
You also need to determine how much resources each user will need on the
system. This depends on how many applications each user will be running
concurrently, what the apps are, how much CPU and memory each requires, and
whether any memory is sharable. Or you can read the report and decide whether
your shop is similar to my experiment setup, in which case you can use my
numbers: 0.3 VUPs and 1.8 megs per user.
OK, suppose you have a 5 VUP, 64 meg machine. You have reason to believe that
1.5 VUPs and 27 megs are being used for other work (Monitor shows that 30% of
the CPU is being utilized on average). You don't want to go any higher than 4
VUPs or 51.2 megs since a fully utilized system makes for choppy performance.
5.0 VUPs 64.0 megs total system resources
- 1.0 - 12.8 less 20% reserve capacity
----- ------
4.0 51.2
- 1.5 - 27.0 less resources already in use
----- ------
2.5 24.2 resources available for use
So you have 2.5 VUPs and 24.2 megs to give to remote DECwindows users.
How many users is that?
2.5 VUPS available
------------------------ = 8 users (rounding down)
0.3 VUPs per user needed
24.2 megs available
------------------------ = 13 users (rounding down)
1.8 megs per user needed
According to these numbers, your memory is able to support 13 active users,
but you only have enough CPU to support 8. How many subscribers?
8 active users X 2.5 stretch factor = 20 subscribers
So it looks like this hypothetical machine has enough excess capacity to
support 20 remote DECwindows subscribers IF 20% reserve capacity is enough and
not too much, IF the Monitor data accurately reflects the current workload, IF
the users put a load on the remote system similar to that of my experiments,
and IF you estimated the users-to-subscribers ratio correctly.
Your mileage may vary, offer void where prohibited...
|