[Search for users]
[Overall Top Noters]
[List of all Conferences]
[Download this site]
| Title: | DEC TCP/IP Services for OpenVMS |
| Notice: | Note 2-SSB Kits, 3-FT Kits, 4-Patch Info, 7-QAR System |
| Moderator: | ucxaxp.ucx.lkg.dec.com::TIBBERT |
|
| Created: | Thu Nov 17 1994 |
| Last Modified: | Fri Jun 06 1997 |
| Last Successful Update: | Fri Jun 06 1997 |
| Number of topics: | 5568 |
| Total number of notes: | 21492 |
5457.0. "An SMTP README for those supportinG UCX SMTP" by UCXAXP::ZIELONKO () Wed Apr 23 1997 06:29
Hi,
For a long time the lack of complete UCX SMTP documentation suitable for someone
who really wants to know what's going on has been bugging me. To this end I have
finally bitten the bullet and written up a brain dump on UCX SMTP - an
SMTP_README. Alot of this stuff is mentioned in other notes here and there and
every where. It contains "internals" type stuff, problem diagnopsisi type stuff,
configuration information and tips. It is intended mostlly for people solving
UCX SMTP problems.
It got to be very long - over 3000 lines so you may want to print it out and
read it in bits.
As changes are made to the code this file will be updated.
I have passed this information on to the UCX Doc's team to reduce and include in
the UCX SMTP doc's.
Karol
And awaaaaaaayy we go-o-o-o-!
Date: April, 14 1997
Author: Karol Zielonko
UCX-Version: V4.1 ECO 4, V4.2 SSB
------------
Contents
------------
Introduction
Part 1: SMTP Overview
What is SMTP?
What is MIME?
SMTP Protocol Overview
SMTP Protocol Overview
SMTP Client commands
SMTP Server Replies
SMTP Message Format
SMTP Mail Addresses
Local-part Notes
Domain Notes
Miscellaneous Addressing Topics
Source Routing
MX Record Overview
MX Example 1
MX Example 2
MX Example 3
Part 2: UCX SMTP
UCX SMTP's interface with VMS mail environment
Format of VMS mail message
Storing SMTP messages in VMS mail: Big square peg and small round hole.
UCX SMTP and callable VMS mail
Callable VMS mail overview
UCX SMTP's use of mail$send routines of callable mail
VMS mail Foreign Transport/MAIL$PROTOCOL Interface
VMS mail Foreign Transport
MAIL$PROTOCOL Interface
UCX SMTP Control File
CF Filenames - what they mean.
Control file layout
Address Area
Header Area
Text Area
Components of UCX SMTP
The UCX SMTP symbiont/queues
The UCX SMTP Receiver
The UCX SMTP MAIL$PROTOCOL code
The UCX SMTP control file code
UCX MX Lookup
Displaying MX records with UCX
Local vs. non-local SMTP mail
The SMTP Alternate Gateway/Zone
How UCX decides where to send non-local mail
How the symbiont sends a mail using the UCX MX Lookup gateway list
How the symbiont sends a mail through the alternate gateway
Further enhancements in worklist for mail destined for alternate gateway
Forwarding to an SMTP address
UCX SMTP and non-SMTP mail
SMTPfication of non-SMTP addresses
Example SMTPfications
When is SMTPfication necessary?
Mail from non-SMTP to SMTP
An example
SMTPfication as simple, generic mechanism for gatewaying mail
Mail from SMTP to non-SMTP
Another SMTP to non-SMTP case
Miscellaneous SMTP/non-SMTP issues
SMTP/non-SMTP worklist items
Undeliverable Mail
Configuring UCX SMTP
UCX SMTP Common directory
Local Aliases File
UCX SMTP Configuration database entry (UCX SET CONFIG SMTP)
Postmaster field
UCX SMTP configuration logicals
Miscellaneous VMS mail notes
VMS mail native support for SMTP addresses
Setting forwarding to an SMTP address
Miscellaneous SET FORWARD/USER tricks
The UCX SMTP cent sign hack
User use of the cent sign hack
UCX SMTP mailing lists
VMS Majordomo
Miscellaneous Stuff
Queue watcher program
Terms
--------------------------------------------------------------------------------
Introduction
--------------------------------------------------------------------------------
This document is for system managers who will be supporting VMS systems running
the UCX SMTP product. It is divided into two parts. The first part gives a
general overview of SMTP and the second part discusses UCX SMTP specifics. If
you are already familiar with SMTP and Internet mail you may want to skim or
skip the first part entirely.
This document mentions "UCX worklist items". These are meant to inform and not
to be taken as a commitment to do any particular item according to any time
schedule.
--------------------------------------------------------------------------------
Part 1: SMTP Overview
--------------------------------------------------------------------------------
This part gives a general overview of SMTP and Internet mail. It provides
information that is not specific to any operating system or implementation of
SMTP. It is a summary, a primer, an overview. It does not cover every detail.
Those who wish to know more are urged to look at the RFCs.
----------------------------------------
What is SMTP?
----------------------------------------
SMTP (for Simple Mail Transfer Protocol) is the Internet's mail transfer
protocol. The protocol itself is specified in RFC 821. The format for the text
of a mail message carried by an SMTP MTA is specified in RFC822. These RFCs are
amended by RFC 1123 Chapter 5.
----------------------------------------
What is MIME?
----------------------------------------
MIME (for Multipurpose Internet Mail Extensions) specifies an extension to RFC
822 to allow for mail message content other than simple 7 bit ASCII. It is
specified in RFCs 1521, 1522 and 1523. A MIME message is by definition RFC 822
compliant.
UCX SMTP currently has no special handling for MIME content and doesn't need to
since UCX SMTP is an MTA and creation/interpretation of mail message content is
not the job of an MTA. Therefore, discussion of MIME is outside the scope of
this document.
----------------------------------------
SMTP Protocol Overview
----------------------------------------
SMTP is a store and forward mail protocol which means that mail is routed** from
the originating host through 0 or more intermediate hosts before it reaches its
final destination host. Contrast this with DECnet's MAIL11 protocol where mail
is sent directly from the originating node to the destination node.
** This document uses the terms "route" and "relay" interchangably to refer to
SMTP application layer transfer of mail messages from one host to the next to
the next, etc. The term routing does not denote IP layer routing.
Some advantages of a store and forward mail system are:
Mail can be sent between hosts that cannot connect directly to one another,
for example when the originating and destination hosts are on opposite sides
of a firewall.
If the destination host is not available when mail is sent to it the mail can
be queued up on one of the intermediate hosts (or the originating host). The
MTA on the host where the mail is queued up will retry periodically and when
the destination host is available again the mail will go through.
Conventionally MTA's will continue to try to send a mail for some locally
configurable period of time after which the mail is bounced as undeliverable.
One advantage of a non store and forward mail system is that the sending user
knows immediately whether or not a mail message has been delivered to the
destination. This feature however can also be provided by SMTP if both the
originating and destination hosts support the Return-Receipt: message header.
Intermediate hosts need not support Return-Receipt. Currently UCX SMTP doesn't
support Return-Receipt but it is in the worklist.
The SMTP protocol is a set of simple ASCII commands and replies. A TCP
connection is requested by the SMTP client and accepted by the SMTP server
listening on port 25. The SMTP client sends the commands and the server replies
to them. The SMTP protocol uses TCP as its transport protocol and an SMTP server
listens on port 25.
--------------------
SMTP Client commands
--------------------
There are five commands which are essential for the transmission of SMTP mail.
They are "HELO", "MAIL FROM", "RCPT TO", "DATA" and "QUIT". Command descriptions
follow. The commands appear in upper case and the command parameters (where
applicable) in lower case.
HELO domain
domain The name of the sending (ie. client) host.
With the HELO command the client identifies itself to the server.
MAIL FROM:<reverse-path>
<reverse-path> The mail address to which the mail should be bounced if
it cannot be delivered. When the mail is delivered to
the destination host mailbox the address in the
reverse-path becomes the Return-Path mail header.
The angle brackets are required.
With the MAIL FROM command the client tells the server where to bounce the
mail. Only one MAIL FROM command per mail message is allowed. It *is* allowed
for the MAIL FROM to be "null". That is, "MAIL FROM:<>". This is required
when a mail message is bounced to prevent bounced mail about bounced mail.
That is, when an SMTP MTA tries to deliver a mail message and can't do so the
message must be "bounced" or sent back to the sender. The MTA that bounces
the mail creates a bounced mail message and the MAIL FROM address used in the
SMTP protocol dialog when sending this bounced mail is MAIL FROM:<>. This
indicates to any SMTP MTA that receives the bounced mail message that it
canot be further bounced. How that SMTP MTA deals with the situation where it
has a message with a null return path (ie. a null MAIL FROM address) is up to
the SMTP MTA.
RCPT TO:<forward-path>
<forward-path> A mail address to which the mail message should be
delivered.
The angle brackets are required.
With the RCPT TO commands the client tells the server where to deliver the
mail. For cases where mail is to be sent to multiple recipients there must be
one RCPT TO command sent for each recipient. That is, the RCPT TO command
cannot contain multiple addresses.
Note that although the mail message text itself may contain a To: header (or
other destination type header, eg Resent-To:) it is the address(es) in the
RCPT TO command(s) that the SMTP MTA sends the mail to.
DATA
The DATA command signals the end of the RCPT TO commands and tells the server
to prepare to receive the mail message itself.
When the client sends the DATA command the server returns a 354 Intermediate
reply (see "SMTP Server Replies"). At this point the client sends the lines
of the mail message. The end of each line of text is indicated by a <CRLF>.
The client indicates the end of the mail message by sending a '.' on a line
by itself (<CRLF>.<CRLF>). Note that the '.' and the following <CRLF> are
not part of the mail message itself. They simply constitute an SMTP protocol
"end of file" marker.
Once the SMTP server receives the <CRLF>.<CRLF> indicating the end of the
mail message it must acknowledge the receipt of the message and accept
responsibility for its delivery. It does so by returning a "250 OK" reply.
When the client receives this reply it knows that responsibility for delivery
of the mail now belongs to the server. At this point the client is free to
send another MAIL FROM command for the next mail message or to send a QUIT
command.
QUIT
With the QUIT command the client indicates it has no more commands to send.
On receipt of this command the server must send an OK reply and close the
connection.
The other commands described in RFC 821 are either non-essential or
inconsequential to our discussion of SMTP here.
Miscellaneous Protocol Notes:
- The SMTP client commands must end in <CRLF>.
- The SMTP server replies must end in <CRLF>.
- Text lines within the mail message must end in <CRLF>.
- Each text line in the mail message has a maximum length of 1000 characters
including the <CRLF>.
- The client must send the commands in this order - HELO, MAIL FROM, RCPT TO
and DATA.
- The addresses in the MAIL FROM and RCPT TO commands make up what is often
refered to as the "envelope". As stated above the addresses in these commands
hold the real destination and return (ie. bounce) addresses for the mail. An
SMTP MTA must save this information outside of the message itself.
It is not required that the addresses from the envelope match any addresses
in the headers sent with the mail. Take the case of a mailing list where the
RFC To: header is the name of the list itself but the RCPT TO addresses are
the names of the individual recipients. The From: field is the username of
the sender and the Reply-To field is the list itself. But the MAIL FROM
address is the username of the "owner" of the mailing list so that if any of
the recipients bounce the owner of the list (not the sender or the list
itself) receives the bounced mail.
- The Return-Path header is special in one respect that bears repeating. No
Return-Path header is ever sent via an SMTP protocol dialog in the SMTP
message headers. The Return-Path header is manually added into the list of
headers when the message reaches its final destination. When depositing an
SMTP mail message into the destination mailbox the destination host SMTP MTA
manually constructs a Return-Path header using the address of the MAIL FROM
command and puts that header into the mail message's headers. In the case
where a message is being routed by an SMTP MTA to another host the MAIL FROM
address is propagated as it was received when the mail came in.
--------------------
SMTP Server Replies
--------------------
Each SMTP command elicits a reply from the SMTP server. The format for the SMTP
server reply is
"nnn text"
where
"nnn" is a string of three ASCII digits known as the reply code.
"text" is free form ASCII text for the convenience of humans. The contents of
text are not specified by any RFC.
The individual digits in the reply code each have their own significance and are
interpreted separately. Only the first digit must be interpreted by the client.
It has four legal values - 2, 3, 4 and 5. The meanings of the first digit of the
reply code follow:
Value Meaning Comments
2 Postive Completion Requested action completed successfully
3 Positive Intermediate Requested action accepted. More info
needed from client. **
4 Transient Negative Completion Request failed due to temporary error
condition. Eg. Disk full. Client should
try again later.
5 Permanent Negative Completion Request failed due to permanent error
condition. Eg. No such user as the user
specified in the RCPT TO address. Client
should bounce the mail message.
** value 3 is a special case. It is sent only for the DATA command. The
"additional info" required is the text of the mail message itself.
Reply codes are explained in detail in Appendix E of RFC 821.
----------------------------------------
SMTP Message Format
----------------------------------------
RFC 822 specifies the format for an SMTP mail message. (ie. the format of the
data sent after the SMTP DATA client command). In a nutshell a mail message
consists of mail headers followed by a blank line followed by the mail body.
(The body is the text of the message.)
Mail headers are fixed format text lines which contain addressing, control,
diagnostic, identification and other miscellaneous information about the mail
message. All header lines must conform to a general header syntax specified by
RFC 822. Each header has a field name followed by a ":" followed by a field body
followed by a <CRLF>. For example with the header
Date: Fri, 28 Mar 1997 09:46:15 -0500<CRLF>
The field name is "Date". The field body is "Fri, 28 Mar 1997 09:46:15 -0500".
If the body of the header is to be extended accross multiple lines of text a
<CRLF> may be inserted into the field body followed by at least one "linear
white space" (TAB, space) character. An example of a multi line header is:
Received: by mysystem.someplace.dec.com with Microsoft Mail<CRLF>
id <[email protected]>; Thu, 27 Mar 1997<CRLF>
08:37:20 -0500<CRLF>
Most header bodies have additional syntactic structure imposed on them by RFC
822. The body of the Received: header above is a good example of such a complex
syntax.
Note that the <CRLF> at the end of lines is required by the SMTP protocol.
However, no requirement is made on how the mail is actually *stored* in the
local mailstore on the destination host. If a mail system stores the mail with
<LF> as the end of line character it must change the <LF>'s to <CRLF>'s when
transmitting the mail via SMTP.
----------------------------------------
SMTP Mail Addresses
----------------------------------------
As we have seen SMTP addresses appear in some of the SMTP client commands. They
also appear in many of the SMTP message headers. Understanding SMTP addressing
is essential for the diagnosis and resolution of mail problems.
RFC 822 specifies a comprehensive sytnax for SMTP addresses. An SMTP address
consists of two parts - the local-part and the domain. Syntactically they are
separated by an at sign (@). Ie. local-part@domain.
local-part
The local-part identifies a local delivery point for a mail message. It is
sometimes refered to as the user, username or left hand side. It is also
refered to by RFC 822 as the "domain-dependent string" since its
significance depends on what is understood by the MTA serving the domain
specified in the domain of the address. It should be interpreted only by
the final destination host not by any intermediate hosts through which the
mail is routed.
domain
The domain is sometimes refered to as the right hand side or the hostname.
It specifies either a hostname (or an IP address) where the recipient's
mailbox resides or an MX destination domain where the recipients mailbox
is served. (See "MX Record Overview".) This part of the address is used by
the originating host and any intermediate hosts to decide where to route
the mail.
--------------------
Local-part Notes
--------------------
Quoted and unquoted local parts:
The syntax allowed for a local-part allows a local part to take one of two
forms - a quoted string or an unquoted string. A quoted local part begins and
ends with a quotation mark (aka. double quote (") ). A local part that is
not quoted does not. Here are some example addresses with unquoted local
parts:
[email protected]
[email protected]
Here are some example addresses with quoted local parts:
"SUMNOD::BORIS"@xyzcorp.com
"XYZMTS::MRGATE::\"XYZMTS::MRGATE::NATASHA\""@xyzcorp.com
Rules for quoting the local part:
RFC 822 defines a set of characters which it calls "special characters". They
are:
()<>@,.;:\"[]
If the local part needs to contain any one of these characters except a
dot (.) then it MUST be quoted. (The dot character is a special character
but it is allowed in an unquoted local part. This syntactical exception is
what allows an address like [email protected].)
For example, if the SMTP MTA on xyzcorp.com wants to create an address
with the local part being a DECnet mail address SUMNOD::BORIS it must
quote the local part. Eg:
"SUMNOD::BORIS"@xyzcorp.com
because the local part contains special characters - namely colons.
Nested double quotes in local part:
Say the SMTP MTA on xyzcorp.com needed to make an address with the local part
being an MTS address of
XYZMTS::MRGATE::"XYZMTS::MRGATE::NATASHA"
Not only does this address contain the colons but it also contains double
quotes. This raises the issue of how double quotes are nested. RFC 822
specifies a 'C' like "escape" mechanism to allow for nested double quotes in
quoted strings. To nest a double quote in a quoted string the nested double
quote is preceded with a backslash character (\). So, given the desired local
part above, the address created by the MTA on xyzcorp.com would be:
"XYZMTS::MRGATE::\"XYZMTS::MRGATE::NATASHA\""@xyzcorp.com
UCX SMTP in fact does create such complicated address as the ones in this
section. The reasons for this are discussed in "UCX SMTP and non-SMTP mail".
--------------------
Domain Notes
--------------------
Syntactically the domain part takes one of the following two forms:
1) A DNS type hierarchical name as in "something.xyzcorp.com" in this
address:
[email protected]
2) An IP address as in "[1.2.3.4]" in this address:
jones@[1.2.3.4]
The use of IP addresses in SMTP mail addresses is allowed but strongly
discouraged. The square brackets are required.
--------------------
Miscellaneous Addressing Topics
--------------------
---------------
Source Routing
---------------
SMTP allows (although discourages) two ways for the sending mail user to specify
a particalur route for their mail to take as it goes from the originating host
to the destination host. This capability is called source routing. One syntax is
provided for explicitly in RFC 821. The other is an alternative that is a
defacto standard.
RFC 821 source routing:
RFC 821 and 822 provide for a syntax for source routing:
@a,@b:username@c
where a, b, and c are hosts. With such an address the final destination for
the mail is host c but it first must go to host a and then to host b.
Percent sign hack source routing:
A defacto standard for source routing is also generally supported by most
SMTP MTA's. It is called the %-hack. Its syntax is:
username%c%b@a
where a, b, and c are hosts. With such an address the final destination for
the mail is host c but it first must go to host a and then to host b. Ie.
this example accomplishes the exact same thing as the first.
Both of these syntaxes are supported by UCX SMTP.
----------------------------------------
MX Record Overview
----------------------------------------
SMTP MTA's make use of DNS MX records to assist them in the routing of mail
messages. An MX record tells an MTA where to route mail for a particular
destination domain. The DNS maintains MX records but SMTP MTA's *use* them. An
MX record has the following fields:
destination domain
This field matches the domain part of the address to which a mail is
going. It is essentially the "key" field of the MX record. For
example, if mail is to be sent to [email protected] an MX lookup is
done on the destination domain "xyzcorp.com".
Multiple MX records for the same destination domain are allowed.
Taking the key analogy further this is like saying the destination
domain field allows duplicate keys.
gateway hostname
The name of the host to which mail sent to the destination domain
should be routed.
preference
Prioritizes multiple MX records for the same destination domain. The
lower the preference value the higher the priority for the MX record.
That is, lower preference MX records are tried before higher
preference MX records.
It is allowed to have multiple MX records for the same destination
domain with the same preference.
Some advantages of multiple MX records for the same destination domain are:
1) They enable load balancing between mail routers. This is the case where
there are multiple MX records for the same destination domain with the same
preference.
2) They allow one mail router to be taken down with another left up so that
mail can continue to go through.
3) They provide MX based routes for mail internal and external to a firewall.
--------------------
MX Example 1
--------------------
For example, imagine the following three MX records for the destination domain
"xyzcorp.com":
Destination domain: Gateway: Preference:
xyzcorp.com mailhub1.xyzcorp.com 10
xyzcorp.com mailhub2.xyzcorp.com 10
xyzcorp.com otherhost.xyzcorp.com 20
Given these MX records, any SMTP MTA trying to send mail to the destination
domain xyzcorp.com using MX records should first try to send the mail to either
mailhub1.xyzcorp.com or mailhub2.xyzcorp.com. If that fails the MTA should try
the other of the two "mailhub" hosts. If that too fails it should try
otherhost.xyzcorp.com. If that fails the MTA should requeue the mail message in
its internal queueing mechanism and retry later.
The MX records in this example provide for the advantages of multiple MX records
listed above as number 1 and 2:
1) The fact that mailhub1 and mailhub2 are at the same MX preference provides
mail load balancing between the two hosts. This is because the DNS returns MX
records in a different order each time so that records with the same
preference are not always seen in the same order by the SMTP MTA.
2) Since there are multiple MX records for the domain xyzcorp.com, mail sent
to xyzcorp.com can still be relayed from a sending host to one of the MTA's
within the xyzcorp network as long as any of the xyzcorp hosts are alive.
Mailhub1 can be taken down for maintenance and mail will still continue to
flow through mailhub2.
Given the example MX records it is assumed that at least the lowest preference
mail hubs for the xyzcorp.com domain (ie. mailhub1 and mailhub2) are configured
internally to know how to deliver mail to each user in the xyzcorp.com domain.
For example user [email protected] mail reside on host zzz.xyzcorp.com.
Something on the mailhub system has to exist so that it knows to send mail to
[email protected] to jones on the zzz.xyzcorp.com host. It could be that
xyzcorp's network is small enough so that this can be accomplished with a simple
alias file for all corporate users. (That would be a SET FORWARD/USER=JONES
SMTP%"""[email protected]""" for a VMS system.) Otherwise more complex
mechanisms will need to be configured. These mechanisms depend on the SMTP
implementation running on the hosts and so are beyond the scope of this
document. (The creation of additional MX records as in MX Example 3 is useful in
getting mail from SMTP mail hubs to internal hosts on a large network.)
--------------------
MX Example 2
--------------------
What if both mailhub1 and mailhub2 were down? In this case remote MTA's will try
and fail to send the mail to the mailhub hosts and will succeed in sending the
mail to otherhost.xyzcorp.com. One of two possible scenarioes will occur
depending on how otherhost is configured:
1) If otherhost is configured internally to know how to deliver mail to each
user in the xyzcorp.com domain (as are mailhub1 and mailhub2) it will
deliver mail using this mechanism without further use of MX records.
2) If otherhost is not configured to know how to deliver mail to each user in
the xyzcorp.com domain it will act merely as a holding place for all mail
sent to xyzcorp.com. It will continually retry to route the mail to mailhub1
and mailhub2. As long as both of these systems are down it will continue to
requeue and retry the mail. Eventually either one of the mailhub hosts will
become available again and the mail will be routed there or the retry time
limit on otherhost will expire and the mail will bounce.
--------------------
MX Example 3
--------------------
In this example we switch gears from looking at imaginary xyzcorp's MX records
to Digital's own MX record setup. Here we illustrate the third advantage listed
of multiple MX records for the same destination domain which has to do with
firewall issues. We also illustrate a use of MX records that allows a setup
where corporate mailhubs do not have to know where each individual user lives
thus avoiding the problems mentioned at the end of Example 1.
Here are the MX records for the SMTP destination domain ucx.lkg.dec.com:
Destination domain: Gateway: Preference:
ucx.lkg.dec.com ucxaxp.ucx.lkg.dec.com 50
ucx.lkg.dec.com mail2.digital.com 100
ucx.lkg.dec.com mail1.digital.com 100
ucx.lkg.dec.com crl.dec.com 200
SMTP mail internal to corporate firewall:
Say user boris on host mrbig.ucx.lkg.dec.com sends mail to
[email protected]. Assuming the SMTP MTA on host mrbig uses MX records it
will first try to route the mail to ucxaxp.ucx.lkg.dec.com since that system
is in sole possesion of the lowest preference (ie. highest priority) MX
record for the destination domain of ucx.lkg.dec.com. Since both systems are
within the corporate firewall the smtp connection from mrbig to ucxaxp can be
made and the mail is transfered to ucxaxp. Once there it is delivered as
local mail to user jones.
Note: ucxaxp is a VMS UCX system serving mail for the domain ucx.lkg.dec.com.
See "Configuring UCX SMTP" and "Local vs. non-local SMTP mail" for more
information on how UCX SMTP can be configured to accept mail as local for
a domain that isn't the same as the UCX hostname.)
SMTP mail coming from outside corporate firewall:
Say user bullwinkle on host moose.wassamadda-u.edu sends mail to
[email protected]. Assuming the SMTP MTA on host moose uses MX records it
will first try to route the mail to ucxaxp.ucx.lkg.dec.com just as
mrbig.ucx.lkg.dec.com does. However, in this case connection to ucxaxp fails
because the two hosts are on opposite sides of the Digital firewall. Once the
connection to ucxaxp fails the next MX record is tried. In this case the next
MX record will be for either mail2.digital.com or mail1.digital.com since
they both have the same preference level. Which one is tried depends on which
MX record is returned first from DNS. The connection works because mail1 and
mail2 can accept SMTP connections from outside the firewall. At this point
say the connection is to mail1.digital.com and that the mail is transfered
successfully. Once the transfer is complete the SMTP MTA on mail1.digital.com
repeats the MX lookup process from the start. It sees that the lowest
preference MX record for mail addressed to the domain ucx.lkg.dec.com is for
ucxaxp.ucx.lkg.dec.com and so tries to connect to that host. This time the
direct connaction to ucxaxp works because mail1 can connect to hosts within
the firewall. So, the mail is transfered to ucxaxp and delivered locally to
uses jones as with the previous example.
Note: This example assumes that moose.wassamadda-u.edu is not inside its own
wassamadda-u firewall. However if it is inside a firewall then MX records
alone won't work since moose.wassamadda-u.edu won't be able to connect to
any hosts in the MX records for ucx.lkg.dec.com. In this case the moose
host must be configured to send mail destined to the world outside its
firewall to some host similar to mail1.digital.com that *can* connect
directly to the outside world. How this is done on host moose depends on
the SMTP implementation running there. See "Configuring UCX SMTP" and "How
UCX decides where to send non-local mail" for information on how to do
this for UCX SMTP.
These two examples show how MX records can be used to create different mail
routes depending on whether mail is within or accross the boundaries of
firewalls. It also demonstrates one way that a mail hub straddling a firewall
can be used to accept inbound mail from outside the firewall and route it to the
host inside the firewall that knows how to deliever it without the mailhub
knowing anything about individual users.
MX Record Notes:
- The fields of MX records and handling of them are not case sensitive.
- An MX destination domain may be a DNS host name but it is not requried to
be.
- For information on how to display MX records using UCX see "Displaying MX
records with UCX".
--------------------------------------------------------------------------------
Part 2: UCX SMTP
--------------------------------------------------------------------------------
This section is intended for
- The VMS system manager who is tasked with maintaining a system running UCX
SMTP.
- Anyone tasked with diagnosing problems with UCX SMTP.
The types of information included are:
- VMS implementation details and UCX SMTP implementation details where they
may help problem solvers diagnose problems through understanding of
underlying operations.
- Handy hints and tricks
- Configuration suggestions
- Examples to illustrate points
----------------------------------------
UCX SMTP's interface with VMS mail environment
----------------------------------------
As a VMS based SMTP MTA UCX SMTP has an interface with the VMS mail environment.
It uses this interface to:
- Deliver a mail message to VMS mail coming from SMTP
- Receive a mail message from VMS mail to send out via SMTP
It uses two distinct VMS mail interfaces to accomplish these tasks. To deliver
mail from SMTP to VMS mail it uses what is called "callable VMS mail". To accept
a mail message from VMS mail to send out via SMTP UCX SMTP implements a VMS mail
"foreign transport" using the the undocumented MAIL$PROTOCOL interface. This
section discusses these two interfaces and details about UCX SMTP's use of them
and VMS mail.
--------------------
Format of VMS mail message
--------------------
Just as an SMTP messages has a format so does a VMS mail message. Unlike SMTP
messages which have a rich set of headers VMS mail messages have only a small
set of headers. They are From, To, Subj and CC. As with SMTP the headers of a
VMS mail message are separated from the body of the VMS mail message by a single
blank line. In the case of an SMTP mail message delivered to VMS mail the body
of the VMS mail message is the entire text of the SMTP mail message - the SMTP
headers followed by blank line followed by the SMTP message body.**
** This assumes that UCX SMTP has been configured with /OPTION=TOP_HEADERS. See
"Configuring UCX SMTP".
---------------
Storing SMTP messages in VMS mail: Big square peg and small round hole.
---------------
Because of VMS mail's lack of support for the rich set of headers provided for
by SMTP messages there is bound to be a "loss of fidelity" when an SMTP message
is passed to VMS mail. The biggest loss of fidelity is with the "originator"
headers from the SMTP mail message.
SMTP messages can have many different "originator" type header fields - From,
Reply-To, Return-Path - each with a different purpose. However VMS mail supports
only one originator type header, the From header, so when an SMTP message is
passed to VMS mail UCX SMTP has to make a choice of which of the SMTP originator
fields to pass to VMS mail's From header. UCX uses a priority list of all the
possible SMTP originator headers to do this. When deciding which of the
originator headers to use in the VMS mail From line UCX SMTP starts at the top
of the priority list scanning the SMTP message headers for the particular header
from the priority list. If the header from the priority list is is found in the
message that header is used in the VMS mail From line. If not the scan goes to
the next header from the priority list. The priority list from first choice to
last choice is:
Resent-Reply-To
Resent-From
Reply-To
From
Resent-Sender
Sender
Return-Path
So, for example, if an SMTP mail message had a Reply-To, a From and a
Return-Path header the VMS mail from line would be taken from the Reply-To
header since Reply-To appears first in the priority list.
--------------------
UCX SMTP and callable VMS mail
--------------------
---------------
Callable VMS mail overview
---------------
VMS provides a suite of routines collectively refered to as "callable mail". The
routines are divided into four groups. There is one group of routines for
manipulating mail files, one for manipulating existing messages stored in
folders within a particular mailfile, one for getting and setting user profile
information and one for delivering a new message. Each of the four groups of
routines has a routine name prefix which all of the routines in the group start
with. They are:
Prefix: Purpose:
mail$mailfile_... Manipulate mailfiles
mail$message_... Manipulate existing messages in folders
mail$user_... Get/set user profile information
mail$send_... Deliver new mail to VMS mail
To deliver mail from the SMTP world into the VMS mail world UCX SMTP uses the
mail$send routines. UCX SMTP uses the mail$user routines to see the forwarding
string of a local user. (For more information on this see "Forwarding to an SMTP
address".)
Callable VMS mail is documented fully in the VMS Utility Routines Reference
Manual.
---------------
UCX SMTP's use of mail$send routines of callable mail
---------------
The mail$send routines provide a series of calls to pass a mail message to VMS
mail. The mail$send routines of interest are:
Routine: Use:
mail$send_add_address Passes the address of a recipient for the VMS
mail message.
mail$send_add_attribute Passes the text of the four VMS mail header
lines for the mail message. There is one
separate input item for each of the four
header lines - From, To, Subj, CC. Note that the
text of the To line as passed in the call to
mail$send_add_attribute is *not* the same as the
address passed to mail$send_add_address. The
text of the To line is uninterpreted by VMS
mail.
mail$send_add_bodypart Passes a single line of mail text to VMS mail.
Called in a loop.
mail$send_message Tells VMS callable mail that text of message is
complete and that it should now deliver the
message.
Example:
Say UCX system ucxsys.xyzcorp.com is delivering a mail from
[email protected] sent to [email protected]. To pass the mail to
VMS mail to be delivered UCX SMTP on ucxsys performs the following VMS
callable mail calls:
- Calls mail$send_add_address passing the address "boris".
- Calls mail$send_add_attribute passing the the text of the SMTP
message's To, CC (if any), and Subject field verbatim to VMS mail for the
text of the respective VMS mail header lines.
- Decides which originator field to use (see "Storing SMTP messages in
VMS Mail") for the VMS mail From line. Puts the SMTP% jacket around the
address to make it legal for a VMS mail reply and passes it to
mail$send_add_attribute as the text of the VMS mail from line.
- Calls mail$send_add_bodypart for each text line of the SMTP message
which includes the RFC 822 message headers, a blank line and the message
text.
- Calls mail$send_message to tell VMS mail to deliver the message to the
user.
--------------------
VMS mail Foreign Transport/MAIL$PROTOCOL Interface
--------------------
Up to this point we've discussed how SMTP mail coming into UCX SMTP is delivered
to VMS mail but not how mail is composed on VMS mail and sent out via UCX SMTP.
The mechanism to do this is called a "foreign transport". Foreign transports are
implemented using the MAIL$PROTOCOL interface.
---------------
VMS mail Foreign Transport
---------------
The VMS mail foreign transport mechanism allows mail to be sent over a mail
protocol other than Mail11. Foreign transport addresses begin with a special
prefix to identify them as such to the VMS mail address parser. It is a string,
which is the identifier for the foreign transport, followed by a percent sign
(%). After this comes the mail address itself in the syntax of the foreign
transport. The identifier for the UCX SMTP foreign transport is the string
"SMTP". So to send a mail via UCX SMTP from VMS mail the address entered in the
VMS mail To: must start with SMTP%. Because of VMS mail address parsing issues
there is also an extra set of double quotes needed** around an SMTP address so
the whole thing looks like:
MAIL> send
To: SMTP%"[email protected]"
** See "VMS mail address parsing notes"
---------------
MAIL$PROTOCOL Interface
---------------
When VMS mail sees a mail addressed to a foreign transport it needs to pass the
address to the foreign transport mechanism along with the mail message so that
the foreign transport can send it out. VMS mail does this using the
MAIL$PROTOCOL interface which is an undocumented, unsupported calling interface
between VMS mail and a foreign transport. It is implemented as a routine called
MAIL$PROTOCOL which resides in an image provided as part of the foreign
transport mechanism. When VMS mail sees a mail message to a foreign transport it
goes through a search process to locate the MAIL$PROTOCOL routine for the
foreign transport and then goes through a sequence of calls to the MAIL$PROTOCOL
routine to pass the address(es) and VMS header lines and message text along to
the foreign transport's MAIL$PROTOCOL routine.
MAIL$PROTOCOL search:
When VMS mail sees a mail message to a foreign transport it needs to find the
foreign transport's MAIL$PROTOCOL routine. As a first resort it looks for an
image file SYS$LIBRARY:xxx_MAILSHR.EXE where "xxx" is the foreign transport
string. Eg. SYS$LIBRARY:SMTP_MAILSHR.EXE. If it finds such an image it looks
for a routine named MAIL$PROTOCOL in the image. If it finds it then the
search is over.
If no xxx_MAILSHR file is found or it doesn't contain a MAIL$PROTOCOL routine
VMS mail will attempt to translate a logical MAIL$PROTOCOL_xxx where "xxx" is
the foreign transport string. If it finds one then the equivalence name of
the logical must be the filename (directory defaults to SYS$LIBRARY, file
extension defaults to .EXE) of the image file where the MAIL$PROTOCOL routine
is to be found. UCX SMTP uses this mechanism to point to its MAIL$PROTOCOL
routine in SYS$LIBRARY:UCX$SMTP_MAILSHR.EXE.
If no MAIL$PROTOCOL_xxx logical is found or the file specified doesn't
contain a MAIL$PROTOCOL routine then VMS mail signals an error message:
%MAIL-E-ERRACTRNS, error activating transport xxx
MAIL$PROTOCOL Calling sequence:
Once the MAIL$PROTOCOL routine is found and loaded VMS mail calls it
repeatedly to pass the mail information to the foreign transport. Each call
to the MAIL$PROTOCOL routine passes a function code to indicate what is to be
done, a pointer to a context block (created by the MAIL$PROTOCOL routine on
the first call) and eight parameters whose meanings vary depending on the
function code. There is an "initialization" function code sent first for each
message and one sent last to tell the MAIL$PROTOCOL routine to send the
message. There is a function code for a recipient address, one function code
each for the text from the From:, To:, Subj:, and CC: lines and a code for
the text of the mail itself (a pointer to an RMS rab for the text of the
messages is passed).
MAIL$PROTOCOL process context:
The MAIL$PROTOCOL code runs in the context of the process that sends mail to
the foreign transport address.
MAIL$PROTOCOL invocation with DECnet mail:
Consider this example:
MAIL> send
To: SUMNOD::SMTP%"[email protected]"
...
Such an address is entirely legal. It might be used from a system that has
DECnet but no TCP/IP installed as a means to still send SMTP mail. In what
process context is the MAIL$PROTOCOL interface being invoked in this example?
When such an address is used the MAIL$PROTOCOL code is running in the context
of the DECnet MAIL network object on the remote DECnet node. In this case
that's SUMNOD. The same ecanrio occurs when DECnet mail is sent to a user who
has mail forwarded to SMTP. This is important to know when debugging problems
with mail being sent between DECnet and SMTP mail. See "UCX SMTP and non-SMTP
mail".
----------------------------------------
UCX SMTP Control File
----------------------------------------
With UCX SMTP each mail message is packaged into a special purpose binary file
called a control file. This control file is submitted to a generic UCX SMTP
queue to be processed by the UCX SMTP symbiont. Each control file contains one
and only one SMTP mail message. Note that an SMTP message addressed to multiple
recipients is stored in one control file.
--------------------
CF Filenames - what they mean.
--------------------
The name of a control file and the directory it's in tell you something about
what type of mail it is. The format for the name of a control file is:
yymmddhhsshh_username.UCX_scsnode
where
"yymmddhhsshh" is a timestamp taken when the file is created.
"scsnode" is the value of the SYSGEN SCSNODE parameter.
"username" is the username of the process in which the CF was
created.
The different possiblities for "username" are:
Username: Meaning:
UCX_SMTP This is mail that came onto your system via SMTP. The control
file was created by the UCX SMTP receiver process which runs in
the UCX_SMTP account.
MAIL$SERVER This is mail that came in over DECnet and was destined for an
SMTP address. The control file was created by the DECnet Mail11
network object which runs in the MAIL$SERVER account. This
happens when the username on your system to which the DECnet/MTS
mail is destined has forwarded their mail to and SMTP address.
SYSTEM This is one of two things. If the control file is sitting in the
UCX_SMTP account's directory then it is a bounced mail message
created by the UCX SMTP symbiont which runs in the SYSTEM
account. Otherwise the control file will be in the default mail
directory of the SYSTEM user in which case the control file is
simply an SMTP mail sent by the SYSTEM user.
regular user
name This is mail composed by the user and sent to an SMTP address.
--------------------
Control file layout
--------------------
The UCX SMTP control file is a binary file that holds a single SMTP mail
messages. It has separate areas for the destination address information,
selected, pre-parsed RFC headers and the raw text itself. This section describes
these three areas.
---------------
Address Area
---------------
The address area of the control file contains each of the SMTP recipient
addresses. (ie. the addresses for the RCPT TO SMTP protocol command(s).) It is
implemented as a linked list of linked lists. The primary list is a list of the
domain parts of the addresses. Each domain node in the linked list points in
turn to a list of addresses within that domain. Say a mail was to delivered to
[email protected], [email protected] and [email protected]. The list would
look like this:
list head
|
|
V
| +-------------------+ +---------------------+
+--->| xyzcorp.com |-->| sumskool.edu |-->End-of-domain-list
+-------------------+ +---------------------+
domains | |
--------------+-------------------------+-----------
addresses | |
V V
+-------------------+ +---------------------+
| [email protected] | | [email protected] |
+-------------------+ +---------------------+
| |
V V
+-------------------+ End-of-address-list
| [email protected] |
+-------------------+
|
V
End-of-address-list
All nodes in the list (domain nodes and address nodes) also contain a block of
status bits. There are bits to indicate, among other things, if the address is
local or not (see "Local vs. non-local SMTP mail") , if the mail has been
successfully delivered, needs to be bounced or needs to be requeued.
---------------
Header Area
---------------
As the mail message is built into the control file certain headers are
pre-parsed into an area called the header area so that they can be accessed
without reparsing the entire header block. Each header value parsed into the
header area has a code to identify the name of the header (eg. FROM, TO, etc.)
and the value associated with it. The headers which are parsed into the header
area are:
Received, Return-Path, Date, From, To, Subject, CC, Sender, Comments,
Errors-To, Keywords, Reference, X-Organization, X-VMS-To, X-VMS-CC, Reply-To,
Resent-Reply-To, Resent-From, Resent-Sender and Message-ID
When you issue a UCX SHOW MAIL command the headers it shows you are actually
those from the header region, not from the text area. This explains why headers
other than those in the list above don't appear in the output of UCX SHOW MAIL.
The value of each header in the header area is stored in its "unfolded" format.
That is, multi-lined values are folded into one single line with all extraneous
white space removed.
Note about storage of the MAIL FROM address/Return-Path header:
As said in the SMTP Protocol review section the "envelope" addresses (those
from the MAIL FROM and RCPT TO commands) are to be stored separately from the
message headers. The RCPT TO addresses are stored separate from the headers
in the address area of the control file. However the MAIL FROM address is
stored in the Return-Path header. Note that if a mail message is to be sent
to another SMTP host the UCX SMTP client uses the address in the Return-Path
header in the address region of the control file to form the MAIL FROM
command. The Return-Path header is *not* sent by UCX SMTP as part of the RFC
822 message headers themselves. This is required by the RFCs since the
Return-Path header is only to be added into the headers by the final
destination host system. We only store it in the Return-Path header for the
convenience of our own implementation.
---------------
Text Area
---------------
The text area contains the raw text of the mail message. This includes the
headers, the blank line and the message body. In some cases the text area of the
control file contains a pointer to a text file that contains the text. This text
file is named exactly the same as the control file except that it's extension is
".UCX_scsnode_TEXT".
----------------------------------------
Components of UCX SMTP
----------------------------------------
This section introduces the different components of UCX SMTP. For purposes of
this discussion we break them up into the symbiont, the receiver, the
MAIL$PROTOCOL code, and the control file code.
--------------------
The UCX SMTP symbiont/queues
--------------------
As was said earlier each mail message is packaged into a special purpose binary
file called a control file and submitted to a generic UCX SMTP queue. This
generic queue points to one or more SMTP execution queues which run the UCX SMTP
symbiont hereafter refered to simply as the symbiont. ** The symbiont receives
jobs one at a time from the generic SMTP queue and delivers them either locally
(via callable VMS mail ) or remotely (via SMTP).
The UCX SMTP queues are set up for you. By default the generic SMTP queue points
to one SMTP execution queue. You can change this with the SET CONFIGURATION
SMTP/QUEUE command. See "Configuring UCX SMTP".
The UCX SMTP queues are started by the UCX START MAIL command and stopped by the
UCX STOP MAIL command. The SMTP satrtup file UCX$SMTP_STARTUP.COM does a UCX
START MAIL and the SMTP shutdown file does a UCX STOP MAIL. The symbiont process
(or processes for multiple execution queues) are created when the queues are
started. The symbiont process runs in the SYSTEM account.
The symbiont processes *all* mail on UCX SMTP. It sends outbound mail via SMTP
and delivers inbound mail to users via callable VMS mail. It also decides when
and if a mail message needs to be bounced and creates the bounced mail message
and sends that mail too. None of the other UCX SMTP components actually send any
mail. They package it into a control file and submit it to the symbiont.
** See your VMS documentation for more information about the VMS queueing system
and symbiont processes. For those who care the UCX SMTP symbiont is a single
streamed, asynchronous delivery SMB$ type symbiont. (See the VMS Utility
Routines Reference Manual for information on the implementation of VMS
symbionts.)
--------------------
The UCX SMTP Receiver
--------------------
The UCX SMTP receiver is invoked by the UCX Auxiliary Server when an inbound
TCP/IP connect comes into port 25 (assuming the UCX SMTP service has been
enabled). The Auxiliary server runs the COM file specified in the SMTP service
database entry UCX$SMTP_RECV_STARTUP.COM - which runs the receiver. The receiver
image is SYS$SYSTEM:UCX$SMTP_RECEIVER.EXE. The receiver process runs in the
UCX_SMTP account.
The receiver acts as the SMTP server in the SMTP client-server protocol dialog.
It saves the mail message from the client in a control file and submits it to
the symbiont once the message is received successfully. The receiver doesn't
deliver mail itself. Nor does it validate the existence of users to whom the
mail is being sent.
The receiver process is single threaded and once it serves a mail message it
exits. That is, for each new inbound connection to port 25 a new SMTP receiver
process is created.
--------------------
The UCX SMTP MAIL$PROTOCOL code
--------------------
This was discussed extensively in the section on the MAIL$PROTOCOL interface. To
review the MAIL$PROTOCOL code is invoked by VMS mail when a message is to be
delivered to an SMTP address. It runs in the context of the process that is
sending the mail. This is either a user who is sending SMTP mail from the VMS
mail UI or the DECNet Mail11 network object running in the MAIL$SERVER account.
The only additional information is that as the MAIL$PROTOCOL code receives the
mail message from VMS mail it packages it up into a control file and submits the
control file to the symbiont to deliver.
--------------------
The UCX SMTP control file code
--------------------
This code is called by each of the other four components. The receiver and
MAIL$PROTOCOL code use it to create control files. The symbiont uses it to read
control files and to create control files for bounced mail messages. The control
file code contains routines (among others) to:
- Create, delete, open and close a control file
- Put new domain, address, RFC header or line of text to file **
- Read existing domain, address, RFC header or line of text from file
- Read/write domain/address status blocks
- Submit a control file to the queue
The error reporting in this code is not up to snuff. 99% of the different
problems they incur are reported as either a generic "control file error" or
"Bad parameter". Fixing this is in the worklist.
** See "UCX SMTP Control File" for more information on the contents of the
control file.
----------------------------------------
UCX MX Lookup
----------------------------------------
As discussed earlier an SMTP mailer needs to access DNS maintained MX records in
order to help it route mail. UCX has a procedure called MX lookup to do this. It
has some value added over a raw DNS MX lookup. This section documents the UCX MX
lookup procedure.
The UCX MX lookup procedure is a routine that takes as input the destination
SMTP domain and returns a list of gateway hosts and preferences (well, sometimes
preferences, keep reading) associated with them. The value added over a raw DNS
MX lookup is twofold:
- The UCX MX lookup returns MX records sorted in ascending order of
preference (MX records returned higher priority before lower priority).
When there are multiple MX records with the same preference their order in
the return list is the same as returned by DNS. Since DNS returns records
in a different order each time the records with the same preference will
not appear in the same order in the UCX MX lookup sorted list each time.
This is essential for proper functioning of multiple MX records with the
same preference as a load balancing mechanism between peer mailhubs. For
an example of this see "MX Record Overview - MX Example 1".
- Four database search
As said earlier the UCX MX lookup procedure returns a list of gateway
hosts and preferences. What was not said is that the search for these
gateway hosts extends beyond DNS MX records. The UCX MX lookup actually
searches four "databases" (for lack of a better word) to resolve an MX
query. Following are the four databases that are searched in the order
they are searched:
UCX Local MX database
DNS MX lookup
UCX local host database
DNS host lookup
As you can see the last two databases listed here are looking at host
records not MX records. Given this the term "UCX MX Lookup" is a bit of a
misnomer because the MX lookup is looking at something in addition to MX
records. The UCX MX lookup takes the destination mail domain to be looked
up and performs queries on this domain in each of the above four
databases. In the case of the host databases it returns a preference of
"0" since there is no preference value associated with a host record.
The UCX SHOW MX command and the UCX SMTP code both call the UCX MX Lookup
routine which lives in SYS$LIBRARY:UCX$ACCESS_SHR.EXE. Consequently if you
want to see what MX records UCX SMTP sees for a particular non-local mail
domain just use UCX SHOW MX. (See "Displaying MX records with UCX".)
The MX lookup routine filters duplicate gateway names. It will never
return a record for a gateway that it has already passed back to the
caller. If it's already told you about a possible gateway there's no point
in repeating it. Try doing a UCX SHOW MX on a hostname that has an MX
record where the gateway name is the hostname itself. Notice that UCX SHOW
MX shows you the MX record but not the host record since it would simply
be a repeat of the MX record.
--------------------
Displaying MX records with UCX
--------------------
UCX supports two methods for MX lookup - the Unix based nslookup program and the
UCX SHOW MX_RECORDS command. UCX nslookup hooks directly to the DNS and returns
the MX records just as they are passed from the DNS - unsorted. The UCX SHOW MX
command provides some additional functionality on top of the bare DNS MX lookup.
See "UCX MX Lookup".
The following examples show how to use UCX's nslookup and UCX SHOW MX to do an
MX lookup on the destination domain "ucx.lkg.dec.com".
Nslookup invocation:
$ mcr ucx$nslookup
Default Server: ucxaxp.ucx.lkg.dec.com
Address: 16.20.208.53
> set type=MX
> ucx.lkg.dec.com
Server: ucxaxp.ucx.lkg.dec.com
Address: 16.20.208.53
ucx.lkg.dec.com preference = 100, mail exchanger = mail2.digital.com
ucx.lkg.dec.com preference = 200, mail exchanger = crl.dec.com
ucx.lkg.dec.com preference = 50, mail exchanger = ucxaxp.ucx.lkg.dec.com
ucx.lkg.dec.com preference = 75, mail exchanger = lassie.ucx.lkg.dec.com
ucx.lkg.dec.com preference = 100, mail exchanger = mail1.digital.com
ucx.lkg.dec.com nameserver = ucxaxp.ucx.lkg.dec.com
ucx.lkg.dec.com nameserver = lassie.ucx.lkg.dec.com
mail2.digital.com inet address = 204.123.2.56
crl.dec.com inet address = 192.58.206.2
ucxaxp.ucx.lkg.dec.com inet address = 16.20.208.53
lassie.ucx.lkg.dec.com inet address = 16.20.208.100
mail1.digital.com inet address = 204.123.2.50
> exit
The nslookup MX lookup first gives you the name and IP address of the DNS
server from which the information came. In the example it was
ucxaxp.ucx.lkg.dec.com. Next it gives you a list of MX records for the
destination domain you queried on. Note that the MX records are not sorted by
preference. This is because nslookup hooks directly into DNS which presents
the MX records to the caller in this unsorted fashion. Finally, as a niceity,
nslookup shows you the IP addresses for the mail exchanger (ie. gateway
hostname) fields.
UCX SHO MX_RECORDS invocation:
UCXAXP> ucx
UCX> sho mx ucx.lkg.dec.com
BIND MX database
Server: 16.20.208.53 ucxaxp.ucx.lkg.dec.com
Gate address Preference Gate name
16.20.208.53 50 ucxaxp.UCX.LKG.DEC.COM
16.20.208.100 75 lassie.UCX.LKG.DEC.COM
204.123.2.56 100 mail2.digital.com
204.123.2.50 100 mail1.digital.com
192.58.206.2 200 crl.dec.com
BIND database
Server: 16.20.208.53 ucxaxp.ucx.lkg.dec.com
Host address Host name
16.20.208.208 UCX.LKG.DEC.COM
UCX> exit
Note that there are differences in the output of nslookup and UCX SHOW MX.
These are due to the value added by the UCX MX Lookup. For more information
see "UCX MX Lookup".
----------------------------------------
Local vs. non-local SMTP mail
----------------------------------------
To deliver mail to any address UCX SMTP needs to assess whether the address is
"local" or not. If an address is local the symbiont strips off the domain part
of the address and delivers the mail to the local-part of the address via VMS
callable mail. If an address is non-local it is relayed via SMTP off to another
host for it to deliver (or to relay further).
UCX SMTP determines if an address is local when it initially parses the address
and stores it into the control file address region. Once the address is parsed
into a local-part and domain part the SMTP address parsing code looks at the
domain part of the address to see if it is local. In order to determine if the
domain part of the SMTP address is local or not the UCX SMTP parsing routine
performs the following steps:
- Compare the domain to the SMTP substitute domain. If the domain matches
the SMTP substitute domain it is considered local. This step is not performed
if:
- The logical UCX$SMTP_NO_SUBS_DOMAIN_INBOUND is defined. See
"Configuring UCX SMTP".
- No substitute domain is configured.
- Compare the domain case insensitively to the domains listed in the local
aliases file. If a match occurs then it is considered local.
- Build a list of all IP addresses for the domain and compare
this list to a list of all of the local host's IP addresses (including any IP
cluster alias). If a match occurs then the address is local.
Miscellaneous notes:
- The status blocks of the domain and address nodes in the linked list of
the control file address area contain a "local" bit which is set if the
address is local. After this any code that reads this address from the
control file knows if it is local or not.
- When talking about local and non-local mail it is important to remember
that a mail message can be both. In the case where there are multiple SMTP
recipients and one of them is local and another is not the mail is delivered
locally to the local address and relayed out via SMTP to the non-local one.
So strictly speaking it's not the mail that's local or non-local it's the
address. By convention though when we are talking about delivering mail to a
local address we call it local mail and when we are delivering mail to a
non-local address we call it non-local mail.
----------------------------------------
The SMTP Alternate Gateway/Zone
----------------------------------------
The SMTP alternate gateway along with the SMTP zone provide an alternative to
using straight MX records to send non-local mail. When you define an alternate
gateway any mail destined for a domain outside the zone is sent to the alternate
gateway. If an alternate gateway is defined and no zone is defined then the
value of the UCX$INET_DOMAIN logical is used as the zone. This means that all
mail to systems outside your DNS zone will be sent to the alternate gateway.
----------------------------------------
How UCX decides where to send non-local mail
----------------------------------------
This section describes how UCX SMTP decides where to send a mail to a non-local
address since that has been a source of alot of confusion. There are a few
things that affect where the symbiont tries to route the mail:
- The alternate gateway and zone and the results of the UCX MX lookup on the
alternate gateway. See "The SMTP Alternate Gateway/Zone"
- The results of the UCX MX Lookup on the domain part of the recipient
address. See "UCX MX Lookup".
- Definition of a few select UCX SMTP configuration logicals:
- UCX$SMTP_ALTGATE_ALWAYS
- UCX$SMTP_MX_IF_NOALTGATE
- UCX$SMTP_NO_MX
See "Non-local mail routing options" under "UCX SMTP configuration
logicals" for more information on these logicals.
As described in the section about the alternate gateway and zone mail destined
for a domain outside the zone is sent to the alternate gateway.
If no alternate gateway is configured then all non-local mail is delivered using
the UCX MX Lookup gateway list. (See "UCX MX Lookup").
If an alternate gateway is configured but the domain of the recipient address
is within the zone then the message is sent to that address using the UCX MX
Lookup gateway list.
Modifications to this behavior:
If you have configured the UCX$SMTP_ALTGATE_ALWAYS logical then all non-local
mail is sent to the alternate gateway. No zone check is performed.
If you have configured the UCX$SMTP_MX_IF_NOALTGATE logical then any mail
that is to be sent to the alternate gateway will be sent using the UCX MX
Lookup gateway list *if* the symbiont is unable to get to the alternate
gateway. If you have not configured this option and UCX SMTp caanot get to
the alternate gateway the message will be requeued.
If you have configured the UCX$SMTP_NO_MX logical then mail that is not
destined for the alternate gateway will not be sent using the UCX MX Lookup
gateway list. Rather, the domain part of the address will be translated into
a DNS host name directly and the mail sent directly to that address. If the
domain of the address doesn't translate as a DNS hostname the mail will
bounce. This feature isn't particularly useful.
--------------------
How the symbiont sends a mail using the UCX MX Lookup gateway list
--------------------
When the symbiont decides that mail to a particular recipient is not going out
via the alternate gateway (eg. no alternate gateway is defined or the recipient
address is within the zone) the symbiont must relay the mail itself. First it
calls the UCX MX Lookup routine passing it the domain part of the recipient
address. As described in "UCX MX Lookup" this returns a list of potential
gateways for the mail in a prioritized order. The symbiont tries to send the
mail via the SMTP protocol to the first gateway in the list. If this fails it
tries the next gateway in the list and on and on until either the mail has been
sent or the list is exhausted. If the list is exhausted then the mail is
requeued to be tried at a later time.
--------------------
How the symbiont sends a mail through the alternate gateway
--------------------
To send a mail to the alternate gateway you might conclude that the symbiont
translates the alternate gateway to an IP address using gethostbyname and
connects to the address. This assumption is understandable but incorrect. To
send mail to the alternate gateway the symbiont calls the UCX MX Lookup routine
(see "UCX MX Lookup") on the alternate gateway and uses the resulting list of
gateways to get the mail to the alternate gateway. To understand the benefit of
this consider this example which is how we at UCX engineering have UCX SMTP
configured on our own system.
First our alternate gateway and zone:
UCX> sho config smtp
...
Alternate gateway: relay.dec.com
...
Zone: dec.com
...
What if we try to translate the alternate gateway, "relay.dec.com" as a host name?
UCX> sho host relay.dec.com
%UCX-W-NORECORD, Information not found
-RMS-E-RNF, record not found
There is no such host as relay.dec.com because relay.dec.com is just an MX
destination domain with multiple records at the same preference:
UCX> sho mx relay.dec.com
BIND MX database
Server: 16.20.208.53 ucxaxp.ucx.lkg.dec.com
Gate address Preference Gate name
192.208.46.10 100 mail11.digital.com
192.208.46.20 100 mail12.digital.com
192.208.46.30 100 mail13.digital.com
204.123.2.56 200 mail2.digital.com
204.123.2.50 200 mail1.digital.com
193.56.15.21 300 mail21.digital.com
When UCX SMTP on our system sees a mail destined for a domain outside of
"dec.com" it uses the above list of MX records to send the mail to what is
really an imaginary entity called "relay.dec.com". So, even when mail is routed
through the alternate gateway the UCX MX Lookup list is used.
---------------
Further enhancements in worklist for mail destined for alternate gateway
---------------
This section documents some items which are in the worklist with respect to mail
destined for the alternate gateway.
Enhance caching of alternate gateway MX records:
The symbiont calls the UCX MX Lookup procedure for the alternate gateway once
at initialization time and caches the resulting list of gateways. This is
done for efficiency to save the cost of an MX lookup on the alternate gateway
each time mail is to be sent to the alternate gateway. The symbiont refreshes
the cached list of MX records for the alternate gateway every 24 hours.
There is a problem with this. Consider the case of our own relay.dec.com
alternate gateway. The domain relay.dec.com has multiple MX records defined
for it at the same preference. As we discussed in "MX Example 1" in the
section "MX Record Overview" in part 1 this is to achieve load balancing
between the gateway hosts that share the same preference. Load balancing
should occur because the MX records with the same preference appear in a
different order each time they are looked up. However since the MX records
for the alternate gateway are cached by the UCX SMTP symbiont it uses the
same MX record sequence every time and so cannot take advantage of the load
balancing that has been set up for the alternate gateway.
For example if the UCX SMTP symbiont has cached the MX records for
relay.dec.com as above it would always try to connect to mail11.digital.com
first. No mail would get sent through mail12.digital.com or
mail13.digital.com. When the MX records are refreshed after 24 hours the
order of the gateways with the same preference may change and then the
symbiont will connect to a different host - but always the same one - until
another 24 hours passes and the alternate gateway MX cache is refreshed
again.
The solution for this is to periodically reshuffle the order of the MX
records (with the same preference) in the alternate gateway MX list.
Send all mail to alternate gateway in one SMTP session:
UCX SMTP has a design inefficiency with respect to sending mail via the
alternate gateway for mail with multiple recipients outside the zone. This is
most commonly encountered when sending mail to a mailing list in which there
are multiple recipients outside the zone. With the present design of the
control file the the symbiont does everything on a domain by domain basis. It
sends mail to one domain and then to the next and to the next, etc. If
addresses in different domains are all destined for the alternate gateway
they will still be sent separately.
Say for example you have a zone of z.com with an alternate gateway of
mail.z.com and you have a mailing list with the addresses [email protected] and
[email protected]. Since these two addresses are in different domains they are
processed separately by the symbiont. First we connect to the alternate
gateway and go through the whole SMTP dialog for [email protected]. Eg:
send: HELO whereever.z.com
recv: 250 mail.z.com Hello whereever.z.com pleased to meet you
send: MAIL FROM:<[email protected]>
recv: 250 <[email protected]>... Sender ok
send: RCPT TO:<[email protected]>
recv: 250 Recipient ok
send: DATA
recv: 354 Enter mail, end with "." on a line by itself
send: message headers
send: [blank line]
send: message text
send: .
recv: 250 Message accepted for delivery
send: QUIT
recv: 221 closing connection
Then the link is disconnected and we go on to do the exact same thing for
[email protected]. For mailing lists with many recipients outside the zone this is
costly. It would be far more effient if the mail was sent through the
alternate gateway with multiple RCPT TO commands for each of the domains
destined for the alternate gateway. Eg:
send: HELO whereever.z.com
recv: 250 mail.z.com Hello whereever.z.com pleased to meet you
send: MAIL FROM:<[email protected]>
recv: 250 <[email protected]>... Sender ok
send: RCPT TO:<[email protected]>
send: RCPT TO:<[email protected]>
recv: 250 Recipient ok
send: DATA
recv: 354 Enter mail, end with "." on a line by itself
send: message headers
send: [blank line]
send: message text
send: .
recv: 250 Message accepted for delivery
send: QUIT
recv: 221 closing connection
There is a work around for this. You need to trick UCX SMTP into dealing with
each of the addresses destined for the alternate gateway as if they were in
the same domain. You can do this by defining the addresses of users in a
mailing list who are outside the zone using the percent sign hack. Taking the
example above you'd create the list like so:
$ Create alist.dis
user1%[email protected]
user2%[email protected]
<CTRL-Z>
Using the percent sign hack you have given both of the addresses the same
domain part - the name of your alternate gateway. As far as UCX SMTP is
concerned they are in the same domain. This means that when the symbiont goes
to deliver the mail each of these addresses will be in the linked list under
the "mail.z.com" domain and will be dealt with together.
----------------------------------------
Forwarding to an SMTP address
----------------------------------------
Consider the case where an SMTP mail comes into your system that is to be
delivered locally and the recipient has a mail forwarded to an SMTP address.
This section discusses some special case handling in the symbiont to handle
such forwarding to an SMTP address which is refered to as SMTP-to-SMTP
forwarding.
When the symbiont delivers mail locally it checks the local-part of the address
to see if there is a forwarding string (ie. SET FORWARD) for it. It does so
using the mail$user_* VMS callable mail routines. If there is a forwarding
string the symbiont checks to see if it is an SMTP address. If it is then some
special handling is in order.
Let's take an example to show what would happen if there was *no* special
checking for delivering SMTP mail to a local recipient with their mail forwarded
to SMTP. Say user jones on a UCX system called systm2.xyzcorp.com has forwarded
their mail to SMTP%"[email protected]". Now say mail arrives on
systm2.xyzcorp.com for [email protected]. What would happen if the
symbiont didn't make special checks for SMTP forwarding here? The symbiont would
deliver mail via callable mail to the reciopient address "jones". What is VMS
mail going to do with this? It sees the mail is forwarded to SMTP% and so it
locates and invokes the UCX SMTP MAIL$PROTOCOL routine (see "VMS mail Foreign
Transport/MAIL$PROTOCOL Interface") to deliver the message. VMS mail passes the
recipient's forwarding address - [email protected] - to the
MAIL$PROTOCOL routine as the recipient address. This happens inline in the
symbiont process but is totally transparent to the symbiont code itself. As far
as the symbiont code is concerned it has made a simple call to
mail$send_add_address for user jones. After the call to mail$send_add_address
the symbiont passes the VMS From, To, etc lines to callable mail and each call
is redirected by VMS mail to the SMTP MAIL$PROTOCOL code. Finally the symbiont
passess each line of text starting with the message's RFC headers and then the
blank line and the text of the message.And agin this is passed to the
MAIL$PROTOCOL code. Sounds fine, no?
This creates a problem though. Invoking the MAIL$PROTOCOL code causes the
generation of a *new* SMTP mail message with *new* SMTP headers. The entirety of
the existing SMTP message that we are trying to send (including its RFC 822
headers) is now hidden away - "encapsulated" - inside the new message as the
text of the message. There are now two blocks of RFC 822 headers in the new
message. The first block of headers are the ones the MAIL$PROTOCOL interface has
just created. They are followed by a blank line and the original RFC 822
headers, another blank line and the original body. This is obviously a pain
since the recipient has more headers to read through. It is also a bug since it
breaks the SMTP loop detection mechanism.
The SMTP loop detection mechanism requires that each MTA that touches a message
append a Received: header and that each MTA count Received: headers and bounce
the mail if the number of Received headers surpasses some locally configurable
limit. (This BTW is UCX SMTP's HOP_COUNT_MAX.) In this fashion mail that has
taken "too many hops" will be bounced as looping. Going back to the example say
that user jones had inadvertantly set their forwarding on host "othersys" right
back to systm2.xyzcorp.com - the classic mail forwarding loop. In this scenario
the mail would bounce back and forth between the two systems getting one
additional set of headers each time it was processed by systm2. Since each new
block of headers encapsulates the existing message into a new message thus
making the old block of headers invisible (they become the body of the message)
a Received: header is added on each time to the outermost block of headers and
then this block of headers is encapsulated within the message and a new header
block is created. So we have a mail message of many, many separate blocks of RFC
headers each with two Received: headers (one from systm2 and one from othersys)
rather than a single block of headers with *all* the Received: headers. As a
consequence the code that counts Received: headers to detect looping can't work
and the mail loops forwever. (BTW. UCX SMTP had this bug for a long time.)
To fix this the UCX SMTP symbiont checks to see if a local user to whom it is
delivering an SMTP message has mail forwarding set to an SMTP address. If it
does it dynamically adds a new address - the user's forwarding address - to the
control file address list. It then marks the address status block of the old
address as having been successfully delivered. In the case of our example that
means that the recipient address [email protected] is marked as delivered
successfully and jones's forwarding address [email protected] is
dynamically added to the control file. The symbiont will see this address and
then deliver the mail to othersys.xyzcorp.com via SMTP without the encapsulation
and bugs described above.
Situation where encapsulation problem can still occur:
If you define a SYSTEM logical name pointing to an SMTP address and the
symbiont tries to deliver mail to that address that situation is not caught
yet and you'll get the header encapsulation. EG:
$ DEFINE/SYSTEM JONES SMTP%"""[email protected]"""
Try it. You'll see.
Support for setting forward to SMTP address without SMTP% jacket:
For some later versions of VMS you can do:
MAIL> SET FORWARD/[email protected]
Ie. you don't need the SMTP jacket. UCX SMTP can handle this too. That is,
its special case check to see if an address has been forwarded to SMTP
detects this kind of forwarding.
----------------------------------------
UCX SMTP and non-SMTP mail
----------------------------------------
One issue for any VMS based SMTP mailer is how to deal with mail coming from or
going to non-SMTP mailers. This happens mostly when users have their mail
forwarded. If a user's mail is forwarded to an SMTP address and a non-SMTP mail
(say DECnet) arrives that's the case where mail is going from a non-SMTP MTA to
SMTP. If a user's mail is forwarded to a non-SMTP address (say DECnet) and an
SMTP mail arrives it's the case of mail going from SMTP to a non-SMTP MTA. This
section discusses what happens when mail moves accross the boundary between SMTP
and non-SMTP MTA's.
--------------------
SMTPfication of non-SMTP addresses
--------------------
Before discussing the specifics of what's happening when mail moves accross the
boundary between SMTP and non-SMTP MTA's it is necessary to understand a
procedure implemented in UCX SMTP for transforming any non-SMTP address into an
SMTP address. This procedure is called SMTPfication and is used on addresses as
they cross the boundary from non-SMTP land to SMTP land. With SMTPfication the
non-SMTP address is munged and this munged string becomes the local-part of the
created SMTP address. The domain part of the created SMTP address becomes the
UCX SMTP substitute domain or, if no substitute domain is configured, the host's
hostname.
The SMTPfication steps are as follows:
1) Scan the non-smtp address and prefix a backslash (\) character to each
double quote character (if any).
2) Put the entire string in double quotes.
3) Append an '@' followed by the substitute domain (or hostname if no
substitute domain)
---------------
Example SMTPfications
---------------
This section gives some examples of address being SMTPfied. They all assume that
the UCX SMTP host doing the SMTPfication has a substitute domain of
xyzcorp.com.
Example 1 - DECnet address:
Before SMTPfication:
SUMNOD::SMITH
After Step 1 (Prefix double quotes with a \):
SUMNOD::SMITH
After Step 2 (Put entire string in double quotes):
"SUMNOD::SMITH"
After Step 3 (append @domain):
"SUMNOD::SMITH"@xyzcorp.com
Example 2 - MTS address:
Before SMTPfication:
SUMNOD::XXXMTS::MRGATE::"YYYMTS::A1::SMITH"
After Step 1 (Prefix double quotes with a \):
SUMNOD::XXXMTS::MRGATE::\"YYYMTS::A1::SMITH\"
After Step 2 (Put entire string in double quotes):
"SUMNOD::XXXMTS::MRGATE::\"YYYMTS::A1::SMITH\""
After Step 3 (append @domain):
"SUMNOD::XXXMTS::MRGATE::\"YYYMTS::A1::SMITH\""@xyzcorp.com
Example 3 - another foreign transport address:
Before SMTPfication:
DSN%SRQ$Z123456-7890
After SMTPfication:
"DSN%SRQ$Z123456-7890"@xyzcorp.com
Example 4 - Quoted DECnet address:
Before SMTPfication:
SUMNOD::"[email protected]"
After SMTPfication:
"SUMNOD::\"[email protected]\""@xyzcorp.com
Example 5 - X.400 --> MTS address. Yuk!:
Before SMTPfication:
SUMNOD::MRGATE::"XXX::1=FI::2=XXX::3=XYZ::JOE SMITH"
After SMTPfication:
"SUMNOD::MRGATE::\"XXX::1=FI::2=XXX::3=XYZ::JOE SMITH\""@xyzcorp.com
---------------
When is SMTPfication necessary?
---------------
SMTPfication becomes necessary whenever the UCX SMTP code needs to form an
address from a VMS address for an RFC 822 header or an SMTP protocol command and
the string to be used for the local-part of the address is not a simple
username. SMTPfication is not needed for a simple VMS username because a VMS
username is syntactically legal for the local-part of an RFC 822 address. For
example the simple VMS mail address of JONES becomes [email protected] not
"jones"@xyzcorp.com. However the latter form is actually legal and would work.
SMTPfication is also performed by the UCX POP server when it creates the
dummied up From: RFC 822 header from the VMS mail From: line. For more
information on POP's dummied up RFC 822 headers see the POP documentation.
--------------------
Mail from non-SMTP to SMTP
--------------------
The first case we will discuss is the one where a mail comes from a non-SMTP
mail environment and is to be delivered into the SMTP environment. The most
common cuase for this is when a non-SMTP mail arrives for a user with mail
forwarded to an SMTP address.
The key issue in accepting mail from a non-SMTP MTA and passing it into the SMTP
world is getting the From: and Return-Path: (er um, the MAIL FROM) addresses
correct. If the From: address is not correct the recipient will not be able to
reply. If the address that we put in the MAIL FROM: command is not correct and
the mail needs to be bounced then it will not be bouncable.
When mail arrives for a user with mail forwarded to an SMTP address VMS mail
invokes the SMTP MAIL$PROTOCOL code (see "VMS mail Foreign Transport/
MAIL$PROTOCOL Interface") and passes the mail message to it to deliver. As
discussed in the MAIL$PROTOCOL section one of the calls that VMS mail makes to
the SMTP MAIL$PROTOCOL routine passes the VMS mail From: line. The SMTP
MAIL$PROTOCOL code uses the VMS mail from: line to form the RFC 822 From: and
Return-Path: headers. To do so it uses SMTPfication on the VMS mail From: line.
---------------
An example
---------------
Take an example where user SMITH on DECnet node SYSTM1 sends an DECnet mail to
user JONES on DECnet SYSTM2. Say that DECnet node systm2's UCX host name is also
systm2.xyzcorp.com and that its substitute domain is xyzcorp.com. Finally say
that user JONES has their mail forwarded to SMTP%"[email protected]".
On SYSTM1 SMITH does:
MAIL> send
To: SYSTM2::JONES
Subject: Test
test text here
<CTRL-Z>
On SYSTM2 the MAIL$SERVER network object (running SYS$SYSTEM:MAIL_SERVER.EXE)
delivers the mail to user JONES and sees that it is forwarded to SMTP%. This
means the UCX SMTP MAIL$PROTOCOL routine is called to deliver the message. When
the From: line is passed to MAIL$PROTOCOL it's:
From: SYSTM1::SMITH
At this point the MAIL$PROTOCOL routine forms an RFC 822 From: line of
From: "SYSTM1::SMITH"@xyzcorp.com
And the Retun-Path header (which will be put into the MAIL FROM SMTP protocol
command) is the same. Ie:
Return-Path: "SYSTM1::SMITH"@xyzcorp.com
Note that nothing special needs to get done with the To: field. The SMTP
MAIL$PROTOCOL code forms the RFC 822 To: field from the value passed to it with
the recipient address function code which is whatever was inside the SMTP%
jacket. In this case it's the forward string for user jones which is
[email protected]. Ie. This scenario doesn't complicate the
construction of the To: header.
After the rest of the MAIL$PROTOCOL interface calls are made the mail is
packaged up into a control file, submitted to the symbiont and delivered to
[email protected].
---------------
SMTPfication as simple, generic mechanism for gatewaying mail
---------------
Consider the RFC From address "NODE::USER"@dom.ain. As illustrated by the
example above, the SMTPfication procedure applied to a mail address as it passes
from outside SMTP to SMTP is a way preserving the path that the mail has taken
within the mail address itself. Such a mechanism does produce syntactically
complex addresses but it is also useful as a generic mechanism for gatewaying
mail that works for any possible non-SMTP address that the UCX SMTP
MAIL$PROTOCOL code could be passed.
--------------------
Mail from SMTP to non-SMTP
--------------------
Now consider the case where mail is sent from the SMTP world into a non-SMTP
MTA. The example above is a good starting point. Say that user jones on
othersys.xyzcorp.com receives the example mail from SMITH and replies. If
othersys is a UCX SMTP system the VMS mail From: line will look like:
From: SMTP%"�SYSTM1::SMITH�@xyzcorp.com"
For the time being ignore the cent signs in this address. They are required
because of historical address parsing deficiencies in VMS mail which are being
worked on. See "The UCX SMTP cent sign hack" for more information. Suffice it to
say that this how an RFC 822 From: line of `From: "SYSTM1::SMITH"@xyzcorp.com'
is stuffed into the VMS mail From line by UCX SMTP.
Say that user jones replies to this mail. This will cause a mail to be sent to
the address in the VMS mail From: line - SMTP%"�SYSTM1::SMITH�@xyzcorp.com".
Since this address starts with SMTP% VMS mail invokes the UCX SMTP MAIL$PROTOCOL
code on othersys. In this case the construction of the From: header is easy. The
From: line passed to the MAIL$PROTOCOL code is JONES so the RFC 822 From: header
is simply [email protected]. (Let's assume that this system has no
substitute domain.)
When the SMTP MAIL$PROTOCOL code is passed the recipient address function code the
address looks like: �SYSTM1::SMITH�@xyzcorp.com. The MAIL$PROTOCOL code converts
the cent signs back to double quotes and this yields an RFC 822 To: header of:
To: "SYSTM1::SMITH"@xyzcorp.com
This address is also put into the control file address area (so that the
symbiont sends it out with a RCPT TO:<"SYSTM1::SMITH"@xyzcorp.com>). Once the
rest of the message is packaged up into a control file and submitted the
symbiont gets it and sends the mail using the SMTP protocol to
systm2.xyzcorp.com. (Let's assume that the MX record for xyzcorp.com points to
systm2.xyzcorp.com so systm2 receives the mail.)
When systm2 receives the mail it sees that it's local mail because the domain of
the RCPT TO: address is xyzcorp.com which matches its substitute domain. (See
"Local vs. non-local SMTP mail".) So the symbiont delivers the mail locally via
callable VMS mail. (See "UCX SMTP and callable VMS mail".) To construct the
recipient address string to pass to callable VMS mail's mail$send_add_address
routine the symbiont now un-SMTPfies the recipient address. This is just the
reverse of the SMTPfication. So starting with this:
"SYSTM1::SMITH"@xyzcorp.com
it strips off the @domain to get this:
"SYSTM1::SMITH"
and strips off the outer double quotes to get this:
SYSTM1::SMITH
In this example there are no \" sequences but if there were it would remove the
'\' characters too as the last step of un-SMTPfying the address. So, the string
that the symbiont passes to callable mail as the recipient address is
SYSTM1::SMITH.
To form the VMS mail From: line the symbiont does nothing special. It chooses
the From: RFC 822 header to be placed into the VMS mail from line and packages
it into an SMTP% jacket to make it a legal VMS incarnation of an SMTP address.
So the VMS mail From line that is passed to callable mail by the symbiont is:
From: SMTP%"[email protected]"
Now the rest of the mail is passed to VMS callable mail and the symbiont is done
delivering the mail. At this point the MAIL11 server takes over and delivers the
mail to SMITH on SYSTM1. (Actually it's done in line with the symbiont call to
callable mail because, after all, MAIL11 is not a store and forward protocol but
that's an unimportant detail.)
Note that the VMS mail From: line that SMITH sees will be enhanced by the MAIL11
code from what the SMTP symbiont passed to callable mail. The VMS mail From:
line that SMITH sees will be:
From: SYSTM2::SMTP%"[email protected]"
The SYSTM2:: is added to the front because the mail made the hop from SYSTM2 to
SYSMT1 via DECnet.
Now say a reply is done. This will basically do the same thing as the original
send that SMITH did to SYSTM2::JONES. The only difference between sending to
SYSTM2::JONES and SYSTM2::SMTP%"[email protected]" is that sending to
SYSTM2::SMTP%"[email protected]" saves the extra step performed in the
MAIL$SERVER on SYSTM2 of looking to see that jones has their mail forwarded
SMTP%"[email protected]".
---------------
Another SMTP to non-SMTP case
---------------
Another case similar to the example above is the case where SMTP mail is to be
delivered locally to a user who has their mail forwarded to a non-SMTP address.
Say systm2.xyzcorp.com is delivering a mail to [email protected] and user DOE has
mail forwarded to SYSTM1::DOE. In this case the symbiont doesn't know it's
gatewaying mail. It simply passes the string "doe" to VMS callable mail as the
recipient address and the VMS mail From: line is SMTP%"[email protected]".
When user DOE on SYSTM1 reads their mail it the VMS mail From: line appears in
exactly the same format as the previous example:
From: SYSTM2::SMTP%"[email protected]"
--------------------
Miscellaneous SMTP/non-SMTP issues
--------------------
Not just DECnet that works like above:
The examples above all used DECnet as the prototypical non-SMTP MTA. However
all the same things happen with other non-SMTP address like MTS addresses and
other foreign transport address too (eg. DSN%, NM%, etc.)
Unix sendmail.cf files tend to munge these addresses.
If some of these addresses (eg. From: "NODE::USER"@dom.ain) are passed
through or delivered to Unix systems there is a high likelyhood that the Unix
system will modify and corrupt the address. The cause of this is a Unix
sendmail.cf file that doesn't understand the address format and so misparses
it and rewrites it incorrectly. If you see this happen it is important to
understand that these addresses although syntactically complex are entirely
legal according to the RFC 822 address syntax.
Often the corruption of the address will be such that it still manages to get
delivered to where it is going but is unreplyable. Sometimes it never even
gets to where it's going and has to be bounced. Usually in that case the mail
can't be bounced either since the MAIL FROM address is in the same format as
the From address and if the one couldn't be understood then the other won't
be either. In that case it will most likely become a dead letter.
Here are some examples of correct versions of these types of addresses and
the types of munging that are commonly performed by bad sendmail.cf files.
Correct:
From: "SUMNOD::SMITH"@xyzcorp.com
Incorrect variant:
From: SUMNOD::[email protected]
This address is incorrect because the local part contains a colon
which is an RFC 822 "special" character. The RFC address syntax
requires that a local-part that contains a special character must be
in double quotes.
Correct:
From: "SUMNOD::XXXMTS::MRGATE::\"YYYMTS::A1::SMITH\""@xyzcorp.com
Incorrect variants:
From: "SUMNOD::XXXMTS::MRGATE::"YYYMTS::A1::SMITH""@xyzcorp.com
This variant is wrong because it doesn't "escape" the nested double
quotes with backslashes.
From: SUMNOD::XXXMTS::MRGATE::\"YYYMTS::A1::SMITH\"@xyzcorp.com
This variant is wrong for the same reason that
SUMNOD::[email protected] is wrong. Special characters in a
local-part require double quotes around the local part.
From: SUMNOD::XXXMTS::MRGATE::"YYYMTS::A1::SMITH"@xyzcorp.com
This variant is wrong for both reasons.
--------------------
SMTP/non-SMTP worklist items
--------------------
This section documents some items in the UCX worklist which will hopefully make
life easier for users who want to send mail between non-SMTP environments and
the Internet via UCX SMTP. The problem is that the complex addresses generated
by UCX SMTP's SMTPfication are usually munged by Unix sendmail.cf files.
Provide mechanism to hide DECnet addresses:
UCX may add support for hiding DECnet addresses inside SMTP addresses. This
is a commonly supported practice. The two options are
1) Have a special SMTP domain (perhaps configurable by the UCX system
manager) that UCX SMTP recognizes as mail destined for DECnet. For mail
coming from SMTP to this domain the SMTP mailer essentially takes whatever
is before the first "." in the domain and that is the DECnet nodename. For
mail coming from DECnet destined for SMTP we take the DECnet node name and
tack on the special DECnet domain to form an SMTP address. Digital's own
"enet.dec.com" domain is an example of this. When one sends mail to
[email protected] the mail goes via SMTP to a Digital mail hub that
has a wildcard MX for *.enet.dec.com pointing to it. This mail hub
transforms [email protected] to NODNAM::USER and forwards the mail
to that address via DECnet.
For example:
Say the new feature allows the system manager to configure their
special DECnet domain and the manager of xyzcorp.com has theirs set to
decnet.xyzcorp.com. Further assume they have a wildcard MX record set
up so that mail to be sent to *.decnet.xyzcorp.com is to be sent to
systm2.xyzcorp.com. Say a mail is sent from the Internet somewhere
outside xyzcorp's corporate network to [email protected].
It is routed via SMTP to systm2.xyzcorp.com. At this point UCX SMTP
sees that the domain of the SMTP address is the "special" DECnet domain
and so transforms the recipient address from
[email protected] to systm1::smith and delivers the mail
locally via callable mail to the address systm1::smith.
Coming in the other direction DECnet mail sent from SMITH to
SYSTM2::JONES where jones has mail forwarded via SMTP as in the
examples earlier in this section would no longer be From:
"SYSTM1::SMITH"@xyzcorp.com. It would now be From:
[email protected]
2) The other option is just a different syntactical technique to do the
same thing. With option 2 we put hooks in the mailer to support special
interpretation of the percent sign hack when the domain following the
percent sign indicates a particular foreign transport.
For example:
smith%[email protected]
Provide mechanism hide MTS addresses:
It may be possible to do the same thing for MTS (aka Message Router, MRGATE)
mail too. Eg: To: [email protected] funnels mail off to joeuser via
MTS when it arrives at xyzcorp.com.
For sites where users have VMS mail forwarded to ALL-in-1 see the description
of the UCX SMTP UCX$SMTP_REWRITE_MTS_FROM logical in section "Special case
logicals for MTS interaction".
----------------------------------------
Undeliverable Mail
----------------------------------------
When UCX encounters a mail that it cannot deliver it must bounce it back to the
sender. To do so it forms an undeliverable mail message (aka. a "bounced mail")
and sends it to the address in the SMTP Return-Path header (which is from the
MAIL FROM SMTP protocol command). In other words the MAIL FROM address of the
mail we were not able to deliver becomes the RCPT TO address of the bounced mail
message. The bounced mail message contains information about what went wrong,
which addresses got the mail and which didn't (for multiple recipient mail
messages) and also the entire text of the mail message being bounced. As
required by the SMTP RFC's the bounced mail message is sent with a null MAIL
FROM command (ie. MAIL FROM:<>) to prevent bounced mail about bounced mail.
What if we can neither deliver nor bounce a mail?
In the case where we discover that we cannot deliver a mail message and then
try to bounce it and cannot bounce it either what to do? Under such
circumstances UCX SMTP delivers the mail to the UCX_SMTP account under the
assumption that UCX_SMTP is forwarded to some account that is read regularly
by someone who wants to know about dead letters. (See "Postmaster field"
section of "UCX SMTP Configuration database entry".) If, for some reason, the
message cannot be delivered to the UCX_SMTP account it is left as an orphan
control file. ie. A dead letter. A message is signalled to the UCX SMTP
symbiont log.
What to do if bouncing bounced mail?:
Since a null MAIL FROM is legal an SMTP mailer must handle the case where it
is trying to bounce a mail message and the MAIL FROM is null. This is the
case where we are trying to deliver a message which is itself a bounced mail
and so cannot be bounced. In this case UCX SMTP will send the message to the
UCX_SMTP account. As above, if the message cannot be delivered to the
UCX_SMTP account it is left as an orphan control file and a message is
signalled to the UCX SMTP symbiont log.
----------------------------------------
Configuring UCX SMTP
----------------------------------------
This section documents the buttons knobs you have to configure UCX SMTP.
--------------------
UCX SMTP Common directory
--------------------
In earlier versions of UCX SMTP all files were stored in the UCX_SMTP system
specific directories. UCX SMTP now allows the system manager to create a
directory visible to all cluster nodes (or a subset) and store certain SMTP
files there. To use this feature the system manager must create the directory
and define a system logical UCX$SMTP_COMMON to point to it. At this time the
following files may go into UCX$SMTP_COMMON:
- UCX SMTP mailing list (.DIS) files. See "UCX SMTP Mailing Lists"
- The Local Aliases file. (See "Local Aliases File")
--------------------
Local Aliases File
--------------------
The local alias feature allows the system manager to define a list of domains
that UCX SMTP will interpret as local. (See "Local vs. non-local SMTP mail".) If
it sees mail to any one of the domains specified as local aliases it will
deliver the mail on the local system via callable VMS mail rather than
forwarding it on to another system via the SMTP protocol.
To define the aliases that you want to be considered local you simply use your
text editor or the DCL create command to create a file called
UCX$SMTP_LOCAL_ALIASES.TXT. You can put the file in either the UCX$SMTP_COMMON
directory or, if you don't define UCX$SMTP_COMMON, you can put the file in
SYS$SPECIFIC:[UCX_SMTP]. Each line in the file is one domain that you want UCX
SMTP to recognize as local. For example:
$ SET DEF UCX$SMTP_COMMON
$ CREATE UCX$SMTP_LOCAL_ALIASES.TXT
!
! This is a comment.
alias1.mydomain.edu
alias2.mydomain.edu
alias3.mydomain.edu
<CTRL-Z>
$ @SYS$MANAGER:UCX$SMTP_SHUTDOWN.COM
$ @SYS$MANAGER:UCX$SMTP_STARTUP.COM
The above example tells UCX SMTP to recognize "alias1.mydomain.com",
"alias2.mydomain.com" and "alias3.mydomain.com" as local. If the symbiont on
your host sees mail to [email protected], [email protected] or
[email protected] it will consider the mail local and try to deliver it
to the user on your system.
The entries in the local alias file must adhere to the following syntax rules.
- There may be only one alias entry per line.
- Comments follow the same rules of SMTP .DIS files. The comment character,
a "!", must appear in the first column.
- UCX SMTP does not append the local domain name to an entry that it reads
with no "." in it. For example, if your local domain is "mydomain.com" and
you want "alias1.mydomain.com" to be recognized as local you must enter
"alias1.mydomain.com" into the local alias file. Entering "alias1" is not
sufficient.
Other miscellaneous notes:
- When UCX SMTP compares entries in the local alias file with the domain of
an address it does so case insensitively. For example, if the local alias
file contains "alias1.mydomain.edu" and a mail is sent to
"ALIAS1.MYDOMAIN.EDU" a match occurs and the mail is considered local.
- The local aliases file protection must be W:RE.
- There is a maximum of 255 aliases each of which can be a maximum of 64
characters long.
- After changing the local alias file you must stop and restart UCX SMTP in
order for the changes to take effect.
One use of the local aliases file is in a cluster where you want each host to
accept inbound mail for all of the cluster members. Simply enter the names of
the cluster members into the local aliases file in UCX$SMTP_COMMON.
--------------------
UCX SMTP Configuration database entry (UCX SET CONFIG SMTP)
--------------------
This section suplements the other UCX SMTP documentation on the UCX SMTP
CONFUGRATION database entry.
---------------
Postmaster field
---------------
There are too many bugs that arise when users try to change the Postmaser field
in the UCX SMTP configuration database and it's inclusion as a configurable item
was at best poorly executed and at worst ill-conceived. Therefore
- the account that UCX SMTP runs in is UCX_SMTP.
- The Postmaster field is a deprecated field.
- Leave it alone. Ie. Leave it as UCX_SMTP.
- Do not muck around with the UCX_SMTP account unless you have been asked to
do so by Digital.
All sites should also set forward for two key SMTP usernames to an account which
is read regularly. The system account is the most likely candidate. Eg:
MAIL> SET FORWARD/USER=UCX_SMTP SYSTEM
MAIL> SET FORWARD/USER=POSTMASTER SYSTEM
Setting forward for UCX_SMTP to SYSTEM means that any messages which could
neither be delivered to the recipient nor bounced back to the sender are
delivered to the SYSTEM user.
--------------------
UCX SMTP configuration logicals
--------------------
This is to document the SMTP configuration logicals which, heretofore, have been
documented only in release notes.
Notes about SMTP logicals:
- SMTP logicals governing the behavior of the symbiont are translated at
queue startup time. If the logical value is changed or deassigned then the
queue must be stopped and started in order for the change to take effect. Ie.
you need to do UCX STOP MAIL, define your logicals, then do UCX START MAIL.
- SMTP logicals governing the behavior of the receiver are translated when
the receiver image starts up. Since a new receiver process is started for
each inbound connection to port 25 there is nothing to start or stop. The
logicals take effect for the next receiver process created.
- Some SMTP configuration logicals are "boolean" in nature in that they are
associated with an SMTP configuration option that is either on or off. If
the logical is defined then the option is considered to be on. If it is not
defined then the option is off. The actual *value* of the logical is not
examined by the symbiont although by convention these logicals are defined to
"1". Defining the logical to "OFF" or "NO" or "0" will still have the effect
of turning the option on! To turn the option off don't define the logical at
all.
- The SMTP configuration logicals must be defined as /SYSTEM except where
noted.
Here is a list of all of the UCX$SMTP logicals. Boolean logicals are marked as
such.
********************************************************************************
* Diagnostic logicals
********************************************************************************
Name: UCX$SMTP_LOG_LEVEL
Type: Numeric
Values: 2,3 or 5
This logical tells the SMTP symbiont what diagnostic information to write to the
log file. It can have one of three values, 2, 3 or 5 with 2 being the least
verbose and 5 being the most. The values are cumulative in that all messages
printed out by one logging level are printed out by each of the more verbose
logging levels. A value of "3" is recommended.
Value "2"
- All of the SMTP configuration info is logged when the symbiont starts
up.
- The "next open file" message is printed giving the name of each control
file before processing on it begins.
- All mail headers and mail recipients in a control file are logged after
control file processing is complete.
Value "3"
- Logs additional information about symbiont initialization and symbiont
activity during control file processing without being overly verbose.
Value "5"
- Full symbiont diagnostics. For use only when diagnosing problems. Much
of this will only be of use to Digital engineering staff but a diligent
user can still get a sense of what's going on.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_NOSEY
Type: Boolean
Values: NA
This logical tells the symbiont to print out the Subject RFC header when it logs
the RFC headers if UCX$SMTP_LOG_LEVEL is defined. If this logical is not defined
then the Subject header is logged as "Subject: <omitted>".
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_LOG_LINE_NUMBERS
Type: Boolean
Values: NA
If defined line numbers are written to UCX SMTP logs. This includes the
symbiont, receiver and MAIL$PROTOCOL (DEBUG.TXT) logs.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_SYMB_TRACE
Type: Boolean
Values: NA
This logical tells the symbiont to log all messages received from and
transmitted to remote SMTP servers. It allows you to trace the SMTP application
layer protocol. Any unprintable and/or control characters sent/received are
printed as \n where 'n' is the hex value of the character. For example, command
lines and replies are terminated with a <CR><LF> which would appear in the log
like this:
send buf=MAIL FROM:<[email protected]>\d\a
recv buf=250 <[email protected]>... Sender OK\d\a
The "\d\a" is the <CR><LF>. CR is hex 'd' and LF is hex 'a'.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_RECV_TRACE
Obsoletes: UCX$SMTP_PROTO_TRACE
Type: Boolean
Values: NA
This logical tells the receiver to log all messages received from and
transmitted to remote SMTP clients. The same conventions for logging unprintable
and/or control characters are used.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_RECV_DEBUG
Obsoletes: UCX$SMTP_PROTO_DEBUG
Type: Boolean
Values: NA
This logical tells the receiver to log full diagnostics. It performs a similar
function to the symbiont's UCX$SMTP_LOG_LEVEL 5.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_VMSMAIL_SEND
Type: Boolean
Values: NA
If this logical is defined the MAIL$PROTOCOL will log diagnostics to a file
named DEBUG.TXT in the default directory. Much of this will only be of use to
Digital engineering staff but if you are familiar with the MAIL$PROTOCOL
interface you can see what is coming into the SMTP MAIL$PROTOCOL code. If you
are debugging the mail$protocol code running in a user process sending mail
then this logical need not be a system logical. If you are debugging the
MAIL$PROTOCOL code running when DECnet mail comes in then it's running in the
context of the DECnet Mail11 network object and so the logical needs to be a
system (or group logical).
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_VMSMAIL_PARSE
Type: Boolean
Values: NA
This causes the UCX SMTP address parsing code to log diagnostics. The symbiont,
receiver and the MAIL$PROTOCOL code are all sensitive to this logical. This
logical is primarily for the use of Digital engineering staff.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_LOG_MEMORY, UCX$SMTP_LOG_EFS
Type: Boolean
Values: NA
These options are primarily for use of Digital staff in tracing memory leaks and
event flag leaks.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_SYMB_SNAPSHOT_BLOCKS and UCX$SMTP_RECV_SNAPSHOT_BLOCKS
Type: Numeric
Values: 1-1000
In cases where errors are being signalled in the symbiont or receiver logs
intermittently and are not reproducable at will it is often not desirable to
simply run with full diagnostics turned on and wait for the error to occur
again. This wastes disk space creating huge logs and slows down processing of
mail dramatically.
The symbiont and receiver have a feature called "snapshot logging" which allows
you to run with full diagnostics turned on but only write the diagnostics to the
log file if an error is signalled. This saves disk space and allows the receiver
and/or symbiont to run at a normal speed. As each line of diagnostic text is
generated it is saved in an internal "snapshot" buffer rather than to the disk.
The buffer is "circular" in that once it fills up, new lines of text simply
start to overwrite the old data already there. This provides an up to date
snapshot of the last "so many" lines of diagnostic text.
There are two logicals to turn this feature on. One for the symbiont and one for
the receiver. They are respectvely UCX$SMTP_SYMB_SNAPSHOT_BLOCKS and
UCX$SMTP_RECV_SNAPSHOT_BLOCKS. The value you assign to these logicals is the
desired size of the snapshot buffer in VMS blocks (1 block being 512 bytes).
Defining either one of these logicals for the given component(s) (symbiont
and/or receiver) turns on the snapshot logging function.
When turning this feature on you are still required to define the other SMTP
diagnostic logicals that tell UCX SMTP what types of logging you want. (EG. For
the symbiont you still want to define UCX$SMTP_LOG_LEVEL to 5.) Here are two
examples of setting up symbiont and receiver snapshot logicals.
This sets log level to 5 and turns on snapshot logging for the SMTP symbiont
with a snapshot buffer of 200 blocks.
$ DEFINE/SYSTEM UCX$SMTP_LOG_LEVEL 5
$ DEFINE/SYSTEM UCX$SMTP_SYMB_SNAPSHOT_BLOCKS 200
This sets all of the receiver diagnostics on and turns on snapshot logging for
the receiver with a snapshot buffer of 200 blocks.
$ DEFINE/SYSTEM UCX$SMTP_RECV_DEBUG 1
$ DEFINE/SYSTEM UCX$SMTP_RECV_TRACE 1
$ DEFINE/SYSTEM UCX$SMTP_RECV_SNAPSHOT_BLOCKS 200
Note that when you have snapshot buffering turned on for the symbiont it takes
some time for the symbiont process to stop when you do a UCX STOP MAIL or stop
the queue. This is because stopping the queue causes the symbiont to signal the
"Symbiont stopping" condition which is not an error but which still causes the
snapshot buffer to be dumped. The amount of time writing out the snapshot buffer
takes depends on its size and the speed of the system and its disks. When the
queue state is "Stopped" rather than "Stopping" the symbiont is process is
really stopped.
********************************************************************************
* Miscellaneous Configuration logicals
********************************************************************************
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_NO_SUBS_DOMAIN_INBOUND
Type: Boolean
Values: NA
Tells UCX SMTP not to consider mail sent to the substitute domain as local.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_COMMON
Type: String
Values: NA
This logical specifies cluster common directory where certain UCX SMTP files can
be placed by the system manager. UCX SMTP will now look for .DIS files and for
the local alias file in this directory if it is defined. It is the system
manager's responsibility to define this logical before UCX SMTP startup and to
create the directory and to move the .DIS files and/or local alias file there.
If UCX$SMTP_COMMON is not defined then UCX SMTP will look for the files in
SYS$SPECIFIC:[UCX_SMTP].
This logical may be a search list. You may wish to have UCX SMTP look at the
clusterwide directory first and SYS$SPECIFIC:[UCX_SMTP] second. For example:
$ DEFINE/SYSTEM UCX$SMTP_COMMON WORKDISK:[SMTP_DIS], -
SYS$SPECIFIC:[UCX_SMTP]
Note that other files like control files and logs are still kept in
SYS$SPECIFIC:[UCX_SMTP] directory.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_JACKET_LOCAL
Type: Boolean
Values: NA
This option affects the VMS mail From: line of mail delivered by the SMTP
symbiont. The SMTP symbiont contains logic to detect when SMTP mail that
originated on the local host is also being delivered to VMS mail on the local
host. In this case the From: line text that the symbiont passes to VMS mail will
contain only the username rather than SMTP%"username@host". Put another way, the
symbiont won't put the VMS mail From: line in the SMTP% jacket on local-to-local
mail. Since the VMS mail From: line contains only the username it is impossible
to tell that the mail message went through SMTP simply from looking at the VMS
mail From: line. This causes problems for the POP server which looks at the
From: address when it's deciding whether or not it can safely ignore the VMS
mail headers and, instead, use the SMTP RFC headers from the body of the mail.
The UCX$SMTP_JACKET_LOCAL option tells the symbiont to put the SMTP jacket on
local-to-local mail to short circuit the above described special case handling
of local-to-local mail.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_INBOUND_NOXVMS
Type: Boolean
Values: NA
This option affects the VMS mail To: and CC: lines of mail delivered by the SMTP
symbiont. It tells the symbiont not to use the RFC X-VMS-To header (if any) as
the text of the VMS mail To: line and the X-VMS-CC header (if any) as the text
of the VMS mail CC: line. Instead the RFC To: and CC: lines are used. If
UCX$SMTP_INBOUND_NOXVMS is *not* defined the SMTP symbiont will use the text of
the X-VMS-To and X-VMS-CC lines (if present) for their respective VMS mail
header lines.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_VMSDEF_TO
Type: Boolean
Values: NA
This option affects the VMS mail To: line of mail delivered by the SMTP
symbiont. It tells the symbiont not to pass any text for the To: line to VMS
callable mail when delivering local mail. This causes VMS callable mail to use
the default text for the To: line which will be the username. This option
overrides UCX$SMTP_INBOUND_NOXVMS for the To: field.
********************************************************************************
* Special case logicals for MTS interaction
********************************************************************************
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_MTS_ALLIN1
Type: Boolean
Values: NA
When relaying mail from the SMTP environment to MTS (aka. Message Router) the
symbiont puts "UCX_SMTP" into the From field. This hack, the "UCX_SMTP hack",
was needed because without it older versions of MR/MRGATE would purposefully
bounce the mail with a "Return path too complicated" error.
The UCX_SMTP hack is no longer needed provided that you are running MR V3.3A and
MRGATE V3.3A and either UCX V3.2 ECO 8 or UCX V3.3 ECO 1 or UCX V4.0. Version
3.3A of MR/MRGATE will not bounce the mail and will pass it on to VMSmail with
the nested double quotes. VMSmail will not reject the mail because of the nested
double quotes in the from field but *the mail will be unreplyable*. With a
future version of VMS mail it will be replyable. That is, nested double quotes
will be supported by VMS mail.
The UCX$SMTP_MTS_ALLIN1 option disables the UCX_SMTP hack. This will allow the
correct From: address to be received by your All-in-1 users.
In summary, if you want to short circuit the UCX_SMTP hack then you need to
upgrade MR and MRGATE to version V3.3A and you need to define the
UCX$SMTP_MTS_ALLIN1 logical before SMTP queue startup.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_REWRITE_MTS_FROM
Type: Boolean
Values: NA
Many sites make heavy use of All-in-1 and MTS. They have all or many of their
users forwarded to All-in-1 via MTS addresses. For example:
MAIL> !All-in-1
MAIL> set forwa/user=bjarne MRGATE::A1::BJARNE
This setup can cause problems when an SMTP mail comes in from the Internet, is
forwarded to MTS and is replied to from the all-in-1 account. Either the reply
is bounced once it makes its way onto the Internet or the mail makes it to the
other end but can't be replied to because the return address has been corrupted.
To fully understand what the trouble is an example is required. Say there is a
UCX SMTP system with the host name xyz.com and on this system mail to user
bjarne is forwarded to All-in-1 as above. Now say that user [email protected] sends
an SMTP mail over the Internet to [email protected]. This mail will be relayed to
the UCX xyz.com system and then the SMTP symbiont on xyz.com will hand the mail
off to VMS mail which will hand it off to MTS (because of the forwarding for
user bjarne) which will deliver it to the all-in-1 user on the other system.
Now, say that the user BJARNE on that system decides to reply to the mail. If he
does, the reply is sent via MTS back to the XYZ.COM system. At this point the
SMTP symbiont on xyz.com must form a From RFC header for the mail to be sent
into SMTP and out onto the Internet. It use the UCX SMTP SMTPfication procedure
to do this. So the SMTP From header would look something like this:
From: "MRGATE::SUMNOD::\"MRGATE::SUMNOD::BJARNE\""@XYZ.COM
Once this is done the mail is sent out into the Internet and UCX SMTP is done
with it. The problem is that, although this address is absolutely correct, most
U*ix systems on the Internet do not have sendmail.cf files that can handle such
addresses; the result, bounced mail and/or corrupted addresses. See
"Miscellaneous SMTP/non-SMTP issues".
The UCX$SMTP_REWRITE_MTS_FROM configuration logical allows the system manager to
tell the symbiont that when it tries to form an address in the From field such
as the one above it should parse the simple USERNAME out of the complex MTS
address and append the local host name to that instead. It's a type of simple
address rewriting. This way only a simple address gets sent out on the Internet.
Since the user is forwarding their mail on XYZ.COM to MTS, a reply coming *back*
to this user will be correctly relayed back to MTS.
Given the above example where mail comes from the Internet to user
[email protected] and is forwared to All-in-1 and replied to, the RFC From: header
that UCX SMTP sends onto the Internet for the reply will no longer be:
From: "MRGATE::SUMNOD::\"MRGATE::SUMNOD::BJARNE\""@XYZ.COM
It will now be:
From: [email protected]
Now when the mail goes out into the net the reply path will be simple and cause
no porblems. If the recipient replies the reply will come back to user bjarne on
xyz.com and, because this user has their mail forwarded it will go to MTS and on
to All-in-1 as desired.
There is an important caveat to this feature which requires that it be used with
caution. There is one scenario where it won't work right. Say a user on host
xyz.com has their mail forwarded to SMTP%"[email protected]" and that this
user receives a mail from MTS (say it's from MRGATE::A1::JONES). If the
UCX$SMTP_REWRITE_MTS_FROM logical is defined on xyz.com then user "mary" will
receive the mail on myhost.xyz.com as being from [email protected] because the SMTP
symbiont on xyz.com has parsed out just the username from the MTS From address.
If user mary replies to this it will cause a mail to be sent to [email protected].
In this example note that we have not said whether or not user jones even has an
account on xyz.com at all. If they do then the mail will be delivered to their
account on host xyz.com. If the account is forwarded it will go to where it was
forwarded to. If jones doesn't have an account though, which is likely, then the
mail will be bounced. For this reason this feature is really only for use on
sites that have most or all of their users forwarded to All-in-1 as above.
********************************************************************************
* Non-local mail routing options
********************************************************************************
"Non-local mail" is mail that is destined for another system rather than "local"
mail which is to be delivered locally to VMS mail.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_ALTGATE_ALWAYS
Type: Boolean
Values: NA
This option tells the symbiont to send all non-local mail to the alternate
gateway. No zone check is performed.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_MX_IF_NOALTGATE
Type: Boolean
Values: NA
Tells the symbiont to try to use MX records to get to a host if the alternate
gateway cannot be reached.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Name: UCX$SMTP_NO_MX
Type: Boolean
Values: NA
Ths option tells the SMTP symbiont not to use MX records to tell it where to
route mail. Rather it will try to translate the domain part of each SMTP address
into a host name and send directly to that address. Note that if the host name
doesn't translate to an address the mail will bounce. If the host is not
available then the mail will be requeued.
----------------------------------------
Miscellaneous VMS mail notes
----------------------------------------
--------------------
VMS mail native support for SMTP addresses
--------------------
Version 6.2+ of VMS contains built in support for SMTP type addresses in the VMS
mail To: line field without the SMTP% jacket. Eg: One can do
MAIL> send
To: [email protected]
rather than
MAIL> send
To: SMTP%"[email protected]"
When mail is sent to an SMTP address without the SMTP% jacket it is still
funneled to the MAIL$PROTOCOL interface just as it is when the SMTP% jacket is
used. This new feature makes use of SMTP address more seamless. See VMS
documentation for more information on VMS mail's native support for SMTP
addresses. Also see the UCX User Guide chapter on SMTP.
--------------------
Setting forwarding to an SMTP address
--------------------
On most VMS versions one needs to set forward to an SMTP address as follows:
MAIL> SET FORWARD SMTP%"""[email protected]"""
More recent versions allow:
MAIL> SET FORWARD SMTP%"[email protected]"
or even
MAIL> SET FORWARD [email protected]
See your VMS documentation and release notes or try it yourself to see what your
system supports. If you try it yourself always remember to do a SHOW FORWARD to
see the result. Older systems which expect the triple double quotes on either
side of the SMTP forwarding address will accept SET FORWARD
SMTP%"[email protected]" but will result in a forwarding address of
SMTP%[email protected] which then can't be parsed by VMS mail. What you want is
a SHOW FORWARD result that shows one pair of double quotes around the address.
Eg:
MAIL> sho forw
JONES has mail forwarded to SMTP%"[email protected]"
On clusters with different versions of VMS installed on different cluster
members there's no problem as long as you use the SMTP% jacket and the result of
a SHOW FORWARD is as above. (Ie. one pair of double quotes.)
All currently supported versions of UCX SMTP support users who've done a SET
FORWARD directly to an SMTP address without the SMTP% jacket regardless of which
VMS version is running. This support becomes an issue when UCX SMTP delivers
mail to a user who has in turn forwarded their mail out via SMTP. See
"Forwarding to an SMTP address".
--------------------
Miscellaneous SET FORWARD/USER tricks
--------------------
The VMS mail SET FORWARD/USER command allows one to create "aliases" for many
different purposes. Its syntax is:
MAIL> SET FORWARD/USER=alias-string forwarding-address
The /USER qualifier requires SYSNAM to prevent unprivileged users from
malisciously changing forwarding for other users. In the case where the SET
FORWARD/USER capability is needed by an unprivileged user the user must simply
request that the forward be set up on their behalf by a privileged user.
The key to the SET FORWARD/USER command is that the alias-string need not be a
username. Nor is it constrained by VMS username sytnax. The value can be up to
31 characters (compared with VMS username limit of 12 characters).
Here are some examples of functionality that can be gained through use of the
SET FORWARD/USER command and UCX SMTP:
1) Funnel mail for an alias to multiple users without a list file
UCX SMTP supports use of SET FORW/USER to create an alias that points to
an inline list of users. Say for example you wanted to create a "tech
suport" alias for technical support for some reason. You want anyone to be
able to send mail to this user (actually it's not a user, just an alias)
on your system and have it go to all of the the technical support people.
Further say there are at most only two or three peope in this role and you
don't want to set up and SMTP mailing list for it. You can do this:
MAIL> set forw/user=tech-support smtp%"""jones, [email protected]"""
MAIL> sho forw /user=tech-support
TECH-SUPPORT has mail forwarded to SMTP%"""jones, [email protected]"""
Note that you can't do this with plain old VMS mail addresses since VMS
mail doesn't support this. Eg. This can't be done
MAIL> set forw/user=xxxx jones,smith
%CLI-W-NOLIST, list of parameter values not allowed - check use of
comma (,) \JONES,SMITH\
But you can do it like so:
MAIL> set forw/user=xxxx SMTP%"""jones,smith"""
MAIL> sho forw/user=xxxx
XXXX has mail forwarded to SMTP%"jones,smith"
2) Use Firstname.Lastname syntax
The alias-string can contain a dot (.) which makes it possible to receive
mail where the local-part of the address contains a dot. One use of this
is the Firstname.Lastname syntax. Eg:
MAIL> SET FORWARD/USER=Beth.Jones SMTP%"""[email protected]"""
3) SHOW FORWARD/USER takes wildcards.
MAIL> sho forw/user=j*
Username Forwarding address
JIM_KIRK ENTPRS::SPOCK
JOE.BLOW SMTP%"[email protected]"
JONATHAN.QUINCY.HORNBLOWER SMTP%"[email protected]"
JONES SMTP%"[email protected]"
--------------------
The UCX SMTP cent sign hack
--------------------
Many SMTP address formed by UCX SMTP for the RFC 822 From field use the double
qoute character. For example:
From: "SYSTM1::SMITH"@xyzcorp.com
This has historically posed a problem when it comes time to put the RFC 822 From
header into the VMS mail From: line. To put the RFC 822 From: address into the
VMS mail From: line VMS mail (for most of its life) has required the address to
be enclosed in the SMTP% jacket. It also has required that the address in the
SMTP% jacket is itself enclosed in double quotes. Eg:
From: SMTP%"[email protected]"
^^^^^^ ^
VMS mail || |
requires || |
all of || |
this --->--++----->------>--+
Finally, VMS mail (until only very recently) hasn't allowed nested double
quotes. **
Given all this it was necesary to contrive some means to slip nested double
quotes by VMS mail in the VMS mail From: line. If this wasn't done mail from
SMTP with SMTP From: addresses containing double quotes would not be replyable
since the VMS mail From: line would not be syntactically correct. To "mask" the
nested double quotes UCX SMTP changes each one into a cent sign character before
passing it to VMS callable mail as the VMS mail From: line. This has come to be
known as the cent sign hack (not to be confused with the percent sign hack).
Now an example to illustrate. The RFC 822 From header of:
From: "SYSTM1::SMITH"@xyzcorp.com
becomes the VMS mail "cent sign hacked" From: line of:
From: SMTP%"�SYSTM1::SMITH�@xyzcorp.com"
If a reply is done to this what happens? VMS mail invokes the UCX SMTP
MAIL$PROTOCOL routine and passes this recipient address:
�SYSTM1::SMITH�@xyzcorp.com
When parsing the recipient address UCX SMTP's MAIL$PROTOCOL code changes the
cent signs back to double quotes and then continues normally.
** Well, more recent versions of VMS mail (VMS V7.?) have added support for
nested double qoutes. It's in the UCX SMTP worklist to change our code so
that you can either use the cent sign hack or some new code to properly nest
double quotes. (This will be configurable so UCX SMTP can contiue to function
with older versions of VMS mail that don't support nested double quotes.)
---------------
User use of the cent sign hack
---------------
Although the cent sign hack was invented for the reason above it can also be
employed by users who need to send mail to an SMTP address that requires double
quotes. (This is a rare occurence but it is still supported.) You can
purposefully compose and address like this providing you know how to use your
keyboard's compose key. The compose sequence for the cent sign is "C|". Working
in a DECterm the use of the compose key is non-intuitive. To type a cent sign
you need to
1) Hold down the compose key and then touch the space bar and release both
keys.
2) type the c key
3) type the | key
And voila - �
----------------------------------------
UCX SMTP mailing lists
----------------------------------------
A mailing list is a list of addresses contained in a file that can be sent to as
if it were a simple username. In UCX SMTP the name for a mailing list is the
name of the file itself. UCX SMTP mailing list files can reside in onw of two
places - either SYS$SPECIFIC:[UCX_SMTP] or UCX$SMTP_COMMON. The file must have a
.DIS extension and should be readable by the UCX_SMTP account. @@@
The entries in the local alias file must adhere to the following syntax rules.
- There may be only one address entry per line.
- The address must be SMTP addresses, not VMS addresses.
- No SMTP% jacket
- To get DECnet address in a UCX SMTP.DIS file enter it as:
<"NODNAM::USER">
The <" and "> are required.
- The comment character, a "!", must appear in the first column.
--------------------
VMS Majordomo
--------------------
Majordomo is a popular Unix based mailing list manager program it. It automates
maintenance of lists to a large degree. It also pre-processes each mail sent to
this list and allows for:
- Canned headers and text to be appended to each message sent to the list.
- Users to subscribe and unsubscribe themselves from the list.
- Moderated lists
- Digests
With a digest messages sent to the list are saved and compiled into a
single message - a "digest" - and then mailed out as one large mail
message.
- Archives
- users can see directories "served" by Majordomo and fetch files via email
Majordomo has been ported to VMS/UCX and is available for use as *unsupported*
software. The get more information simply send a mail message to
[email protected]
The subject of the mail doesn't matter. In the body of the mail enter
info vms-majordomo-users
----------------------------------------
Miscellaneous Stuff
----------------------------------------
--------------------
Queue watcher program
--------------------
In cases where queues are crashing it is desirable to have a workaround to let
mail keep flowing without constant human intervention. UCX SMTP provides such a
feature in the form of an unsupported "queue watcher" program called
UCX$RESTART_SMTPQ.COM which is supplied, as is, in the UCX$EXAMPLES directory.
This procedure file watches your SMTP queues and restarts them it it finds them
crashed. It runs as a self-perpetuating batch job. Among it's features are:
It won't restart your SMTP queues when you have shut down UCX SMTP.
It will find the likely "culprit job" that caused the queue to crash and put
that job on hold so that its simply doesn't come back into the queue and
crash the queue again.
It will optionally check the symbiont process's page file count and
gracefully stop it if it is running low. (This is the case where there is a
memory leak.) After the queue stops it will restart the queue transparent to
you. Stopping and restarting the queue stops the old symbiont process and
starts a new one with a fresh, full sized page file count again. This lets
mail keep flowing in spite of memory leaks. being incurred in the symbiont
process.
The UCX$RESTART_SMTPQ.COM procedure file contains instructions on its use.
--------------------------------------------------------------------------------
Terms
--------------------------------------------------------------------------------
address: See mail address.
body: The part of the mail after the headers, separated from
the headers by a single blank line. See "SMTP Message
Format Basics" and "Format of VMS mail message".
bounced mail: A mail message composed by an MTA indicating that
another message which the MTA was charged with
delivering couldn't be delivered. A bounced mail message
(aka. undeliverable mail notification) may contain
sections:
- that show what went wrong (often an SMTP protocol
dump or an error returned by the MTA's host mail
system),
- that show which addresses were successfully
delivered and which weren't
- that contain the text of the undelivered mail
message including the headers.
cent sign hack: The UCX SMTP practice of changing a double quote in an
SMTP address to cent sign character (�) before passing
it to VMS mail as the From: line to get around VMS
mail's inability to handle nested double quotes.
CF: Acronym for UCX SMTP control file.
control file: The binary file into which UCX SMTP packages up each
SMTP mail message. See "UCX SMTP Control File".
<CRLF> The sequence of ASCII characters composed of a carriage
return (decimal 13) followed by a line feed (decimal
10).
domain: The part of a mail address after the "@". Also called
the right hand side or the host. In the address
"[email protected]" the string "XYZCorp.COM" is the
domain. See "SMTP Mail Addresses".
envelope: The source and destination addresses sent in the SMTP
protocol's MAIL FROM and RCPT TO commands. The
destination address or addresses from the RCPT TO
command(s) may match certain headers contained in the
mail message but are not required to. The Return-Path
header appended to the mail message's headers at
delivery time must take the address of the MAIL FROM
command. See "SMTP Protocol Overview".
foreign transport: A VMS mail term. A foreign transport is a mail transport
other than VMS mail's native MAIL11 protocol through
which users want to send mail. To cause a mail message
to be sent over a foreign transport the foreign
transport mail address in the VMS mail To: line is
prefixed with a string which is the name of the foreign
transport and a percent sign (%). Common examples are
SMTP% and NM%. See "VMS mail Foreign Transport/
MAIL$PROTOCOL Interface".
headers: The initial lines of a mail message which mus comply
with a pre-defined format. SMTP and VMS mail define
different legal headers. For information on SMTP headers
(aka. RFC 822 headers) see "SMTP Message Format Basics".
For information on VMS mail headers see "Format of VMS
mail message". The syntax of SMTP headers is specified
fully in RFC 822.
local-part: The part of a mail address before the "@". Also called
the left hand side or the user. In the address
"[email protected]" the string "jones" is the
local-part. See "SMTP Mail Addresses".
local mail: Any mail message being sent to an address where the
domain part of the address is such that UCX SMTP must
deliver the mail locally via callable VMS mail to the
local-part of the adrdress. See "Local vs. non-local
SMTP mail".
Mail11: The VMS/DECnet mail protocol.
MAIL$PROTOCOL Interface: The procedure call interface that VMS mail uses to pass
a mail message to a foreign transport.
mail message: The mail text or "payload" that is transfered by the MTA
and delivered by the MUA to the destination mail
address. It is not to be confused with the "envelope".
The text of the mail message consists of the headers
followed by a blank line followed by the body. See
"header" and "body".
The format of a mail message is specified in RFC 822 and
enhanced by the MIME RFCs. See "SMTP Message Format
Basics" and "What is MIME?".
mail address: The text string used to represent a point to which an
MTA delivers a mail message or to identify the point
from which the mail came. Syntax is defined in RFC 822,
Chapter 6.1. In its simplest form an address appears as
the familiar "user@host". Rather than use the terms user
and host the terms "local-part" and "domain" are used
here.
Mail User Agent: See MUA.
Message Transfer Agent: See MTA.
MUA: The Mail User Agent. The user interface component of a
host's mail system. With it users read, compose and
file their mail messages.
MTA: Message Transfer Agent. The software that implements the
SMTP protocol (client and server) to transfer mail
between an originating host and a destination host.
MX record: A DNS maintained resource record used by SMTP MTA's to
assist in the routing of mail. See "MX Record Overview".
non-local mail: Any mail message being sent to an address where the
domain part of the address is such that UCX SMTP must
relay the message to another SMTP host. See "Local vs.
non-local SMTP mail".
orphan control file: A control file for which there is no corresponding entry
in an SMTP queue. This can occur when a catastrophic
error was incurred while processing the control file
(like a corrupt control file) *or* when the message
could neither be delivered, nor bounced nor delivered to
the UCX_SMTP account. See "Undeliverable Mail".
outbound mail: Synonym for non-local mail.
percent sign hack: Aka. BBN hack, %-hack, See "Source Routing".
reply code: The three ASCII digit string sent at the start of each
reply sent by the SMTP server to the client. These are
explained in detail in Appendix E of RFC 821. See "SMTP
Server Replies".
RFC: Request For Comments. In the Internet world the
specifications that define standards, like the SMTP
protocol, are called RFC's.
RFC 822 headers: See definition of headers.
SMTP% jacket: The SMTP text around an SMTP address that identifies it
as such. For most VMS systems this is the SMTP% plus the
additional double quotes. Eg.
SMTP%"[email protected]"
^^^^^^ ^
|| |
++-<- SMTP% jacket -->--->--+
SMTPfication: The process of transforming any non-SMTP mail address
into an SMTP legal mail address. This is strictly a UCX
SMTP term. See "SMTPfication of non-SMTP addresses".
source routing: The capability for mail addresses to explicitly specify
the route the mail message should take as part of the
address itself. See "Source Routing".
stray control file: Synonym for orphan control file.
| T.R | Title | User | Personal
Name | Date | Lines
|
|---|