22 October 2011. SMS. Wget 1.13.4a for VMS ==================== ------------------------------------------------------------------------ This note accompanies a quick VMS port of Wget version 1.13.4, a GNU free network utility to retrieve files from the World Wide Web using HTTP and FTP. The most obvious features seem to work, but many have not been tested. Built and tested (slightly) in these environments: OpenVMS Alpha V8.3, HP C V7.3-009, TCPIP V5.6 - ECO 5 OpenVMS IA64 V8.3-1H1, HP C V7.3-019, TCPIP V5.6 - ECO 5 OpenVMS VAX V7.3, Compaq C V6.4-005, TCPIP V5.1. No testing has been done on VMS versions before V7.3, and a variety of problems may be expected in any environment where __CRTL_VER is less than 70000000 (non-POSIX exit() behavior, for example). ------------------------------------------------------------------------ Home source URLs ---------------- http://www.gnu.org/directory/wget.html http://ftp.gnu.org/gnu/wget/ ftp://ftp.gnu.org/gnu/wget/ ------------------------------------------------------------------------ News ---- Version 1.13.4a 2011-10-22 --------------------------- - General code update to Wget 1.13.4. (Many old VMS-specific changes have been incorporated into the main Wget code stream.) - Fixed some VMS-specific build problems involving the use of in [.src]connect.c. - Restored some lost VMS-specific changes affecting binary v. text attributes on files transferred by FTP. ([.src]ftp.c) - Restored a lost change which caused permissions always to be restored on files transferred by FTP, even when "--preserve-permissions" was not specified. (Oops.) The "--preserve-permissions" option was erroneously marked as "deprecated", and omitted from the "-h" (--help) usage text. ([.src]ftp.c, [.src]init.c, [.src]main.c) - Changed on VMS to create FTP listing files (".listing") with text attributes, not binary. Wget operation should be unchanged, but listing files saved by using "--no-remove-listing" should now be more compatible with normal text editors (Stream_LF instead of fixed-512 record format). ([.src]ftp.c) - The C macro SYSTEM_WGETRC was being used even when it was not defined, causing a %CC-E-UNDECLARED built-time error. ([.src]init.c) - Changed to avoid explicitly closing the log file when it's stderr. ([.src]log.c) - A defect in the option parsing code could cause a %SYSTEM-F-ACCVIO run-time error when any short-form command-line option was specified. ([.src]main.c) - Changed main() to use exit() instead of "return" at program end, to ensure consistent (POSIX) exit status values. ([.src]main.c) - A change in an OpenSSL function argument type after OpenSSL version 0.9.8 could cause a %CC-W-NOTCONSTQUAL build-time warning when using OpenSSL version 0.9.8 (which includes HP SSL V1.4). Adding a non-"const" type cast seems to satisfy everyone. ([.src]openssl.c) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Version 1.11.4a 2008-11-07 --------------------------- - General code update to Wget 1.11.4. - Changed to avoid trying "LIST -a" for a VMS FTP server. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Version 1.10.2c 2006-11-16 --------------------------- - Changed the code which determines the FTP server type to avoid an ACCVIO if no text was returned in response to a "SYST" inquiry. - Fixed a command-line parsing problem with the "--ftp-stmlf" option. - Fixed an unclosed comment (again?) in FTP-LS.C which could have caused problems with ODS5 extended file names. - Changed VMS-WGET.COM to assume that nothing special is now required for use with the MultiNet IP package. For obsolete VMS and/or MultiNet versions where the old MultiNet-specific changes are still needed, specify "MULTINET" on the VMS-WGET.COM command line. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Version 1.10.2b 2004-12-15 --------------------------- - Changed the "-V" version report to include some host type information. - Changed the I/O scheme on VMS for progress dots to avoid spurious newline characters when output is directed to a file. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Version 1.10.2a 2004-10-13 --------------------------- - General code update to Wget 1.10.2. - Many legitimate FTP CWD operations on a VMS FTP server failed. The typical symptom was "No such directory `perfectly/vaild/directory'." - FTP CWD operations on a VMS FTP server now end with a "CWD []". For an FTP URL like "ftp://host/a/b/c/d.e" (and contrary to the RFC), Wget uses a UNIX-like directory specification in its CWD command, here, "CWD a/b/c". This was observed to switch a TCPware FTP server (V5.6-2) into its UNIX emulation mode. Because Wget checks the system type only once at the beginning of a session, the associated change in directory listing format confused the program. The typical symptom was "No such file `-rwx---r-x'." The added "CWD []" command resets the TCPware server to VMS mode, and should be harmless on other VMS FTP servers. - VMS-WGET.COM now finds MMK when it's on DCL$PATH as well as when a foreign command symbol "MMK" is defined. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Version 1.10.1a 2004-09-09 --------------------------- - General code update to Wget 1.10.1. - VMS-specific files are now in the new [.VMS] directory. - Changes to VMS-WGET.COM: - New command-line option, LARGE, enables large-file support on non-VAX systems. - VMS-WGET.COM may now be run from any default directory, not only [.SRC]. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Version 1.9.1e 2005-09-09 -------------------------- - The default User-Agent string on VMS now includes the host architecture and VMS version. For example, "Wget/1.9.1e (VMS Alpha V7.3-2)". - Fixed a problem on VMS with an ODS5 destination file system when a URL-derived file name contained an invalid character ("?" or "*"). A typical complaint might look like: Cannot write to `www.foraddift.non/aspx/index.asp?sid=463e' (file specification syntax error). Now, "?" is converted to "!", and "*" to "#". - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Version 1.9.1d 2005-06-21 -------------------------- - Replaced the remaining instances where ".orig" was appended to a file name. Now (for VMS), "_orig" is used everywhere (except in the documentation). - Disabled (for VMS) the code which appended ".nnn" (decimal "nnn") to an output file name to avoid overwriting an existing file. The file name is now used unchanged, and VMS file versioning is expected to prevent overwriting. (The multi-dot file names made this way caused I/O errors on ODS2 file systems.) - Changed to do translation of CR-LF line endings to local line endings for ASCII FTP transfers. - Changed (on VMS) to use fixed-512 format for binary FTP files. Added the new "--ftp-stmlf" option ("ftpstmlf on" command) to let the user restore the old behavior, where Stream_LF was used for both ASCII and binary. - Added (on VMS) a new open callback function which senses (where supported) the process RMS_DEFAULT values for file extend quantity (deq), multi-block count (mbc), and multi-buffer count (mbf), and sets the FAB/RAB parameters accordingly. The default deq is now much larger than before (16384, was none), and the default mbc is now 127 (was none), speeding creation of a large output file. Explicitly set RMS_DEFAULT values override built-in defaults. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Version 1.9.1c 2004-12-15 -------------------------- - Added https (SSL) support, changing DESCRIP.MMS and VMS-WGET.COM, and adding WGET_SSL.OPT and WGET_SSL_HP.OPT. Limited testing was done on Alpha, using HP SSL V1.1-B and OpenSSL version 0.9.7e. The build procedure automatically enables SSL support if it can find one of the SSL header files. The details are explained in comments in VMS-WGET.COM, including its new command-line options, HPSSL and NOSSL. - Changed CONNECT.C to fix some problems with MultiNet TCP/IP software. - A build problem related to header files (netdb.h) when compiled with MultiNet. Typical symptom: %CC-E-REDEFSTRUCT, In this declaration, the struct "hostent" is redefined. at line number 178 in module NETDB of text library SYS$COMMON:[SYSLIB]DECC$RTLDEF.TLB;1 - A run-time problem with an FTP URL. Typical symptom: accept: bad address - Changed FTP.C to fix various problems with time-stamping ("-N", "--timestamping") when using an FTP URL. Some of the misbehavior was specific to a VMS FTP server, but not all. The typical result was a file with the modification date (and, in some cases, the creation date) left at the date of the download, rather than set to match the date on the server. Some misleading diagnostic messages also have been corrected. - Changed FTP-LS.C to fix a typographical error (unclosed comment). - Changed NETRC.C to fix a potential build problem on non-VMS systems. Adding some VMS-specific code introduced some improper statement ordering, which may generate complaints (or failure) with some C compilers. - Changed VMS.C to fix a problem with the utime() function (supplied with Wget) when the user specified a non-UNIX file name with the Wget "-O" option. Typical symptom: utime(home_sms:[sms.wget]t.h): error 0 As the message might suggest, the file date-time was not being set correctly. With this fix, utime() should work with either a UNIX-like or VMS-like file specification, and it should also set errno properly when it fails, enabling a more informative error message. - New MMS/MMK description files and associated command procedures enable automatic source dependency generation. Changed: DESCRIP.MMS and VMS-WGET.COM. New: DESCRIP_DEPS.MMS, DESCRIP_MKDEPS.MMS, DESCRIP_SRC.MMS, COLLECT_DEPS.COM, and MOD_DEP.COM. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Version 1.9.1b 2004-09-28 -------------------------- - Changed CONFIG.H_VMS to fix a build problem with C RTL before VMS V7.3-1 with C Runtime ECO V3.0 on Alpha: %LINK-W-NUDFSYMS, 2 undefined symbols: %LINK-I-UDFSYM, DECC$GXSNPRINTF %LINK-I-UDFSYM, DECC$GXVSNPRINTF [...] - Changed VMS.C: - Fixed a build problem with C RTL before V7.2: %CC-E-UNDECLARED, In this statement, "DVI$C_ACP_F11V5" is not declared. at line number 262 in file disk:[xxx.WGET-1_9_1A_VMS.SRC]VMS.C;1 - Fixed a build problem with C RTL before VMS V7.3 with C Runtime ECO 2 on Alpha: %CC-I-IMPLICITFUNC, In this statement, the identifier "decc$feature_get_index" is implicitly declared as a function. at line number 155 in file disk:[xxx.WGET-1_9_1A_VMS.SRC]VMS.C;1 %CC-I-IMPLICITFUNC, In this statement, the identifier "decc$feature_get_value" is implicitly declared as a function. at line number 159 in file disk:[xxx.WGET-1_9_1A_VMS.SRC]VMS.C;1 %CC-I-IMPLICITFUNC, In this statement, the identifier "decc$feature_set_value" is implicitly declared as a function. at line number 169 in file disk:[xxx.WGET-1_9_1A_VMS.SRC]VMS.C;1 and the related: %LINK-W-NUDFSYMS, 3 undefined symbols: %LINK-I-UDFSYM, DECC$FEATURE_GET_INDEX %LINK-I-UDFSYM, DECC$FEATURE_GET_VALUE %LINK-I-UDFSYM, DECC$FEATURE_SET_VALUE [...] - Fixed a build problem with Multinet involving use of both the Multinet-specific and generic netdb.h header files: %CC-E-REDEFSTRUCT, In this declaration, the struct "hostent" is redefined. at line number 178 in module NETDB of text library SYS$COMMON:[SYSLIB]DECC$RTLDEF.TLB;1 - Fixed a typographical error (unclosed comment) which kept the C RTL feature DECC$EFS_CHARSET from being enabled as intended. This caused a run-time problem on non-VAX systems with some ODS5 extended file specifications with multiple dots in file names, typically when "-m" ("--mirror"), "-O" ("--output-document"), or "-r" ("--recursive") was used: hostname/test/a.B.c: file specification syntax error or test/a.B.c: non-translatable vms error code: 0x186D4 rms-f-syn, file specification syntax error - Fixed a spurious diagnostic message when doing HTTP and complex FTP transfers: utime(). sys$parse() = 00010001. - Changed FTP.C to fix a variety of problems encountered when using a VMS FTP server. These problems (not seen in Wget 1.5.3h) were caused mostly by inappropriate code added (in the main Wget development stream) to "help" deal with a VMS FTP server. Removing this code and reforming the procedure used for getting to relative directories seems to have fixed the problems. As before, a single-slash URL ("ftp://host/one/two/name.type") will be treated as specifying a relative directory ("one/two"), while a double-slash URL ("ftp://host//one/two/name.type") will be treated as specifying an absolute directory ("/one/two"). As usual, the VMS FTP server will attempt to puzzle out whether "/one/two" should really be interpreted as "one:[two]", but it's better equipped to do it than Wget is. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Version 1.9.1a 2004-04-28 -------------------------- - Changes to VMS-WGET.COM since Wget 1.5.3h: - A separate LIST command-line option now controls compiler listing files. The DBG option no longer implies compiler listing files. The LIST option also generates a link map with cross-reference. - The DBG option may actually work now. - Added support for IA64 architecture. - Changes to DESCRIP.MMS since Wget 1.5.3h: - Compiler listing files (.LIS) and link maps (.MAP) are now put into the system-specific directory with the corresponding object files (.OBJ), namely, [.SRC.ALPHA], [.SRC.IA64], or [.SRC.VAX]. Previously, they were put into the source directory ([.SRC]) itself. - CLEAN target now deletes any .LIS and .MAP files. - New HELP target creates a Wget help library. - The OS_TYPE macro is now defined, including the VMS version. - No more complaints like "%MMS-W-MBREDEFILL, Illegal attempt to redefine macro 'CFLAGS'" on old versions of MMS. - A separate LIST macro now controls compiler listing files. The DBG macro no longer implies compiler listing files. - New support for IA64 architecture. - Built-in initialization of some DECC$* run-time options on non-VAX systems. Currently this behavior is compiled in, so any changes must be made in the VMS.C source file. These changes have no effect on VAX (or older Alpha) systems where the OS and C run-time support is absent. - Command-line case preservation. Wget for VMS enables the following DECC$* run-time option: DECC$ARGV_PARSE_STYLE With extended command parsing enabled ("SET PROCESS /PARSE_STYLE = EXTENDED"), command-line case is preserved. This obviates quoting case-sensitive option switches, URLs, and other parameters. - Extended file names on ODS5 file systems. Wget for VMS enables the following DECC$* run-time options: DECC$EFS_CASE_PRESERVE DECC$EFS_CHARSET Thus, if an ODS5 file system is used as the destination for downloaded files, multiple dots will be permitted and case will be preserved in file names. If an ODS2 file system is used, the same character substitutions will be made as in Wget 1.5.3h, namely, "." -> "_" (where needed), "~" -> "-", and "@" -> "$". ------------------------------------------------------------------------ Changes to Source Files ----------------------- Files new for VMS, in [.lib]: vsnprintf.c GNUlib replacement for vsnprintf(), which is missing on older VMS systems. Files new for VMS, in [.src]: decc_ver.c Diagnostic program to provide build environment information. vms.c Various VMS-specific code: DEC C run-time initialization, ODS2-ODS5 file system detection, ODS2 file name adjustment, file open callback function for better RMS parameters, "utime()" replacement function, and for VMS before V7.0, "getpwuid()" and "localtime_r()". Files new for VMS, in [.lib], [.src]: descrip_deps.mms Source dependency description file. descrip_mods.mms Source module description file. descrip_src.mms Source directory-specific description file. Files new for VMS, in [.vms]: alloca.h Dummy header file. collect_deps.com Dependency collector for descrip_mkdeps.mms. config.h_vms VMS-specific config.h. descrip.mms Main MMK or MMS description ("make") file. descrip_deps.mms Source dependency description file. descrip_mkdeps.mms Source dependency generator description file. descrip_src.mms Source list description file. descrip_src_flags.mms Source compile and link flags description file. errno.h Jacket header file for . find_lib.com DCL procedure used to find a zlib object library. getopt.h GNUlib header file. stdint.h VMS-specific . vms.h VMS-specific header file. For VMS C RTL before V7.3, replacement for . For VMS C RTL before V7.0, replacement for . Declarations of replacement data types/structures and function prototypes for vms.c. vms_ip.h IP software type-specific "#include" directives. vms_notes.txt These notes. wget.hlp Simple help file. wget_ssl_hp.opt LINK options file for HP SSL. wget_ssl_o_olb.opt LINK options file for OpenSSL (.OLB, without "SSL_" name prefix). wget_ssl_o_olb_ssl.opt LINK options file for OpenSSL (.OLB, with "SSL_" name prefix). wget_ssl_o_shr.opt LINK options file for OpenSSL (.EXE, without "SSL_" name prefix). wget_ssl_o_shr_ssl.opt LINK options file for OpenSSL (.EXE, with "SSL_" name prefix). Files changed for VMS, in [.src]: connect.c Added HAVE_ conditionality for and . ftp.c Fixed some binary v. text problems. Restored opt.preserve_perm condition on chmod(). init.c Fixed problems if SYSTEM_WGETRC is not defined. log.c Avoid explicitly closing stderr. main.c Restored lost/"deprecated" material involving "--preserve-permissions". Fixed bad subscript (-1) problem with short-form options. Changed final "return" to exit(). openssl.c Added a type cast to accommodate OpenSSL version 0.9.8. utils.c Corrected a spurious line wrap in a comment. ------------------------------------------------------------------------ Instructions ------------ Extract the files from the distribution kit. Note: The source kit may contain one or more files in the "tests" directory whose names are too long for an ODS2 disk. They will cause errors when the kit is unpacked, but these files are not required on VMS. For example, starting with a Zip archive named "wget-1_13_4a_vms.zip": unzip wget-1_13_4a_vms.zip The MMS/MMK builders typically need some help from the user to enable the desired optional features, and/or to deal with differences between MMK and MMS features. The builder options (MMS/MMK macros) are described in the main builder file, "[.vms]descrip.mms". Normally, "DASHD=1" is specified, to enable the "-d" ("--debug") option. On a reasonably modern non-VAX system, specify "LARGE=1" to enable large-file support. If HTTPS support is desired, then specify either "HPSSL=1" (to use HP's SSL product) or "OSSL=1" (to use OpenSSL). An SSL kit based on an OpenSSL version older than 0.9.8 may cause build problems because it lacks the function "a2i_IPADDRESS()". Thus, the HP SSL V1.3 kit (based on OpenSSL 0.9.7) is probably too old, but the V1.4 kit (based on OpenSSL 0.9.8) should be adequate. An old version of MMK or MMS may need a definition of "MMSDESCRIPTION_FILE". An error message should explain this, if needed. On non-VAX systems, the C compiler may emit some %CC-I-INTCONSTTRUNC complaints when compiling "[.src]hash.c", but these should be harmless. (The VMS system header file declares "uintptr_t" as a 64-bit item ("typedef uint64_t") on any non-VAX system, even in an environment with 32-bit pointers.) On VAX systems, the C compiler may emit some %CC-I-LONGDOUBLENYI complaints when compiling "[.lib]printf-args.c", "[.lib]printf-parse.c", and "[.lib]vasnprintf.c". Executables in pre-built VMS kits are normally built using "DASHD=1" and "LARGE=1" (except VAX), and with no HTTPS (SSL) support. Output from "wget --version" should reveal which SSL product was used to build Wget. "-https" implies no SSL. "+https" with "-ssl" implies HP SSL, and "+https" with "+ssl/openssl" implies OpenSSL. Example build commands: SET DEFAULT [.wget-1_13_4a_vms.vms] ! Required. MMS CLEAN ! Delete all product files ! for the current (small-file) ! architecture. MMS /MACRO = LARGE=1 CLEAN ! Delete all product files ! for the current (large-file) ! architecture. MMS CLEAN_ALL ! Delete all product files ! for all architectures. MMS /MACRO = (DASHD=1, LARGE=1) ! Large-file, no-SSL build. MMS /MACRO = (DASHD=1, LARGE=1, HPSSL=1) ! Add HTTPS support using ! HP SSL. MMS /MACRO = (DASHD=1, LARGE=1, OSSL=1) ! Add HTTPS support using ! OpenSSL. DFLT = F$ENVIRONMENT( "DEFAULT") ! Basic MMK build, with MMK /MACRO = (MMSDESCRIPTION_FILE='DFLT') ! explicit definition of DELETE /SYMBOL DFLT ! MMSDESCRIPTION_FILE. ! ("dev:[dir]" is enough.) MMS HELP ! Create a help library. ! (Optional. See below.) Set the symbol for the foreign command: WGET :== $ actual_device:[actual.directory]WGET.EXE A very basic VMS HELP file is supplied in [.vms]wget.hlp. To be used, it must be added to a HELP library, either an existing library or one just for Wget. As shown in the example above, a new Wget help library may be created using the MMS/MMK target "HELP". When a new HELP library is created, a logical name "HLP$LIBRARY_n" may be defined to allow the HELP utility to find it. See "HELP HELP /USERLIBRARY" for details. Try to read the original documentation to learn how to use Wget. The principal differences in this VMS adaptation are: The "-b" ("background") option is ignored. ("-b" uses "fork()".) Excess dots are replaced by underscores in URL-derived file specifications on ODS2 file systems. For example, on a UNIX system, the URL "ftp://ftp.anywhere.org/x.y.z/a.b.c" could produce the local file "ftp.anywhere.org/x.y.z/a.b.c". With this VMS adaptation, the local ODS2 file would be "[.FTP_ANYWHERE_ORG.X_Y_Z]A_B.C". On an ODS5 file system, the file would be [.ftp^.anywhere^.org.x^.y^.z]a^.b.c". Additional VMS-specific usage information may be found in the VMS HELP file (Source: [.vms]wget.hlp, Help library: [.vms]WGET.HLB) under the topic "VMS_specifics".