Last updated on February 7^th 2002

Contents

Introduction
Compiling and Installing
Wget
Rsync

Introduction

This article introduces a few command line utilities from the Open Source world that can make Solaris a richer, more productive environment for you. It will probably remain in flux as more things are added to it over time.

What kind of things will be presented here? Anything that helps you accomplish tasks under Solaris, like "wget" for example, which lets you retrieve URLs, including files from the Internet to your filesystem. Possibly enhancements or replacements for stock Solaris tools or completely new ones that let you do something that you couldn't - at least easily, before.

All of the applications mentioned in this article are Open Source, or generally under what is known as the "GPL" or "General Public License" (a.k.a. "GNU Public License"). This means that the source code is available to you for use at no cost. Many can be found in binary form, but most will need to be compiled from this source code. As such, you should have a working C compiler on your machine. If not, please read the article, "Installing and Using GCC."

In the event that an application isn't Open Source, doesn't include source code or has any other peculiarities - it will be noted.

Compiling and Installing

If you're familiar with compiling and installing software from source code, you can safely skip this section and move on to the software sections below. This section is for those that are new to adding custom software to their systems.

In the likely event that you'll need to compile an application, it is normally fairly easy to do so providing of course, that you have the prerequisite C compiler mentioned above and any special libraries that the application might need. The method to accomplish this is fairly standard now and any deviations will be noted.

You will start with a "tarball" or "tar" backup of the source code tree which may or may not have been further compressed with "gzip." In the case of the latter, look for an ending such as ".gz" or ".tgz" while in the case of the former, it will simply have ".tar" at the end. Your best bet will be to have both GNU "tar" and "gzip" utilities installed on your system. If you are, the easiest way to extract the source code tree from the tarball is to issue the command:

tar -xzf somefile.tar.gz

This will automatically uncompress and untar the tarball and generate the source code tree, which will usually (but not always) have the same name as the tarball, minus the ".tar, .tgz, .gz" or other extensions in the same directory that you're currently in.

Change into the source code directory and look for a file named "configure." If this file exists, then you can relax knowing that this should be a straightforward, easy install. One thing you'll need to decide (or have decided) by the onset is where you wish to stash all these neat applications. The perennial favorite is the /usr/local/ filesystem, which clearly differentiates things that you've installed on your Solaris box vs. what comes with or has been installed along with Solaris. To do this, you run the configure script with the option, "--prefix=/usr/local" which causes that filesystem to serve as the root for the install. Therefore, a binary or application would then be located under /usr/local/bin while the man pages for instance, are installed under /usr/local/man and so on...

So basically, just type this at the command line to start the process rolling:

./configure --help

What will be output is a list of all available "compile time options" that you can use to control various aspects of the software. In most cases with the software mentioned here there will be very few, if any options and you can simply bypass this part and go right on to the "action" part:

./configure --prefix=/usr/local

You'll see a lot scroll by as configure automatically determines various locations of libraries and such in your environment. Everything should complete successfully and usually there won't be any specific message indicating such, but it should be fairly obvious. Now you need to actually compile the software, and this is done with the "make" command. Again, the GNU replacement for the stock Solaris make might be something you should look into, as it supports multiple CPUs through the use of the "-j X" flag, the "X" being your number of CPUs plus one. This will compile software much, much faster on average. In any event, simply run the make commands as follows:

make [-j3] (e.g. for dual CPU systems)

This will compile the software using any options that you've specified. If all goes well, you can then install the software and begin trying it out. To do this, supply the "install" option to make thusly:

make install

That's it! Now see if your application resides under /usr/local/bin/ or whatever the case may be. If you have other GNU tools or Open Source software installed on your system already, chances are that this filesystem is already in your path as well. No need to specify the full path to it then, just use it like any other command. Enjoy!

Wget

The "wget" utility is much like "curl" and one of my personal favorites for use on development or workstations where I often need to compile software for use on the production servers - which lack any compilers for security reasons. Rather than use an FTP client or other means to retrieve something from a Web or FTP site and then figure out how to get it onto the machine for compilation or have to perform a few steps to get my file - I simply grab a URL to the file I want and copy it to the clipboard. Then at my terminal simply type "wget" and paste the clipboard - finally hitting return. Voila - the file is it's way to the current directory. Take the following line:

wget http://www.apache.org/dist/httpd/apache_1.3.23.tar.gz

This command would download the Apache source code tarball to your current working directory if you wanted to update your Apache software, for example.

Wget lets you retrieve files using HTTP, HTTPS and FTP, the two most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, cron jobs, terminals and so on. Wget has many features to make retrieving large files or mirroring entire web or FTP sites easy. You can resume aborted downloads, use wildcard characters and perhaps best of all, convert absolute links to relative - so that locally mirrored pages refer to each other properly.

The other really useful need that this tool solves is allowing you to mirror a Website to your local filesystem and optionally convert the links as mentioned above. You can then read the site later on, or store it away for backup reasons. Of course, you should check with the Website owner if it isn't yourself first, or they might get a little upset that you're sitting there, sucking down their entire site. If you want to backup your own Website then you're better off using local filesystem-based tools so as to preserve timestamps, file attributes, etc. This is more for mirroring remote, foreign (to you) sites. Still, very useful.

Rsync

Rsync is another indispensible utility that I use often and in a way, is similar to wget above, but is more for file syncronization either locally or remotely rather than mirroring. It uses its own protocol and knows nothing about HTTP or FTP protocols and supports a LOT of options that affect the operation. One of the biggest benefits of rsync is that it works on the "diffs" of files (that is, only the changes between one version of a file and another instead of the entire file) and ignores unchanged files (which can be changed). This makes rsync very, very efficient and fast.

A great use of rsync for example, would be to synchronize a local hard disk or filesystem to another one, be it local or remote. I recently used rsync to make regular snapshots of a Network Appliance (NetApp) RAID mounted over a special, back-end Gigabit Ethernet to a local server hard disk as a sort of backup and in case the back-end network - or the NetApp failed.

Other examples would be to use rsync to synchronize a laptop with your workstation, a development machine with a production server or a local Website tree to the Web server for launch.

There is a separate article that goes into much more detail and specifics about how to use this incredibly handy utility called, "A Tutorial on Using Rsync." You should definately read this if rsync sounds like something for you!