This page documents my port of TCT
to HP-UX 11.00 and 10.20. Please click on the links
below to download the patch and test utilities. Please feel free to report any
problems or observations to the tct-users mailing-list or to knut at acm dot
org. Please observe that the low-level file-system tools like unrm, ils and icat
only work with HFS, HP's incarnation of the Berkeley FFS. HP-UX also knows
the journaling filesystems VxFS and OnlineJFS, the latter being a pay-extra
extension of the former. Both are proprietary products of Veritas Inc. which are
not publicly documented. Therefore it would be very difficult to support them.
Downloads
- patch-tct-1.09-hp3.gz
- 3nd patch release for HP-UX. This release introduces support for 11.00
and fixes a bug that caused grave-robber -I to fail on 10.20. Please download
and use the new release. To patch, unpack
tct-1.09.tar.gz, cd into tct-1.09 and run
gzip -dc ../patch-tct-1.09-hp3.gz | patch -p1
The patch and diff programs that came with HP-UX 10.20 worked very poorly
for me: Diff would just report that there was a new file in the patched tree
but would not generate any further output, so that patch was unable to create
that new file. Patch on the other hand died in an
assertion with a coredump when trying to apply the context diff. So I used
the GNU version of both tools on my Linux box where I mount the HP drive via
NFS. That means there are three options for you to get a reliably patched tree:
- Perform patching on a any box with GNU patch installed and then transfer the patched source to
your HP machine
- Install GNU patch on your HP machine
- Download the full, patched source: tct-1.09-hp3.tar.gz
- patch-tctutils-1.01-hp.gz
- This is a port of Brian Carrier's
tctutils,
you may also download the full, patched source code: tctutils-1.01-hp.tar.gz. Please note that
regretfully the most interesting functionality introduced by tctutils, which is
the ability to retrieve names of deleted files, does not work with HP-UX. In
contrast to other Unices, which keep the name string in the directory
entry of a deleted file and sometimes even the inode number, HP-UX deletes
both, so fls will not report any names of deleted files.
- bigtest-0.6.tar.gz
- Contains two utilities, bigfile generates a (large) file
on stdout consisting of lines that are exactly 1K bytes long and
have a hex and decimal line number at the beginning of each line.
checkbig reads from stdin and reports how many lines it found
and whether all line numbers from 0 to n-1 were found. As one line has exactly
the same size as the standard FFS minimum fragment length, lines should never
be "destroyed" when the file is not allocated contiguously. So checkbig should
find all line numbers in the output file generated by unrm. It also reports the
number of zero 1K areas that it finds, this number should be equivalent to the
space taken up by indirection blocks in the original file.
This is the standard test I
run: Make a new filesystem in an empty partition. Use bigfile to generate a
large file. Then fill the rest of the disk with other files. Finally remove
remove the large file and run unrm against the partition. Then use checkbig to
analyze the output of unrm. All lines of the original file should be found.
In bigfile.h , the maximum file size to be generated/analyzed can be
changed.
The current version 0.6 enhances checkbig to report on the
contiguously allocated chunks of lines it finds. This gives interesting
information about how contiguously the file was stored on disk. Also included
are two perl scripts that convert the output of checkbig into a format that can
be easily plotted with gnuplot. Many thanks to Andreas Thuemmel for developing the
initial version (0.5) and helping with testing in general.
- callme.c
- A small C program for testing pcat. It tries to create substantial amounts
of data in heap and stack, which should be easy to find in the output of
pcat. It also opens /bin/csh readonly and mmaps it, just to create another
memory region of a different type.
Testing
The following tests have been performed by me so far. My test system is a
9000/735 running B.11.00 and B.10.20. I'll happily include further test results
emailed to me. I'm interested in compilation and test results on 11.11. It
should compile and run nicely there. You'll have to install gcc before trying
to compile TCT. Installing lsof is strongly recommended. Binary packages can
be obtained from Software Porting and Archive
Center for HP-UX.
Thanks to Ralf Hildebrandt for pointing out that when compiling with gcc-3.x
on 10.20, several warnings regarding __STDC_EXT__ are generated. By removing
the -D__STDC_EXT__ arguments in the src/*/makedef files, this problem is
solved. On my 10.20 machine, gcc-2.7.2.3 is installed and it requires the
above definition.
ils
- Large UID support: Created a user with UID > 64K
ils lists the inodes of files owned by that user correctly
- Checked with a small FS that ils correctly reports assigned and free inodes.
- List open but deleted files: For some reason, HP-UX removes the
directory entry but does not decrement the refcount from 1 to 0 in the
on-disk inode, so to ils it looks like the file still exists.
I also pressed the reset button once in such a situation just to test how
fsck resolves it: It corrects the refcount from 1 to 0 because it does
not find a direntry for that inode number.
unrm
Complete recovery of a large file, i.e. no block may be
missing, but the order might differ.
The bigfile utility was used to create a file > 2GB on a filesystem that
was generated with newfs -f ufs -o largefiles /dev/[...] After that the
filesystem was filled up and the big file removed. The output of unrm
was successfully analyzed with checkbig to verify that all lines of the original
file were recovered. On 11.00 I repeated the test and also took a close look at
how contiguously the data was allocated on the disk. Click here to see a discussion of the results.
When working with large files, keep in mind that you
have to use a shell that is compiled with large-file support too, because unrm,
bigfile and checkbig work with stdin/stdout, so you have to pipe data to and/or
from them. The original HP-UX /bin/csh is fine, but the tcsh binary I
downloaded from the Porting and Archive Center is not, leading to unexpected
results: It silently breaks a pipe after 2GB have gone through, and you keep
wondering why unrm would only recover 2GB or your file :-)
ils2mac
See grave-robber -i below.
grave-robber
- grave-robber -P
- ps and lsof run fine and produce the expected output
- grave-robber -s
- all system commands run without errors and produce
the expected output. Untested: HPUX does not support ifconfig -a
and therefore I had to implement a workaround that cycles through
the available interfaces. As I have only one physical IF in my
machine, it would be nice if someone else with more IFs could test
this command.
- grave-robber -t
- trust files were copied correctly. As HP-UX does store
per-user crontabs but doesn't allow per-user access to them for
root, crontab files are listed using cat(1). Checked whether
trust/time_trust contains crontab entries.
- grave-robber -m
- verified that the resulting body file can be analyzed
with mactime and that the mactime output is correct, e.g. point
grave-robber to a directory where you recently compiled a large
software package.
- grave-robber -F
- verified that e.g. in /etc all those files
are being copied that are specified by conf_pattern in coroner.cf
- grave-robber -I
- Checked that the right inodes for each process are
copied.
- grave-robber -i
- compile a large sw package, fill up the filesystem
with other files and then remove the package. Then run graverobber
-i and pipe the resulting file in command_out/free_inode_info
through extras/ils2mac. Then run mactime was run to check whether
the MAC times correctly reflected the compilation process.
- grave-robber -p
- Checked that for all current processes a map and
output file is written. Ran pcat specifically on callme.c (see in the
download section above). It contains
easily recognizable strings as constants and in malloc'ed
memory and checked the resulting output file with xd. pcat also
managed to work on the X server process without problems.
The 4 biod's that are normally running return 'no such process'
during ptrace_attach, although they seem to be regular
processes,
i.e. no SYSTEM flag is set.
3 lsof process id's are recorded initially, they run during
suck_lsof, but they do not run any more when pcat()ing, so there
is an error message for them in the error.log.
- grave-robber -o HPUX10 -c corpse -s
- Generated a core dump by pressing
reset button. Copied the root fs with
dump 0f - / | (cd /mnt/corpse; restore xf -)
and ran grave-robber against /mnt/corpse. Verified that a working
copy of vmunix and core were correctly extracted vom
/mnt/corpse/var/adm/crash/core.0 into data/sysname/tmp_core and
that command_out/netstat-an output correctly reflected the network
status at reset time.
lazarus
Ran unrm on the filesystem where the large compile tree was
removed (see graverobber -i above). Ran lazarus on that output
file. Lazarus ran without errors and the output seemed reasonable :-)
Back to my homepage.