This document gives information on how debug DRAMA. It is still under developlement. If you have any problems, please feel free to contact me, Tony Farrell, at tjf@aao.gov.au  

What is running

To find out just what programs are running, enter the tasks command. This produces something like the following IMP_Master local type "0" description "" IMP_Transmitter local type "0" description "" IMP_Receiver local type "0" description "" IMP_Pulse_113 local type "0" description "" TICKER remote type "0" description "" DTCL local type "0" description "" Here, the first column is the task names. The names starting with IMP are IMP layer tasks. The second column indicates the location of a task. In addittion, the task type and description are output. See DitsSetDetails for more on task types and descriptions.

Connections etc.

The following commmands are available to give you information about connections between DRAMA tasks.
impprobe
Gives details of buffers and connections using an X windows dialog.

Remember, Imp connections are one way, but DRAMA uses two way connections, so there are two connections associcated with each DRAMA path, one from task A to task B, and one from task B to task A.

A connection has a assocaited buffer in the target task.

impprobe uses impdump to generate the information.

Note, you must re-run impprobe to update the information.

impdump
this if you want a text copy of the information.

Note impprobe can be passed the name of a file containing the output from impdump for off line analysis.

ImpProbe
Under VxWorks, there is no impdump. Use ImpProbe, the underlying IMP routine.

DITS layer debugging

Please see DRAMA logging for details on DITS layer debugging.

DITS Enquiries (DITS Control Messages)

Various control messages are available to get current infomation, in particular, about control tasks.
DUMPATHS
Writes standard output low level details about paths being used by a task. I.e. Imp connection number, flags, buffer sizes. Note here it is the task task which writes to stdout, not the initiator of the message.

Note, buffer sizes are only valid for when the path was set up by the task in question, not for paths set up by other tasks talking to that task.

DUMPTRANSID
Writes to standard output low level details about transations in progress. Also gives details about the memory allocation for transaction ids
DUMPACTACTIVE
Writes to standard output, details of any running actions (actions which are rescheduling)
DUMPACTALL
Writes to standard output, detail of all actions supported by the task
DUMPMON
Writes to standard output details of parameter monitoring operations
VERSIONS
Writes to standard output version details for the running task (DRAMA, DITS and IMP)
LOGNOTE
Writes a message to the task's log file - if one is enabled. See DRAMA's Logging System for details on the logging system.
LOGFLUSH
Flush the log file to disk - normally done after each message anyway.
SDSLEAKCHK
Output information to help chase SDS ID leaks, which cause a program to slow down
PRINTENV
Output the values of all envrionment variables known to the task
For example, assuming DTCL is running, communicating with a remote TICKER program. Assume DTCL has a path to ticker and is monitoring a parameter in TICKER. If you were to enter ditscmd -c DTCL DUMPPATHS You would get something like this on DTCL's standard output device. Dump of paths held by task DTCL Path d3e68 to task DITSCMD_2b6 (2b6@c0e7a714) Status OK, Imp connection number 2 local, using SDS, Flags = 0 Path was set up by other task - hence buffer sizes not known Path d3f90 to task TICKER (3b85@c0e7a7a8) Status OK, Imp connection number 1 remote, using SDS, Flags = 2 Path was set up by task itself Buffers 8000 1 800 10 Path c7aa0 to task DTCL (0@0) Status OK, Imp connection number 0 local, not using SDS, Flags = 0 Path was set up by task itself Buffers 2000 1 0 0 This shows DTCL has a total of three paths. The first is the path set up by the version of DITSCMD used to send the control message. The second is the path to TICKER. Finally, is DTCL's path to itself, which all DRAMA task have. Note for for each path, the status, Imp connection number and various flag values are shown. In addition, if the path was set up by the task itself, then the buffer sizes are known.

Now assuming you want to find out details about outstanding transactions. You can use the following command ditscmd -c DTCL DUMPTRANSIDS This will produce something like this Dump of transactions in task DTCL UFACE Transactions Transaction d4084 of type DITS_MSG_MONITOR to task TICKER Uface transaction, routine 3d780, code d7488 There were 1 UFACE transactions There were 1 outstanding transactions DTCL:Transaction id memory dump, unit = 60 DTCL:Transaction id memory buffer 0xd0928 DTCL: Buffer 0xd4048, buffer end 0xd4174 DTCL:NextAvail = 0xd40c0, last = 0xd4174, total = 5, alloc = 2 DTCL:Free contains DTCL: 0xd4048 DTCL: Num on free list is 1 You can see that DTCL has one outstanding UFACE (user interface initiated) transaction. This is of type DITS_MSG_MONITOR and is to task TICKER. So, this is our monitor transaction.

In addition, there is a dump of the Transaction id memory allocation table. Here total = gives the total amount of transaction id units for which memory has been allocated so far. There are allocated in units of 5. The alloc = line gives the maximum number which have ever been allocated. If you subtract the number on the free list from this number, you get the current number allocated (1). This dump is probably of little use, but has been left in to help test new versions of DRAMA for memory leaks.

IMP Layer

First see impdump/impprobe above as first step.

To log IMP networking events

This feature allows the logging of network events seen my the IMP network tasks.
  1. Set the environment variable IMP_NET_LOG_EVENTS to 1.
  2. Then start the netwoking tasks (dits_netstart)
  3. Generate the problem.
  4. Enter $IMP_DEV/netmonitor both dump or under VxWorks NetMontior "both","dump" This will dump the networking log buffers to Transmitter_log and Receiver_log.

    Note, if you run the above command again, it will overwrite any existing files of the same name.
  5. The transmitter log file can be analyzed with the program $IMP_DEV/tlog filename specifying the name of the file generate above as the argument. The output is a list of all the messages sent and recevied by the task, along with time tags. (tlog is can example program for dumping these log files, the source is available in the imp source directory and it may need to be modified to help debug particular problems). The program rlog is similar and works for the receiver log file. To analyze files generated by a VxWorks machine, use tlogvx and rlogvx.

    Our you can give the details of the problem and results of the above to Keith Shortridge.

To dump current state of imp

Consider the user of impprobe/impdump programs above, which do a log of the job and output text - but they are restricted to what they know about IMP.

If you run the $IMP_DEV/impbindump program, this generates a directory named sysdump_output, which will contain a file for each currently running task. This file is a binary dump of the IMP structures of that task. Pass the problem details and results to Keith Shortridge. (No extraction program exists as yet)

tasklog

tasklog controls logging of the IMP layer communcations, where as IMP_NET_LOG_EVENTS control logging of the networking (i.e. socket) layer within transmitter/receiver.

For full DRAMA tasks (which use Dits you can control this level of logging using the DITS_LOG_IMPEVENTS flag to DitsSetDebug(). For networking level problems, you will have use the IMP interface directly. Alternatively, there is a command interface:

To turn logging on for a task, say transmitter $IMP_DEV/tasklog IMP_Transmitter On After generating the problem, you can dump the log using $IMP_DEV/tasklog IMP_Transmitter dump filename This will create the specified file.

This file can be analyzed with the program $IMP_DEV/implog filename specifying the file name from above as the argument. The output is a list of all the messages sent and recevied by the task, along with time tags. (implog is can example program for dumping these log files, the source is available in the imp source directory and it may need to be modified to help debug particular problems)

Or you can pass the problem details and results to Keith Shortridge.

Note - under VxWorks, use the command tasklog "IMP_Transmitter","On" tasklog "IMP_Transmitter","dump","filename" Currenlty this does not work for the IMP Receiver network task. But it is not normally required there.

implog does not exist under VxWorks itself - move the files to a Unix machine and use the Unix program implogvx.

Logging IMP events to shared memory

This is a recently added new feature (DRAMA 1.5.1) which allows the logging of imp events to a shared memory buffer. To turn this on, use tasklog task on shared The new $IMP_DEV/taskmon utility can then display the event log from another task while it is running, even if it has hung or (if the shared file hasn't been deleted) crashed. (I'm afraid the DITS layer is pretty good at cleaning up so the file is likely to be gone, through you could try setting the DITS_M_NOEXHAND flag to DitsAppInit() to disable this cleaning up.)

Message code lookup.

If you get a failure to translate a message code, then "messana" may help. A failure to translate message will look something like this %NONAME-E-NOMSG, Message Number 0FD0800C The command messana can be used to analyze this message. The message number is require and should be prefixed by 0x. E.g. messana 0xFD0800C This will give you the Facility number, message number and severity. For example Message fd0800c(265322508) - Facility:2000(DITS), Number:1, Severity:4 Code:%DITS-F-INVARG, Invalid argument to a Dits routine

Note, the environment variable DRAMA_FACILITIES is a space separated list of facility table include files. Add your own facility files here.

Tasks running slow.

If a task slows down over time, it is likely that your tasks is leaking SDS ID's. See here for information on how to debug these.

Click here for the DRAMA home page and here for the AAO home page.

 For more information, contact tjf@aao.gov.au