as) United States
`a2) Patent Application Publication (10) Pub. No.: US 2005/0050389 Al
`(43) Pub. Date: Mar. 3, 2005
`
`Chaurasia
`
`US 20050050389A1
`
`(54)
`
`(76)
`
`METHOD OF AND APPARATUS FOR
`CROSS-PLATFORM CORE DUMPING
`DURING DYNAMIC BINARY TRANSLATION
`
`Inventor: Rajesh Kumar Chaurasia, Bangalore
`(IN)
`
`Correspondence Address:
`HEWLETT PACKARD COMPANY
`P O BOX272400, 3404 E. HARMONY ROAD
`INTELLECTUAL PROPERTY
`ADMINISTRATION
`FORT COLLINS, CO 80527-2400 (US)
`
`(21)
`
`Appl. No.:
`
`10/647,142
`
`(22)
`
`Filed:
`
`Aug. 25, 2003
`
`Publication Classification
`
`61)
`
`Tt, C07 i eeescecccsssssseseescccessssenescccceccessnnenescss HO4L 1/22
`
`(52) US. Che
`
`esssssssssssssstsstsntsesnesnstsntntensvesnee 714/15
`
`(57)
`
`ABSTRACT
`
`Acorefile derived from an application crash is created. The
`application belongs to a source platform and is run on a
`target platform. The application is emulated on the target
`platform.
`In response to the crash occurring, detecting
`whether the crash corresponds to the failure of a process
`corresponding to the emulated application. The crash pro-
`cess is mediated by intercepting information relating to the
`failure of the emulated application and writing a corefile
`corresponding to the failure of the emulated application
`running on the source platform. A dynamic binary translator
`performs the steps and debugs an emulated application as if
`it had crashed and dumpeda core file on its native platform.
`
`
`
`
`
`53
`
`Signal from Kemel
`
`
`Does Program has
`
`on TARGET
`Continue with
`
`a Signal Handler?
`
`
`Computer System
`signal
`
`handling for
`
`program
`
`
`
`Call CGL with parameters
`Signal, si_cide,ss_flags, CORE_EXIT
`
`
`
`
`Construct corefile name based on
`
`
`CORE_FLAGS.
`
`
`If CORE_ADDPID, then core.appname.pid
`
`
`else core.appname
`
`
`
`
`Find out max allowable corefile size by
`using system calls getrlimit(RLIMIT_CORE...)
`
`and ulimit(UL_GETFSIZE)
`
`
`
`
`
`Create the corefile as
`corefile_fd-creat (corefilename, S_IRUSRS_IWUSR)
`
`
`
`
`
`
`Prepare prob_hdr_tbl in memory as described in
`4.4.10. The program header table should
`contain num_dumpable_segments entries
`
`
`
`
`
`
`Google Exhibit 1098
`Google v. VirtaMove
`
`Google Exhibit 1098
`Google v. VirtaMove
`
`

`

`Patent Application Publication Mar. 3, 2005 Sheet 1 of 4
`
`US 2005/0050389 Al
`
`10
`
`16
`
`16
`
`
`
`
`
`Software
`Applications
` Migration
`
`
`
`HOST
`
`COMPUTER
`
`
`
`SYSTEM
`
`
`
`
`
`
` Core File for
`
`
`Software from
`HOST PLATFORM
`
` Private MMAP Area (Only writable)
`
`Figure 2
`
`

`

`Patent Application Publication Mar. 3, 2005 Sheet 2 of 4
`
`US 2005/0050389 Al
`
`53 Signal from Kemel
`
`
` Does Program has
`
`
`on TARGET
`Computer System
`
`
`
`
`a Signal Handler?
`
`
`Is Signal one of those
`described in Section 3.2] No
`Yes
`
`Continue with
`signal
`handling for
`program
`
`Signal, si_cide,ss_flags, CORE_EXIT
`
`Suspend all threads in process, exceptself
`
`i.
`
`55
`
`57
`
`Is program setuid/setgid ?
`N
`
`Yes_.J
`Abort CGL
`
`
`Construct corefile name based on
`
`
`CORE_FLAGS.
`If CORE_ADDPID, then core.appname.pid
`
`
`else core.appname
`
`
`
`
`Find out max allowable corefile size by
`
`using system calls getriimit(RLIMIT_CORE...)
`
`
`and ulimit(UL_GETFSIZE)
`
`
`Is max core file size zero ?
`
`No
`
`Create the core file as
`corefile_fd-creat (corefilename, S_IRUSRS_IWUSR)
`
`Count no. of dumpable segmentsto corefile as
`described in 4.4.8 and record in num_dumpable_segments
`
`
`
`
`
`63
`
`in 4.4.9 and write it out to core file.
`No
`
`
`Prepare prob_hdr_tbl in memory as describedin
`4.4.10. The program header table should
`
`contain num_dumpable_segments entries
`
`64
`
`
`
`Figure 3a
`
`CNEXT
`
`

`

`Patent Application Publication Mar. 3, 2005 Sheet 3 of 4
`
`US 2005/0050389 Al
`
`About
`CGL
`
`Yes
`
`
`
`
`
`Check if max core file size will exceed ?
`Leave space for program header
`table in corefile.
`Write out CORE_VERSION to core
`file as described in 4.4.11
`
`
`
`Check if max core file size will exceed ?
`Write out CORE_KERNEL to core
`file as described in 4.4.12
`
`
`
`Check if max corefile size will exceed ?
`
`Write out COMMAND to core
`
`file as described in 4.4.13
`
`
`
` Check if max core file size will exceed ?
`Construct proc_info for thread i
`
`
`Write out proc_info for thread i to core
`
`file as described in 4.4.14
`
`
`Check if max core file size will exceed ?
`
`Write out DATA seg to core
`file as described in 4.4.15
`
`
`
`Check if max core file size will exceed ?
`
`Write out STACK seg to core
`file as described in 4.4.15
`
`
` For each eligible
`
`
`Checkif max corefile size will exceed ?
`
`shared memory
`
`Write out shared memory seg to core
`
`seg in shm_list_p
`
`file as discribed in 4.4.16
`73
`list,
`
`
`
`
`
`
`
`70
`
`For each active
`thread i in
`
`program
`
`74
`
`Figure 3b
`
`

`

`Patent Application Publication Mar. 3, 2005 Sheet 4 of 4
`
`US 2005/0050389 Al
`
`Cre)
`
`Cet)
`
`
`
`Walk through mprotect_list_p list and
`for each memoryregion having protection
`
`
`as PROT_NONE, changethe protection to
`PROT_READ
`
`
`
`
`
`
`For each memory
`Checkif max core file size will exceed ?
`
`
`seg in mmap_list_p
`Write out writable private MMAP memory
`
`list
`seg to core file as described in 4.4.18
`
`
`
`
`75
`
`Yes
`
`76
`
`Return to DTS
`Stop Target Computer
`System OS from writing
`corefile for DTS.
`
`.
`Write out program
`‘il header table to “ht {
`ile as described in
`4.4.19
`
`79
`
`82 Uninstall signal
`and set return status.
`
`
`
` raise(signal), so that
`
`Continue.
`Check if
`Process is on its way
`
`
`
`core_flags is
`CORE EXIT ?
`shell can print
`to exit.
`
`
`coredump message
`
`handler
`
`
`Revert back original
`
`
`protection bits on all
`
`
`memory regions that
`
`
`
`
`were forced to
`
`
`PROT-READ
`
`Resumeall threads
`
`Figure 3c
`
`

`

`US 2005/0050389 Al
`
`Mar. 3, 2005
`
`METHOD OF AND APPARATUS FOR
`CROSS-PLATFORM CORE DUMPING DURING
`DYNAMIC BINARY TRANSLATION
`
`TECHNICAL FIELD
`
`[0001] The present invention relates to failure analysis of
`software, which belongsto a host computer architecture and
`is run on a target computer system under an emulation mode
`via a dynamic binarytranslator. Moreparticularly, although
`not exclusively, the invention relates to a method of and
`apparatus for generating and analyzing a core dumpfile for
`software applications during their migration from a source
`computer system to a target computer system.
`
`BACKGROUND ART
`
`hardware registers at a specific moment, usually when a
`system failure occurs. A core dump can be thought of as a
`complete snapshot of the RAM andall hardwareregisters of
`the machine and thus represents the state of the machine at
`the time of the system failure. Thus, a core dump is used
`mainly for debugging a program during development of an
`application. In the present context, it is envisaged that core
`dumps are to be used in the maintenance of emulated
`applications where the application needs to be debugged so
`that problems in emulation of the non-native application can
`be resolved. When an emulated application crashes on a
`target machine,the core file produced correspondsto that of
`the emulator. It carries virtually no information relating to
`the emulated application and thus there is no way for an
`application developer to analyse the cause of the fault.
`
`It is therefore an object of the invention to provide
`[0007]
`a method of and apparatus for allowing a user to obtain a
`[0002] A major problem with transitioning to a new com-
`core dump image of a software application belonging to a
`puter architecture is that users usually want applications
`source platform, running onatarget platform, with the help
`which had previously been run on the old architecture to be
`run on the new architecture. Considerable investment in
`of a software emulator, in the situation where the emulated
`application crashes on the target system.
`
`time and moneyis expended in developing applications and
`training people in the use of the new architecture. Users are
`also, understandably, unwilling to change their work tools
`during a migration to a new computerarchitecture. Preserv-
`ing legacy applications can, of course, be achieved by
`porting the application to the new environment. However,
`this requires that the source code be accessible and that the
`application be recompiled specifically for the new platform.
`If every component of the source code is not available,
`porting the application is either extremely time-consuming
`or impossible.
`
`to a degree, using
`[0003] This problem can be solved,
`binary translation techniques wherebythe binary code from
`the legacy application is automatically converted from one
`instruction set to another without the need for high-level
`source code
`
`[0004] An example of a binary translation system is the
`Aries product available from Hewlett-Packard. The Aries
`product assists migration between the PA-RISC (precision
`architecture reduced instruction set computing) architecture
`and JA-64 systems running HP-UX. The Aries fast inter-
`preter accurately emulates a complete set of PA-RISC
`instructions with no user intervention. During interpretation,
`it monitors the applications execution pattern and translates
`only the frequently executed code into native IA-64 code at
`runtime. At the end of the emulation, Aries discards all the
`translated code without modifying the original application.
`Therefore, dynamic translation provides fast emulation and
`preservesthe integrity of the emulated PA-RISC application.
`The Aries dynamic translator also performs Environment
`Emulation which emulates PA-RISC applications system
`calls, signal delivery, exception management and threads
`management.
`
`[0005] When performing the translation of an application
`from a source architecture to a target architecture,
`it is
`extremely useful to have access to the same analytical tools
`as would be available to a programmer developing an
`application for a specified architecture. A key tool/process is
`the facility to analyse a core dump file.
`
`[0006] A core dump can be defined as printing or the
`copying to hard disk the contents of a random access
`memory (RAM) of a machine (e.g., a computer) and all
`
`DISCLOSURE OF THE INVENTION
`
`In one aspect the invention relates to a method of
`[0008]
`creating a core file derived from an application crash, said
`application belonging to a source platform and being run on
`a target platform, comprising the steps of:
`
`[0009]
`
`emulating the application on the target platform;
`
`g
`P
`g
`in response to the crash occuring, detectin
`0010]
`whether the crash corresponds to the failure of a
`process corresponding to the emulated application; and
`
`[0011] mediating the crash process by intercepting
`information relating to the failure of the emulated
`application and writing a core file corresponding to the
`failure of the emulated application running on the
`source platform.
`
`{0012] The mediation process preferably includes the
`steps of:
`
`suspending all processes except the process han-
`[0013]
`dling the creation of the core file;
`
`[0014] writing the contents of the emulated applications
`core file to non-volatile memory.
`
`[0015] The mediation process mayalso include the step of
`writing out data relating to the emulation state of the
`application to the core file.
`
`the mediation step
`In a preferred embodiment
`[0016]
`includes the step of writing out data relating to the process,
`data segments, stack segments, shared memory segments,
`and MMAP memory segments, to the corefile.
`
`[0017] The mediation step may further include one or
`moreof the steps of writing out information identifying the
`core version, the process which caused the crash, the host
`platform kernel version to which the core file belongs, and
`the name of the application which produced the corefile.
`
`[0018] The steps of setting up the core file book-keeping
`parameters preferably include one or more of (1) setting
`permissions for the core file, (2) specifying the core file
`name, (3) determining the maximum allowable size for the
`
`

`

`US 2005/0050389 Al
`
`Mar. 3, 2005
`
`corefile, (4) creating an empty corefile, and (5) determining
`the numberof writable segments to be written to the core
`file.
`
`[0019] The method may further including the step of
`writing out a specified file header to the core file.
`
`[0020] The mediation step is preferably invoked when one
`or more specified signals is received from the target plat-
`forms operating system.
`
`the emulation step
`In a preferred embodiment,
`{0021]
`includes monitoring book-keeping information required to
`mediate a crash and creation of a core dumpfile relating to
`the emulated application.
`
`[0022] Preferably the book-keeping step includes moni-
`toring one or moredata relating to the state of the emulated
`application.
`
`[0023] Preferably the data relating to the state of the
`emulated application includes one or more of (1) the start
`address of the applications data segment, (2) the size of the
`applications data segment, and (3) system calls made by the
`application which specify the state of the emulated applica-
`tion.
`
`[0024] Preferably, the method includes the step of imple-
`menting a signal handler adapted to intercept the signals
`which, on receipt of said signals, would cause a core dump
`in the target platform.
`
`Ina further aspect, the invention relates to a binary
`[0025]
`translator adapted to operate in accordance with the method
`as hereinbefore defined.
`
`In yet a further aspect, the invention relates to a
`[0026]
`computer adapted to operate in accordance with the method
`as hereinbefore defined.
`
`the invention relates to a
`In a further aspect,
`[0027]
`memorystoring instructions specifying a method of oper-
`ating a computer as hereinbefore defined.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0028] The present invention will now be described by
`wayof example only and with reference to the drawings in
`which:
`
`FIG.1: is an illustration of the process of cross-
`[0029]
`platform core file generation via a software emulator;
`
`FIG.2: is an illustration of an exemplary corefile
`[0030]
`layout for a host computer system; and
`
`[0031] FIGS. 3a and 3b: together are a block diagram of
`an example of the operation of a core file generation layer.
`
`DETAILED DESCRIPTION OF THE DRAWINGS
`
`FIG.1 is a schematicillustration of how an appli-
`[0032]
`cation is migrated to a new hostplatform. A legacy platform
`10 running a legacy software application is shownattheleft
`of FIG. 1. According to techniques knownin the art and
`discussed in the Background portion of this specification,
`software is conventionally migratedto the target platform 11
`and takes the form 16. This host application is run on a
`software emulator 13 as described briefly above. There are
`a number of emulation approaches and the example of the
`applicants Aries system is assumed here.
`
`Ifthe program crashes (operation 19), a corefile 20
`[0033]
`is written. However, core file 20 corresponds to the RAM
`and register state for the target machine running the emulator
`13, not the host machine. Core file 20 therefore carries
`virtually no information regarding the state of the host
`application and its detailed interaction with the emulator 13
`and the target platform 11.
`
`[0034] A preferred embodimentof the invention functions
`as shown in the nght hand portion of FIG. 1. Here, an
`intermediary layer 18 is interposed between the software
`emulator 13 and the post-crash process including program
`crashes step 29 and core dumpfile 21 for software from the
`host platform 10. This intermediate layer 18 manages the
`creation of a core dump file 21 in such a waythat the core
`file 21 which is produced correspondsto the core file of the
`original application 23 running on the host platform 10.
`
`(0035] Thus the application 10 can be debugged asif it
`were being run onits native platform 10 whenin fact it is
`being run on an emulator.
`
`[0036] According to the presently described embodiment,
`the intervening layer 18 includes a Corefile Generation
`Layer (CGL) which, when augmenting a software emulator
`13, makes cross-platform core file generation for a host
`application possible.
`
`In the following description, the core file genera-
`[0037]
`tion layer is referred to as the CGL and the software
`emulator, as the Dynamic Binary Translator or DTS. For the
`purpose of this description, a UNIX® like environmentis
`assumed, although other architectures and operating system
`environments are possible and the specific operation of the
`present cross-platform corefile generation technique will be
`clear to one skilled in the art.
`
`[0038] AUNIX® like environmentprovides the following
`abstractions:
`
`[0039]
`
`processes
`
`[0040]
`
`threads
`
`[0041]
`
`signals
`
`[0042]
`
`system calls.
`
`[0043] On any UNIX® system, a core dump image is
`generated in response to a process crashing with an un-
`handled signal. In this situation, the operating system writes
`out an application core file before terminating the process.
`This process of writing the corefile is completely internal to
`the operating system.
`
`[0044] To this end, any UNIX® process which receives
`any of the following signals, and does not have a signal
`handler installed for the same, is terminated with the gen-
`eration of a core dump image. These signals are as follows:
`
`[0045] SIGFPE
`
`[0046] SIGSEGV
`
`[0047] SIGBUS
`
`[0048] SIGQUIT
`
`[0049] SIGILL
`
`[0050] SIGTRAP
`
`[0051] SIGABRT
`
`

`

`US 2005/0050389 Al
`
`Mar. 3, 2005
`
`[0052] SIGEMT
`
`[0053] SIGSYS
`
`[0054] SIGXCPU
`
`[0055] SIGXFSZ
`
`[0056] SIGGFAULT
`
`[0057] The core file which is produced during any such
`system fault has multiple sections. Each section has a header
`which identifies the section, its type, file offset, length etc.
`These sections represent unique information about the pro-
`cesses virtual address space. Somesections of the corefile
`also include information only for identification of the core
`file. For example, the operating system version, hardware
`model, which process produced the core file etc. Typically
`the core file includes the following information
`
`[0058] File header.
`
`[0059] Version identification section.
`
`[0060] The command namethat caused the core file
`to be generated.
`
`proc_info (machine save_state) structure for
`[0061]
`each thread in the process.
`
`[0062] The data segment of the program.
`
`[0063] The stack of the program.
`
`[0064] Writable shared memory(onecorefile section
`for each shared memory object).
`
`[0065] Writable private MMAP regions(onecorefile
`section for each writable private mmap area).
`
`In the present example, the core file layout for the
`[0066]
`host computer is assumed to be that as shown in FIG.2.
`Referring to FIG. 2, an Elf64 core file layout of the host
`machine has architecture-specific extensions to the Elf file
`format. This is in accordance with the applicants own
`specifications.
`
`[0067] The software emulator or Dynamic Binary Trans-
`lator (DTS) is a program whichis able to automatically run
`an application, whichis native to a host computer system, on
`a target computer system. Thatis, a software emulator hosts
`all of the host computer system software on the target
`computer system. For the purposes of this description, we
`will assume the presence of a DTS with certain minimum
`functionalities as follows.
`
`[0068] Whenever software belonging to the host system is
`run on the target system, such hardware is automatically
`emulated by the DTS.
`
`[0069] The DTS is capable of environment emulation.
`That is, the DTSis able to detect system calls of a program
`and either pass the calls on to the operating system of the
`target computer system operating system, or emulate the
`calls within the DTSitself. This assumesthat the DTSis able
`
`to extract the system call arguments and system call return
`values of the program.
`
`arrives and the program does not have a signal handler
`installed for that signal, the DTS calls the Corefile Genera-
`tion Layer (CGL). In this case, the DTS ensures that the
`default behaviour for such a signal
`is to terminate the
`program with a core dumpfile.
`
`the DTS maintains
`In relation to the program,
`{0071]
`information about each thread in the program as well as
`information about critical context (register state) of each
`thread. This includes the per-thread register state, signal
`mask, program counteretc.
`
`In operation, the DTS determines whether a signal
`[0072]
`is intended for DTS internal use or if the signal is to be
`delivered to a program. The DTS makesavailable to the
`CGLall the information needed to write out a core file for
`the emulated program on the target system.
`
`[0073] The embodimentof the invention described herein
`performs three primary functions: making the DTS capable
`of handling the book-keeping information which is required
`by the CGL, detecting the conditions under which the DTS
`should call the CGL, and enabling the CGL to write out the
`core file for the program. The operation of the cross-
`platform core dumping method is explained as follows with
`reference to the exemplary embodimentillustrated in FIGS.
`3a and 3b.
`
`[0074] Asa preliminary point, while the DTS emulates the
`program onthe target computer system, the DTS computer
`system needs to record the start address of the data segment
`of the program in memory and the initial size of the data
`segment as data_start and data_size respectively. The DTS
`also tracks all brk( )/sbrk( ) system calls made by the
`program. This is required as the usage by the program of
`malloc( ) and similar library routines would dynamically
`change the end of data segment address. The following
`pseudocode can track and record the programs brk( )/sbrk(
`system calls.
`
`Initially record data_endaddr = data_start + data_isize
`If (brk/sbrk system call succeeds)
`then
`extract endaddr parameter from brk( }/sbrk( )
`use endaddr to update data_endaddr global variable.
`
`[0075] The DTSalso records the basenameof the program
`as app_basename as well as tracking the mmap( ) system
`calls of the program. One method of tracking and recording
`private writable mmap( ) system calls is:
`
`if (mmap system call succeeds) and
`(mmap does not correspond to mmap of TEXT segment)
`then
`extract protect, flags, address, length arguments from mmap( )
`create a node with extracted parameters
`add the node to a global mmap__list_p linkedlist.
`
`[0070] The DTS needs to be able to deliver signals and
`exceptions to the program. This means that the DTS can
`detect if the program has a signal handler installed for a
`given signal and if it has, upon receipt of such a signal the
`DTS emulates the signal handler of the program. If a signal
`
`[0076] The DTSalso needsto track the mprotect() system
`calls of the program. This is required as the program might
`zero out access permission on somepages in private writable
`mmap regions. The following method can be used to track
`and record mprotect( ) system calls.
`
`

`

`US 2005/0050389 Al
`
`Mar. 3, 2005
`
`If (mprotect system call succeeds)
`then
`
`extract protection, length and address arguments from mprotect( )
`create a node with extracted parameters
`add the node to a global mprotect_list__p linked list.
`
`[0077] The DTS also tracks a number of other systems
`calls of the program. These calls can be enumerated as
`follows with suitable pseudocode methods given.
`
`[0078] munmap( ) System Calls.
`
`If (munmap system call succeeds)
`then
`extract address and length parameters from munmap( )
`based on extracted parameters check if any node in mmap_list_p and
`mprotect_list_p matches to this munmap( )?
`If yes, then remove the node from corresponding list.
`
`[0079]
`
`shmat( ) System Calls.
`
`If (shmat system call succeeds)
`then
`extract shmid, shmaddr and shmflags parameters from shmat( )
`create a node with extracted parameters andinitialize the length shmlen
`of each segmentto zero(to be filled later).
`add the node to a global shm_list_p linkedlist.
`
`-continued
`
`extract shmaddr parameter from shmdt( )
`search shm__list_p for a node that matches shmaddr;
`if a node is found, then deleteit.
`
`[0081]
`
`shmctl( ) System Calls.
`
`If (shmctl system call succeeds)
`then
`extract shmid and cmd parameters from shmetl( )
`if (cmd == IPC_RMID)
`then
`search shm_list_p for a node that matches shmid.
`If a node is found, then delete it
`
`[0082] The DTSalso controls the stack of the program. To
`this end, the DTS records the start and end addresses of the
`stack of the program as stack_start and stack_end respec-
`tively. The DTS makethis information available to the CGL
`and updates the stack_start and stack_end appropriately as
`the programs emulation progresses. Referring to FIGS. 3a
`and 3b,
`the DTS installs its own signal handler for all
`signals, having a default behaviour that
`terminates the
`process with a core dump image. These signals which cause
`this behaviour are enumerated above.
`
`Ilere, the reference numerals refer to the steps or
`[0083]
`operations in the method as shown in FIGS.3a and 3b. For
`the CGL to be called by DTSit is necessary that a default
`signal handler be registered by the software emulator with
`the target computer system OS. The presence of the DTS
`signal handler (operation 51) causes the target computer
`system operating system to send the signal to DTS (opera-
`tion 52). The signal handler, upon receiptof the signal, calls
`the CGL (operation 54) as described as follows.
`[0084] The following method can be used to determine
`when and how the DTS should call the CGL.
`
`SIGFPE, SIGSEGV, SIGBUS, SIGQUIT, SIGILL, SIGTRAP,
`SIGABRT, SIGEMT,SIGSYS, SIGXCPU, SIGXFSZ, SIGGFAULT)
`
`If (signal is one of
`and
`(signal is to be delivered to program)
`and
`(for signal, program’s signal handler == SIG_DFL)
`then
`block all signals from arriving after this point
`call CGL as error = CGL(signal,
`si_code,
`processor dependent save_state flags,
`core flags)
`
`if (error occurred)
`then
`print out error message and abort DTS.
`else
`stop target computer system OS from writing core file for DTS, by
`zeroing out the maximum size of core file. On some UNIX ® flavors
`setrlimit(RLIMIT_CORE,...) will do.
`
`[0080]
`
`shmdt( ) System Calls.
`
`if (shmdt system call succeeds)
`then
`
`[0085] The CGL writes outthe core file in a manner which
`is similar to how the HOST computer system’s operating
`system would write out the core file for the program.
`
`[0086] The following method forms the backboneofthis
`embodiment of the CGL design and is described in detail
`with reference to FIGS. 3a and 3b.
`
`

`

`US 2005/0050389 Al
`
`Mar. 3, 2005
`
`[0087] The DTSinitially calls the CGL with the param-
`eters: signal, si_code, ss_flags and core_flags (operation 54).
`For security reasons corefiles for the program that are setuid
`and/or setgid are not written. The following method can be
`used in this situation.
`
`-continued
`
`Record max_corefile_size = least of (max corefile size, max size of
`file)
`
`If getuid( ) != geteuid( ) or getgid( ) != geteuid( )
`then
`do not write core file for the program.
`Abort CGLsilently and let target computer system write out core
`file for DTS. [See operation 57 in Figure 3a]
`
`[0088] All other threads in the process are suspended
`except
`the present
`thread (operation 55). The CGL can
`optionally write out the core file as core.appname(operation
`58) to differentiate the programscorefiles from DTS’s own
`core files according to the following method.
`
`If (core_flags == CORE_ADDPID)
`then
`corefilename = string concatenation of core.app basename.getpid( }
`else
`corefilename = string concatenation of core.app__basename
`
`the DTS determines the
`[0089] The next step is that
`maximum allowable core file size for the program 59.
`
`[0090] This is done with the proviso that the DTS doesnot
`create a new core file or truncate an existing core file if
`max_corefile_size is 0 (operation 60).
`
`if (max_corefile_size == 0)
`then
`
`abort CGLsilently without creating core file or truncating
`existing corefile.
`
`[0091] The DTS then creates a file corefilename in the
`current working directory and gives the user only permission
`to access the corefile (operation 61). If a corefile already
`exists, its length is truncated to zero. The file descriptor for
`writing out the core file contents is then recorded.
`
`corefile_fd=creat
`[0092]
`S_IRUSER|S_IWUSER)
`
`(corefilename,
`
`[0093] Then the numberof segments to be written to core
`file is counted as below (operation 62). This is required, as
`space needsto be reserved for the program headertable after
`the ELF file headerat the start of corefile.
`
`
`
`Counting num of writable shared memory segments
`Use ge tune() or equivalent system call to find if target computer system
`OS allows writing of read-only and/or read-write shared memory segments to
`e.
`core fil
`if no, tl
`hen no shared memory segment will be written to core file. In this
`case, initialize num of writable shared memory segments to zero.
`if yes,
`extract hen walk through shm_list_p and use system call pstat_getshm( ) to
`attributes corresponding to each shared memory segment represented
`by a nodein list shm_list_p.
`For each eligible shared memory segment, update the length of segment in
`shm_list_p node.
`Markt!
`he eligible nodes for writing to corefile.
`Count
`he numberofeligible nodes in shm_list_p.
`Counting
`num of writable private mmap memory segments
`Walk through mmap__list_p list and for each node checkif protect field is
`PROT_WRITEandflag field is MAP_PRIVATE.
`Mark e:
`Count acheligible node for writing to core file.
`he numberofeligible nodes in mmap__list_p.
`num_dumpable_segments = 1 segmentfor core file version +
`1
`segment
`for
`core
`file
`identification
`1 segment for command name +
`num of active threads in process +
`1 segment for data +
`1 segment for stack +
`num of writable shared memory segments +
`num of writable private mmap memory segments
`
`+
`
`Use system call getrlimit (RLIMIT_CORE,...) to get maxcore file size.
`Use system call ulimit (UL_GETFSIZE) to get maxsize offile that this
`process can create.
`
`Initialize a structure elf64_ehdr of type Elf64_Ehdr as below
`
`[0094] The ELF file header is the written out to the core
`file (operation 63) according to:
`
`

`

`US 2005/0050389 Al
`
`Mar. 3, 2005
`
`-continued
`
`Initialize all architecture and data model specific fields according to
`HOST computer system.
`Initialise elf64_ehdr.e_type = ET_CORE
`Initialize elf64_ehdr.e_phnum = num_dumpable_segments
`Initialize elf64__ehdr.e_phoff = sizeof(elf64_ehdr)
`Write out elf64_ehdr to core file
`write (corefile_fd, &elf64_ehdr, sizeof(elf64_ehdr)
`
`[0095] A program headertable is to be written after the
`elf64_ehdr in the core file. [lowever, the program header
`table will be populated with each segmentthat is written to
`core file. Therefore, the DTS keeps the program headertable
`in memory and writes it out to the core file once all the
`segments are written (operation 64).
`
`check_error( )
`Write out utsnamestructure to core file at current file offset.
`
`Update next entry in prog_hdr_tbl, with p_typefield initialized as
`PT_HP_CORE_KERNEL
`
`[0099] The COMMANDnameis then written to the core
`file (operation 67). This indicates the basename of the
`application that produced the corefile.
`
`Declare program headertable as
`Elf64_Phdr prog__hdr__tbl[num_dumpable_segments] = {0}
`
`check_error( }; Write out app_basenameto core file at currentfile offset.
`Update next entry in prog_hdr_tbl, with p_typefield initialized as
`PT_HP_CORE_COMM.
`
`
`[0096] The CORE VERSIONis then written to core file
`(operation 65). This is generally a constant integer and is
`specific to a computer system.
`
`Leave out space for program headertable in corefile
`Iseek (corefile_fd, sizeof(elf64_ehdr)+sizeof(proghdr_tbl),
`SEEK_SET)
`check_error()
`Write out CORE VERSIONto core file
`write (corefile_fd, &Elf64_CORE_VERSION,sizeof(int)
`Update proghdr__tbl as
`prog _hdr_tbl[O].p_type = PT_HP_CORE_VERSION
`proghdr_tbl[0].p_offset = corefile offset at which
`CORE VERSIONwaswritten
`prog_hdr_tbl[O].p_filesz = sizeof (Elf64_CORE_VERSION)
`prog_hdr_tbl[O].p_memsz = sizeof (Elf64_CORE_VERSION)
`
`[0097] The CORE_KERNELversionis then written to the
`core file. The CORE KERNELversion specifies to which
`computer system this core file belongs. The CORE KER-
`NELisinitialized and written out to core file as described
`
`below. This segmenttypically includes what the “uname -a”
`command shows on the HOST computer system.
`
`proc_info is written out for each thread to the core
`[0100]
`file (operation 68). The proc_info structure specifies the
`machine register context along with the signal name,
`si_code, ss_flags etc. This structure is specific to HOST
`computer system.
`
`for each active thread i in process
`do
`
`construct proc_info structure for thread i, based on critical register
`context supplied by DTS and signal, si_code, ss_flags
`(parameter for CGL).
`This is similar to machine save_save on HOST computer system.
`check_error( )
`Write out proc_info for thread i to core file at currentoffset
`Update next entry in proghdr_tbl with p__type field initialized as
`PT_HP_CORE_PROC
`done
`
`[0098] The system then prepares a structure of type
`utsname for the host computer system and initialize the
`
`[0101] The DATA segmentis then written out to the core
`file (operation 71):
`
`data_segsize = Roundup_on_next_page_boundary(data_endaddr - data_start)
`check_error( )
`Write out data_seg_size bytes from data_start to cor