From Richard.Relph@amd.com  Tue Jan  4 00:41:51 2011
From: Richard.Relph@amd.com (Relph, Richard)
Date: Mon, 3 Jan 2011 18:41:51 -0600
Subject: [Dwarf-Discuss] How best to represent multiple programs in a single
	file...
Message-ID: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>

Our platform allows the creation of a single ELF file that contains multiple 'programs' derived from the same set of source files. We're trying to figure out the best way to represent this in DWARF. We've thought about creating separate sets of DWARF sections for each 'program' (say, .debug_info_programA, etc.), but this would render most DWARF processing tools useless. Yet it isn't obvious to me how to stuff multiple program's worth of DWARF info using just the one set of standard DWARF sections.

For the technically curious, here's the details:

Our platform is a GPU. Each program is loaded on to the hardware fresh, from 'outside', by an 'external' OS, if you will. When the GPU starts executing, it starts from address 0, where we have placed a bootstrap sequence that ends up calling the real code. The real code's source language is OpenCL, which doesn't define a solitary entry point function such as main(), but rather defines a 'kernel' attribute that can be attached to any function. The host code (running on the CPU controlling the GPU) specifies which kernel function to execute at run-time.

Because our GPU doesn't have a 'real' CALL instruction, we essentially treat each 'kernel' function as a complete program in and of itself (ALL functions called by the kernel are inlined by the compiler.) So a single OpenCL source file might have (pseudo-code):

kernel foo() {
    A();
    B();
}

kernel bar() {
    B();
    C();
}

A() {...}
B() {...}
C() {...}

The compiler is invoked once for the source file, producing code and DWARF information for the entire compilation unit. If only that was the end...
But because we don't want the resources utilized by kernel bar() to interfere with resource allocation for kernel foo() (and vice-versa), we post-process this, stripping unused kernels and other dead code and data, producing separately loadable code for each kernel. Each separately loadable kernel is then stored in the ELF for execution when the host program requests it.

The question at hand is how to represent the DWARF information for each separately loadable kernel. We'd like a solution that allows libdwarf, objdump, readelf, dwarfdump, etc. to continue to work, which says we need to put multiple 'programs' worth of debug data in each DWARF section. But then how do the tools (and the debugger) differentiate between data relevant for B() in the context of kernel foo() from B() in the context of kernel bar()?

Any thoughts, comments, suggestions, proofs that it can or cannot work, would be gratefully accepted.

Thanks,
Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/private.cgi/dwarf-discuss-dwarfstd.org/attachments/20110103/762191c5/attachment.htm>


From eager@eagercon.com  Tue Jan  4 02:19:36 2011
From: eager@eagercon.com (Michael Eager)
Date: Mon, 03 Jan 2011 18:19:36 -0800
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single	file...
In-Reply-To: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>
Message-ID: <4D2283B8.9040308@eagercon.com>

Relph, Richard wrote:
> Our platform allows the creation of a single ELF file that contains 
> multiple ?programs? derived from the same set of source files. We?re 
> trying to figure out the best way to represent this in DWARF. We?ve 
> thought about creating separate sets of DWARF sections for each 
> ?program? (say, .debug_info_programA, etc.), but this would render most 
> DWARF processing tools useless. Yet it isn?t obvious to me how to stuff 
> multiple program?s worth of DWARF info using just the one set of 
> standard DWARF sections.
> 
> For the technically curious, here?s the details:
> 
> Our platform is a GPU. Each program is loaded on to the hardware fresh, 
> from ?outside?, by an ?external? OS, if you will. When the GPU starts 
> executing, it starts from address 0, where we have placed a bootstrap 
> sequence that ends up calling the real code. The real code?s source 
> language is OpenCL, which doesn?t define a solitary entry point function 
> such as main(), but rather defines a ?kernel? attribute that can be 
> attached to any function. The host code (running on the CPU controlling 
> the GPU) specifies which kernel function to execute at run-time.
> 
> Because our GPU doesn?t have a ?real? CALL instruction, we essentially 
> treat each ?kernel? function as a complete program in and of itself (ALL 
> functions called by the kernel are inlined by the compiler.) So a single 
> OpenCL source file might have (pseudo-code):
> 
> kernel foo() {
>     A();
>     B();
> }
> 
> kernel bar() {
>     B();
>     C();
> }
> 
> A() {?}
> B() {?}
> C() {?}
> 
> The compiler is invoked once for the source file, producing code and 
> DWARF information for the entire compilation unit. If only that was the end?
> 
> But because we don?t want the resources utilized by kernel bar() to 
> interfere with resource allocation for kernel foo() (and vice-versa), we 
> post-process this, stripping unused kernels and other dead code and 
> data, producing separately loadable code for each kernel. Each 
> separately loadable kernel is then stored in the ELF for execution when 
> the host program requests it.
> 
> The question at hand is how to represent the DWARF information for each 
> separately loadable kernel. We?d like a solution that allows libdwarf, 
> objdump, readelf, dwarfdump, etc. to continue to work, which says we 
> need to put multiple ?programs? worth of debug data in each DWARF 
> section. But then how do the tools (and the debugger) differentiate 
> between data relevant for B() in the context of kernel foo() from B() in 
> the context of kernel bar()?
> 
> Any thoughts, comments, suggestions, proofs that it can or cannot work, 
> would be gratefully accepted.

A couple questions:

How is this ELF executable file organized?

How are these "multiple programs" represented in the executable file?

How are these "separately loaded kernels" selected and loaded?

DWARF doesn't depend on any particular program structure.  There is
no requirement to have a main() function or a single entry point.
It would seem that the DWARF data for your program, with multiple
"kernels", should describe them  correctly.  Of course, when the ELF
file is modified to strip out code, you need to make the corresponding
changes to the DWARF data as well.



-- 
Michael Eager	 eager at eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077




From Richard.Relph@amd.com  Tue Jan  4 16:28:42 2011
From: Richard.Relph@amd.com (Relph, Richard)
Date: Tue, 4 Jan 2011 10:28:42 -0600
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single	file...
In-Reply-To: <4D2283B8.9040308@eagercon.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>
	<4D2283B8.9040308@eagercon.com>
Message-ID: <7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C3C0@sausexmbp02.amd.com>

Michael,
    Thanks for the reply. Below, answers to your questions...

> A couple questions:
> 
> How is this ELF executable file organized?
>
> How are these "multiple programs" represented in the executable file?
Each kernel's "text" is concatenated and placed in the same section and entries are made in the ELF symbol table to note the offset in to that section for each kernel. But only ONE kernel is ever loaded at a time and the loaded kernel's text is always located starting at 0. So if we have N kernels, we have N address 0, N address 1, and so on.

> How are these "separately loaded kernels" selected and loaded?
At execution time, the host program will access the entire ELF file and then load each kernel as needed... sort of like a DLL being identified and then entries in the DLL being invoked as needed... except that in our case, "relocation" or loading occurs as each kernel is invoked, not when the ELF is identified, which (I think) is the crux of the problem.

> DWARF doesn't depend on any particular program structure.  There is
> no requirement to have a main() function or a single entry point.
Understood. But is there an implied requirement that a given instruction location map to no more than one 'source line'? That is, after the (fixed length) bootstrap code, we have at location x different functions depending on which kernel has been invoked. I don't see how to represent this one-to-many mapping in a single instance of, say, the line number program.

> It would seem that the DWARF data for your program, with multiple
> "kernels", should describe them  correctly.  Of course, when the ELF
> file is modified to strip out code, you need to make the corresponding
> changes to the DWARF data as well.
Understood. Given an original source program, it's "full" representation as code, and the corresponding DWARF, I can produce for each individual kernel it's 'stripped' code and DWARF for that. And I know how to concatenate the stripped code for each kernel in to a the form that our system expects so that each kernel can be loaded and executed separately. But I don't know how to store the DWARF for each kernel in a single ELF file so that the standard tools can still be useful.

Thanks,
Richard


From Richard.Relph@amd.com  Tue Jan  4 18:24:43 2011
From: Richard.Relph@amd.com (Relph, Richard)
Date: Tue, 4 Jan 2011 12:24:43 -0600
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single file...
In-Reply-To: <4D23639D.2020700@oracle.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>
	<4D23639D.2020700@oracle.com>
Message-ID: <7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C594@sausexmbp02.amd.com>

Chris,
    Thanks for the reply. We have considered this approach, but it has thus far been rejected for various reasons. Ultimately, what you suggest ends up producing a separate ELF with DWARF for each kernel, which has the obvious advantage of being easy to prove works. But OpenCL treats a compilation of a set of kernels as a unit, so we'd have to design a mechanism to put multiple ELFs in to a single bundle, which we've so far been reluctant to do.
Richard

From: Chris Quenelle [mailto:chris.quenelle@oracle.com]
Sent: Tuesday, January 04, 2011 10:15 AM
To: Relph, Richard
Cc: DWARF
Subject: Re: [Dwarf-Discuss] How best to represent multiple programs in a single file...

My advice is to modify the compiler so it takes the name of a kernel
routine on the command line and only generates the code
that's necessary for that kernel.  Then invoke the compiler once
for each kernel routine using the same source file each time.
You didn't say if all the source is in one file. If you are combining
multiple OpenCL source files, then the modifications to the compiler
are more involved, but still easier than trying to parse, read and rewrite
dwarf information that corresponds to a subset of the original program.  :-)

--chris



On Monday January 3    4:41PM, Relph, Richard wrote:
Our platform allows the creation of a single ELF file that contains multiple 'programs' derived from the same set of source files. We're trying to figure out the best way to represent this in DWARF. We've thought about creating separate sets of DWARF sections for each 'program' (say, .debug_info_programA, etc.), but this would render most DWARF processing tools useless. Yet it isn't obvious to me how to stuff multiple program's worth of DWARF info using just the one set of standard DWARF sections.

For the technically curious, here's the details:

Our platform is a GPU. Each program is loaded on to the hardware fresh, from 'outside', by an 'external' OS, if you will. When the GPU starts executing, it starts from address 0, where we have placed a bootstrap sequence that ends up calling the real code. The real code's source language is OpenCL, which doesn't define a solitary entry point function such as main(), but rather defines a 'kernel' attribute that can be attached to any function. The host code (running on the CPU controlling the GPU) specifies which kernel function to execute at run-time.

Because our GPU doesn't have a 'real' CALL instruction, we essentially treat each 'kernel' function as a complete program in and of itself (ALL functions called by the kernel are inlined by the compiler.) So a single OpenCL source file might have (pseudo-code):

kernel foo() {
    A();
    B();
}

kernel bar() {
    B();
    C();
}

A() {...}
B() {...}
C() {...}

The compiler is invoked once for the source file, producing code and DWARF information for the entire compilation unit. If only that was the end...
But because we don't want the resources utilized by kernel bar() to interfere with resource allocation for kernel foo() (and vice-versa), we post-process this, stripping unused kernels and other dead code and data, producing separately loadable code for each kernel. Each separately loadable kernel is then stored in the ELF for execution when the host program requests it.

The question at hand is how to represent the DWARF information for each separately loadable kernel. We'd like a solution that allows libdwarf, objdump, readelf, dwarfdump, etc. to continue to work, which says we need to put multiple 'programs' worth of debug data in each DWARF section. But then how do the tools (and the debugger) differentiate between data relevant for B() in the context of kernel foo() from B() in the context of kernel bar()?

Any thoughts, comments, suggestions, proofs that it can or cannot work, would be gratefully accepted.

Thanks,
Richard





_______________________________________________

Dwarf-Discuss mailing list

Dwarf-Discuss at lists.dwarfstd.org<mailto:Dwarf-Discuss at lists.dwarfstd.org>

http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/private.cgi/dwarf-discuss-dwarfstd.org/attachments/20110104/4d9e0461/attachment.htm>


From chris.quenelle@oracle.com  Tue Jan  4 18:26:47 2011
From: chris.quenelle@oracle.com (Chris Quenelle)
Date: Tue, 04 Jan 2011 10:26:47 -0800
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single file...
In-Reply-To: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>
Message-ID: <4D236667.7080609@oracle.com>

My advice is to modify the compiler so it takes the name of a kernel
routine on the command line and only generates the code
that's necessary for that kernel.  Then invoke the compiler once
for each kernel routine using the same source file each time.
You didn't say if all the source is in one file. If you are combining
multiple OpenCL source files, then the modifications to the compiler
are more involved, but still easier than trying to parse, read and rewrite
dwarf information that corresponds to a subset of the original program.  :-)
I'm not trying to be flippant here, I think I'm suggesting a much better 
design
that will be easier to implement and maintain over time.

--chris

On Monday January 3    4:41PM, Relph, Richard wrote:
>
> Our platform allows the creation of a single ELF file that contains 
> multiple 'programs' derived from the same set of source files. We're 
> trying to figure out the best way to represent this in DWARF. We've 
> thought about creating separate sets of DWARF sections for each 
> 'program' (say, .debug_info_programA, etc.), but this would render 
> most DWARF processing tools useless. Yet it isn't obvious to me how to 
> stuff multiple program's worth of DWARF info using just the one set of 
> standard DWARF sections.
>
> For the technically curious, here's the details:
>
> Our platform is a GPU. Each program is loaded on to the hardware 
> fresh, from 'outside', by an 'external' OS, if you will. When the GPU 
> starts executing, it starts from address 0, where we have placed a 
> bootstrap sequence that ends up calling the real code. The real code's 
> source language is OpenCL, which doesn't define a solitary entry point 
> function such as main(), but rather defines a 'kernel' attribute that 
> can be attached to any function. The host code (running on the CPU 
> controlling the GPU) specifies which kernel function to execute at 
> run-time.
>
> Because our GPU doesn't have a 'real' CALL instruction, we essentially 
> treat each 'kernel' function as a complete program in and of itself 
> (ALL functions called by the kernel are inlined by the compiler.) So a 
> single OpenCL source file might have (pseudo-code):
>
> kernel foo() {
>
>     A();
>
>     B();
>
> }
>
> kernel bar() {
>
>     B();
>
>     C();
>
> }
>
> A() {...}
>
> B() {...}
>
> C() {...}
>
> The compiler is invoked once for the source file, producing code and 
> DWARF information for the entire compilation unit. If only that was 
> the end...
>
> But because we don't want the resources utilized by kernel bar() to 
> interfere with resource allocation for kernel foo() (and vice-versa), 
> we post-process this, stripping unused kernels and other dead code and 
> data, producing separately loadable code for each kernel. Each 
> separately loadable kernel is then stored in the ELF for execution 
> when the host program requests it.
>
> The question at hand is how to represent the DWARF information for 
> each separately loadable kernel. We'd like a solution that allows 
> libdwarf, objdump, readelf, dwarfdump, etc. to continue to work, which 
> says we need to put multiple 'programs' worth of debug data in each 
> DWARF section. But then how do the tools (and the debugger) 
> differentiate between data relevant for B() in the context of kernel 
> foo() from B() in the context of kernel bar()?
>
> Any thoughts, comments, suggestions, proofs that it can or cannot 
> work, would be gratefully accepted.
>
> Thanks,
>
> Richard
>
>
> _______________________________________________
> Dwarf-Discuss mailing list
> Dwarf-Discuss at lists.dwarfstd.org
> http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/private.cgi/dwarf-discuss-dwarfstd.org/attachments/20110104/f3f80fd6/attachment.htm>


From John.DelSignore@roguewave.com  Tue Jan  4 18:29:38 2011
From: John.DelSignore@roguewave.com (John DelSignore)
Date: Tue, 4 Jan 2011 13:29:38 -0500
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single	file...
In-Reply-To: <7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C3C0@sausexmbp02.amd.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>	<4D2283B8.9040308@eagercon.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C3C0@sausexmbp02.amd.com>
Message-ID: <4D236712.8060202@roguewave.com>

Hi Richard,

I don't know much about OpenCL on GPUs, but I do know a lot about debugging CUDA on GPUs and debugging the Cell Broadband Engine (PPC64/SPU).

IMHO, when debugging a GPU (or the Cell), a very important issue that must be dealt with is separating the address space of the host program (x86*/PPC) from the accelerator (GPU/SPU). In other words, the address space of the host code must be separated from the address space of the accelerator, and the debugger's modeling of the process, address space, and threads must change. So roughly speaking:

* The process becomes a "bag" of the following stuff:
** A host address space.
** A collection of host threads that share the host address space.
** A collection of threads with discrete address spaces representing the GPU/SPU address space and its associated execution context.

The TotalView CUDA debugging model is explained in more detail in the following document: http://www.totalviewtech.com/support/documentation/pdf/CUDATotalViewDebuggerUsersGuideSupplement_V4.pdf

Note that the "processor architecture" is modeled at the thread level, so a process can contain threads with different architectures. For example, in a Linux-x86_64 CUDA program, the host threads are linux-x86_64 and the GPU threads are SAS. And in a Linux-Cell program, the host threads are linux-ppc64 and the SPU threads are SPU.

Anyway... The interesting part for this discussion is on page 5. Associated with each address space (host and GPU) is one or more ELF images. The Linux address space has associated with it the Linux-x86_64 executable and shared libraries. The CUDA address spaces that have their own list of ELF images. There is no attempt to smash together host and GPU information into a single ELF image; I think attempting to do so is a fundamental mistake, and it will be nearly impossible to untangle the mess. You may already be doing this, but it seems to me that at a minimum you should be separating the GPU ELF image from the host ELF image, even if the GPU ELF image is embedded (as data) in the host ELF image.

In both the CUDA and Cell worlds, when an ELF image is loaded onto the accelerator, the debugger receives an event, much like a "dlopen()" event. For Cell, the SPU ELF image may be part of the host executable file or stored separately. For CUDA, the GPU ELF image is calculated by the device driver. In both cases, the SPU/GPU ELF image is a self-contained ELF file containing debug information purely for the accelerator; it does not contain any host debug information.

Some of what follows is easily observable in any CUDA program, but given that you're "@amd.com" I assume that at some level you're probably competing with NVIDIA. So you should stop reading now if you don't want to be contaminated with knowledge of how multiple programs are represented in CUDA.

More comments in-line below...

Relph, Richard wrote:
> Michael,
>     Thanks for the reply. Below, answers to your questions...
> 
>> A couple questions:
>>
>> How is this ELF executable file organized?
>>
>> How are these "multiple programs" represented in the executable file?
> Each kernel's "text" is concatenated and placed in the same section
> and entries are made in the ELF symbol table to note the offset in to
> that section for each kernel. But only ONE kernel is ever loaded at a
> time and the loaded kernel's text is always located starting at 0. So
> if we have N kernels, we have N address 0, N address 1, and so on.

OK, so this is a little different than the CUDA case. In CUDA, the GPU ELF image can contain multiple kernels. Each kernel is stored in its own .text_<kernel_name> section and dynamically relocated when the image is loaded onto the GPU. All of the GPU .text_<kernel_name> sections are linked at 0 and loaded at (a seemingly random) non-0 address. The debugger has a way to figure out which kernel is going to be executed.

When we get a "GPU ELF image loaded" event, the CUDA debug API gives us a copy of both the non-relocated GPU ELF image (from the linker) and the relocated GPU ELF image (from the device driver). Since TotalView is a parallel debugger and wants to share the GPU ELF image across process boundaries, we fix-up the non-relocated GPU ELF image such that the GPU .text_<kernel_name> sections have a unique (non-0) link address. TotalView then applies the .rel* section relocations to correct the link addresses in the DWARF section. This gives us a fixed-up non-relocated GPU ELF image that can be shared across processes. TotalView then uses the relocated GPU ELF image provided by the debug API to figure out the image load relocations performed by the device driver for that address space instance, which allows us to map a link address to/from a load address.

>> How are these "separately loaded kernels" selected and loaded?
> At execution time, the host program will access the entire ELF file
> and then load each kernel as needed... sort of like a DLL being
> identified and then entries in the DLL being invoked as needed...
> except that in our case, "relocation" or loading occurs as each
> kernel is invoked, not when the ELF is identified, which (I think) is
> the crux of the problem.

This shouldn't be a problem as long as there is no overlap in the kernel link addresses or you have a way to fix up the original non-relocated GPU ELF image. You said that the kernels are appended together, but is that a "dumb" append or are they linked together such that each kernel gets its own link address? If each kernel gets its own link address, the debugger can tell which kernel is loaded, the kernel boundaries are clear (e.g., in their own section), and the relocation information is preserved, then the debugger should be able to sort it out.

>> DWARF doesn't depend on any particular program structure.  There is
>> no requirement to have a main() function or a single entry point.
> Understood. But is there an implied requirement that a given
> instruction location map to no more than one 'source line'?

I'm not sure what DWARF says about this, but it sure doesn't make any sense to me to have a given instruction location map to more than one source line, block, or subroutine.

> That is,
> after the (fixed length) bootstrap code, we have at location x
> different functions depending on which kernel has been invoked. I
> don't see how to represent this one-to-many mapping in a single
> instance of, say, the line number program.

I don't think you can. Like I said above, for CUDA programs TotalView fixes up the GPU ELF images to separate out the kernels, and applies the .rel* section relocations to fix-up the DWARF.

>> It would seem that the DWARF data for your program, with multiple
>> "kernels", should describe them  correctly.  Of course, when the ELF
>> file is modified to strip out code, you need to make the corresponding
>> changes to the DWARF data as well.
> Understood. Given an original source program, it's "full"
> representation as code, and the corresponding DWARF, I can produce
> for each individual kernel it's 'stripped' code and DWARF for that.
> And I know how to concatenate the stripped code for each kernel in to
> a the form that our system expects so that each kernel can be loaded
> and executed separately. But I don't know how to store the DWARF for
> each kernel in a single ELF file so that the standard tools can still
> be useful.

I suppose your choices are:
1) have the device driver provide the relocated executable to the tool, or
2) preserve the .rel* relocation sections and require that the tool to apply the relocations.

AFAIK, CUDA-GDB (provided by NVIDIA) uses technique #1. TotalView uses technique #2.

Hope this help...

Cheers, John D.


> Thanks,
> Richard
> _______________________________________________
> Dwarf-Discuss mailing list
> Dwarf-Discuss at lists.dwarfstd.org
> http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org
> 



From eager@eagercon.com  Tue Jan  4 18:50:16 2011
From: eager@eagercon.com (Michael Eager)
Date: Tue, 04 Jan 2011 10:50:16 -0800
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single file...
In-Reply-To: <7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C594@sausexmbp02.amd.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>	<4D23639D.2020700@oracle.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C594@sausexmbp02.amd.com>
Message-ID: <4D236BE8.3030704@eagercon.com>

Relph, Richard wrote:
> Chris,
> 
>     Thanks for the reply. We have considered this approach, but it has 
> thus far been rejected for various reasons. Ultimately, what you suggest 
> ends up producing a separate ELF with DWARF for each kernel, which has 
> the obvious advantage of being easy to prove works. But OpenCL treats a 
> compilation of a set of kernels as a unit, so we?d have to design a 
> mechanism to put multiple ELFs in to a single bundle, which we?ve so far 
> been reluctant to do.

Perhaps treat each kernel as a separate loadable ELF segment,
each starting at address zero.

-- 
Michael Eager	 eager at eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077




From Richard.Relph@amd.com  Tue Jan  4 19:19:15 2011
From: Richard.Relph@amd.com (Relph, Richard)
Date: Tue, 4 Jan 2011 13:19:15 -0600
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single	file...
In-Reply-To: <4D236712.8060202@roguewave.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>
	<4D2283B8.9040308@eagercon.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C3C0@sausexmbp02.amd.com>
	<4D236712.8060202@roguewave.com>
Message-ID: <7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C651@sausexmbp02.amd.com>

John,
    Thanks for the reply. Responses in-line...

> I don't know much about OpenCL on GPUs, but I do know a lot about
> debugging CUDA on GPUs and debugging the Cell Broadband Engine
> (PPC64/SPU).
That's helpful, I'm sure.

> IMHO, when debugging a GPU (or the Cell), a very important issue that
> must be dealt with is separating the address space of the host program
> (x86*/PPC) from the accelerator (GPU/SPU).
True enough. We're using DW_AT_address_class for this. In our implementation of OpenCL, there's host, constant, global, region, local, and private memory spaces, all distinct. We don't concern ourselves with host address space, since the kernel's we are describing and debugging don't have access to host address space. The kernel is strictly executed on the GPU, under the control of the CPU, which has the means for allocating, mapping to host memory, reading, and writing SOME of the address spaces.

> So roughly speaking:
> 
> * The process becomes a "bag" of the following stuff:
> ** A host address space.
> ** A collection of host threads that share the host address space.
> ** A collection of threads with discrete address spaces representing
> the GPU/SPU address space and its associated execution context.
Our ELF/DWARF is solely concerned with the last of these... and these are created dynamically by the host at run-time.

> Anyway... The interesting part for this discussion is on page 5.
> Associated with each address space (host and GPU) is one or more ELF
> images. The Linux address space has associated with it the Linux-x86_64
> executable and shared libraries. The CUDA address spaces that have
> their own list of ELF images. There is no attempt to smash together
> host and GPU information into a single ELF image; I think attempting to
> do so is a fundamental mistake, and it will be nearly impossible to
> untangle the mess. You may already be doing this, but it seems to me
> that at a minimum you should be separating the GPU ELF image from the
> host ELF image, even if the GPU ELF image is embedded (as data) in the
> host ELF image.
We don't try to comingle host and GPU code in a single image. We are solely concerned with the representation of the code and data that may be run on the GPU. Debugging host code is a separate problem, but one well understood by developers generally for their preferred development platform. We don't attempt to replace Visual Studio or gdb for host code debugging.

Perhaps backing up a bit will help here... OpenCL defines a set of APIs for creating, compiling, and executing kernels at run-time. Most OpenCL-based programs exist solely as a host program while on disk and not executing. It's only when the host code is executed that the OpenCL kernels are compiled (usually - there is an 'off-line' compilation capability, but it is not used much.) Yes, we have a full OpenCL compiler in the APIs that implement the OpenCL run-time. Compilation of an OpenCL program consisting of one or more kernels is one API call. Binding of buffers and values to kernel arguments occurs and then a specific kernel is 'enqueued' for execution on the GPU (which usually involves hundreds to millions of threads being dispatched on the GPU.) The ELF/DWARF we're referring to here is transient... it only lives as long as the host program wants it to. True, it CAN be saved, but seldom is.

> >> How are these "multiple programs" represented in the executable
> file?
> > Each kernel's "text" is concatenated and placed in the same section
> > and entries are made in the ELF symbol table to note the offset in to
> > that section for each kernel. But only ONE kernel is ever loaded at a
> > time and the loaded kernel's text is always located starting at 0. So
> > if we have N kernels, we have N address 0, N address 1, and so on.
> 
> OK, so this is a little different than the CUDA case. In CUDA, the GPU
> ELF image can contain multiple kernels. Each kernel is stored in its
> own .text_<kernel_name> section and dynamically relocated when the
> image is loaded onto the GPU. All of the GPU .text_<kernel_name>
> sections are linked at 0 and loaded at (a seemingly random) non-0
> address. The debugger has a way to figure out which kernel is going to
> be executed.
This sounds like it more closely matches the DLL model, where relocation of a set of kernels occurs when the ELF image is opened.

> >> How are these "separately loaded kernels" selected and loaded?
> > At execution time, the host program will access the entire ELF file
> > and then load each kernel as needed... sort of like a DLL being
> > identified and then entries in the DLL being invoked as needed...
> > except that in our case, "relocation" or loading occurs as each
> > kernel is invoked, not when the ELF is identified, which (I think) is
> > the crux of the problem.
> 
> This shouldn't be a problem as long as there is no overlap in the
> kernel link addresses or you have a way to fix up the original non-
> relocated GPU ELF image.
That's the problem. There IS overlap in the kernel link addresses. ALL kernels start at 0. Only 1 kernel is loaded at a time.
(As an aside, there's no "relocation" necessary. Each kernel 'image' is ready for loading at 0.)

> You said that the kernels are appended
> together, but is that a "dumb" append or are they linked together such
> that each kernel gets its own link address?
It's a dumb append. The first kernel is at offset 0, then the second kernel follows immediately after the first, and so on.

> If each kernel gets its own
> link address, the debugger can tell which kernel is loaded, the kernel
> boundaries are clear (e.g., in their own section), and the relocation
> information is preserved, then the debugger should be able to sort it
> out.
The debugger knows what kernel is 'loaded' because the debugger is snooping the OpenCL APIs and 'knows' which kernel the host program requested.

> >> DWARF doesn't depend on any particular program structure.  There is
> >> no requirement to have a main() function or a single entry point.
> > Understood. But is there an implied requirement that a given
> > instruction location map to no more than one 'source line'?
> 
> I'm not sure what DWARF says about this, but it sure doesn't make any
> sense to me to have a given instruction location map to more than one
> source line, block, or subroutine.
Agreed. Hence my sense that I need to split the DWARF information for each kernel somehow, so that the DWARF for one kernel doesn't get confused with the DWARF for other kernels. I was thinking of simply appending the kernel name to each of the standard DWARF section names to create a set of kernel-specific DWARF sections, but that would obvious 'break' tools like libdwarf, dwarfdump, etc. So I came to the list to see if there was something I'm missing...

> > That is,
> > after the (fixed length) bootstrap code, we have at location x
> > different functions depending on which kernel has been invoked. I
> > don't see how to represent this one-to-many mapping in a single
> > instance of, say, the line number program.
> 
> I don't think you can.
Sadly, I don't think I can either... But I was hoping. ;-)

Thanks,
Richard




From dgross@cup.hp.com  Tue Jan  4 19:40:15 2011
From: dgross@cup.hp.com (David Gross)
Date: Tue, 04 Jan 2011 11:40:15 -0800
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
	single file...
In-Reply-To: Your message of "Tue, 04 Jan 2011 13:19:15 CST."
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C651@sausexmbp02.amd.com> 
Message-ID: <201101041940.p04JeFc04842@adlwrk04.cup.hp.com>

Ralph wrote:

> > >> DWARF doesn't depend on any particular program structure.  There is
> > >> no requirement to have a main() function or a single entry point.
> > > Understood. But is there an implied requirement that a given
> > > instruction location map to no more than one 'source line'?
> > 
> > I'm not sure what DWARF says about this, but it sure doesn't make any
> > sense to me to have a given instruction location map to more than one
> > source line, block, or subroutine.

> Agreed. Hence my sense that I need to split the DWARF information
> for each kernel somehow, so that the DWARF for one kernel doesn't
> get confused with the DWARF for other kernels. I was thinking of
> simply appending the kernel name to each of the standard DWARF
> section names to create a set of kernel-specific DWARF sections, but
> that would obvious 'break' tools like libdwarf, dwarfdump, etc. So I
> came to the list to see if there was something I'm missing...

Is it sufficient to put each kernel in its own DW_TAG_compile_unit?

David



From eager@eagercon.com  Tue Jan  4 19:51:59 2011
From: eager@eagercon.com (Michael Eager)
Date: Tue, 04 Jan 2011 11:51:59 -0800
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single	file...
In-Reply-To: <7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C651@sausexmbp02.amd.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>	<4D2283B8.9040308@eagercon.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C3C0@sausexmbp02.amd.com>
	<4D236712.8060202@roguewave.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C651@sausexmbp02.amd.com>
Message-ID: <4D237A5F.7050001@eagercon.com>

Relph, Richard wrote:

>>> Understood. But is there an implied requirement that a given instruction location map to no
>>> more than one 'source line'?
>> I'm not sure what DWARF says about this, but it sure doesn't make any sense to me to have a
>> given instruction location map to more than one source line, block, or subroutine.

When multiple code paths are merged, one instruction can logically
be part of each path, with different source locations.

> Agreed. Hence my sense that I need to split the DWARF information for each kernel somehow, so
> that the DWARF for one kernel doesn't get confused with the DWARF for other kernels. I was
> thinking of simply appending the kernel name to each of the standard DWARF section names to
> create a set of kernel-specific DWARF sections, but that would obvious 'break' tools like
> libdwarf, dwarfdump, etc. So I came to the list to see if there was something I'm missing...

I'm not sure why there would be any confusion.
Reference in DWARF to machine addresses are resolved
by the linker.  If the linker knows how to resolve
an address to the correct kernel, it should
be correct.

You do need to have DWARF which describes each
kernel separately.  You can't have a single DWARF
description for function B() when there are multiple
copies of that routine at different locations in
different kernels.

This "kernel" organization looks to me a lot like
overlays.  I'm not exactly sure how other compilers
have handled this in ELF/DWARF.

-- 
Michael Eager	 eager at eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077



From Richard.Relph@amd.com  Tue Jan  4 21:08:21 2011
From: Richard.Relph@amd.com (Relph, Richard)
Date: Tue, 4 Jan 2011 15:08:21 -0600
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single	file...
In-Reply-To: <4D237A5F.7050001@eagercon.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>
	<4D2283B8.9040308@eagercon.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C3C0@sausexmbp02.amd.com>
	<4D236712.8060202@roguewave.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C651@sausexmbp02.amd.com>
	<4D237A5F.7050001@eagercon.com>
Message-ID: <7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C804@sausexmbp02.amd.com>

> You do need to have DWARF which describes each
> kernel separately.
This is the conclusion I've reached.

> This "kernel" organization looks to me a lot like
> overlays.  I'm not exactly sure how other compilers
> have handled this in ELF/DWARF.
Oh my... Overlays... I'd completely forgotten that term... But that you mentioned it and that I recognized it shows something about our age, doesn't it... ;-)
But you're right, this is very much like the old MS-DOS (or was it CP/M?) style overlays of too many years ago. A program counter address isn't enough information alone to figure out a source file line, you need to know which 'overlay' - or kernel - is currently loaded.

Richard


From eager@eagercon.com  Tue Jan  4 21:19:06 2011
From: eager@eagercon.com (Michael Eager)
Date: Tue, 04 Jan 2011 13:19:06 -0800
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single	file...
In-Reply-To: <7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C804@sausexmbp02.amd.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>	<4D2283B8.9040308@eagercon.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C3C0@sausexmbp02.amd.com>
	<4D236712.8060202@roguewave.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C651@sausexmbp02.amd.com>
	<4D237A5F.7050001@eagercon.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C804@sausexmbp02.amd.com>
Message-ID: <4D238ECA.3050503@eagercon.com>

Relph, Richard wrote:
>> You do need to have DWARF which describes each
>> kernel separately.
> This is the conclusion I've reached.
> 
>> This "kernel" organization looks to me a lot like
>> overlays.  I'm not exactly sure how other compilers
>> have handled this in ELF/DWARF.
> Oh my... Overlays... I'd completely forgotten that term... But that you mentioned it and that I recognized it shows something about our age, doesn't it... ;-)
> But you're right, this is very much like the old MS-DOS (or was it CP/M?) style overlays of too many years ago. A program counter address isn't enough information alone to figure out a source file line, you need to know which 'overlay' - or kernel - is currently loaded.

Actually, I was working with a DSP not long ago
which had a very involved overlay structure, where
applications were loaded over each other in memory.
And overlays predate CP/M or MS/DOS by quite a while.

Binutils has support for overlays, although I
can't say that I recall much about the details.

-- 
Michael Eager	 eager at eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077



From John.DelSignore@roguewave.com  Tue Jan  4 21:25:12 2011
From: John.DelSignore@roguewave.com (John DelSignore)
Date: Tue, 4 Jan 2011 16:25:12 -0500
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single	file...
In-Reply-To: <7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C804@sausexmbp02.amd.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>	<4D2283B8.9040308@eagercon.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C3C0@sausexmbp02.amd.com>
	<4D236712.8060202@roguewave.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C651@sausexmbp02.amd.com>
	<4D237A5F.7050001@eagercon.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C804@sausexmbp02.amd.com>
Message-ID: <4D239038.9000205@roguewave.com>

I had the same thought: your kernels work kind-of like overlays, but I didn't mention it because I think DWARF does not support overlays properly, and you'd need to resort to other hackery. See the following DWARF issue:

http://www.dwarfstd.org/ShowIssue.php?issue=090402.1&type=closed3

YMMV.

Cheers, John D.


Relph, Richard wrote:
>> You do need to have DWARF which describes each
>> kernel separately.
> This is the conclusion I've reached.
> 
>> This "kernel" organization looks to me a lot like
>> overlays.  I'm not exactly sure how other compilers
>> have handled this in ELF/DWARF.
> Oh my... Overlays... I'd completely forgotten that term... But that you mentioned it and that I recognized it shows something about our age, doesn't it... ;-)
> But you're right, this is very much like the old MS-DOS (or was it CP/M?) style overlays of too many years ago. A program counter address isn't enough information alone to figure out a source file line, you need to know which 'overlay' - or kernel - is currently loaded.
> 
> Richard



From davea42@earthlink.net  Tue Jan  4 22:12:06 2011
From: davea42@earthlink.net (David Anderson)
Date: Tue, 04 Jan 2011 14:12:06 -0800
Subject: [Dwarf-Discuss] How best to represent multiple programs in a
 single	file...
In-Reply-To: <7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C804@sausexmbp02.amd.com>
References: <7691DC9FC60C9E4F88A0BBCA628F55FF045493DE26@sausexmbp02.amd.com>	<4D2283B8.9040308@eagercon.com>	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C3C0@sausexmbp02.amd.com>	<4D236712.8060202@roguewave.com>	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C651@sausexmbp02.amd.com>	<4D237A5F.7050001@eagercon.com>
	<7691DC9FC60C9E4F88A0BBCA628F55FF0454A3C804@sausexmbp02.amd.com>
Message-ID: <4D239B36.5010603@earthlink.net>

On 01/04/2011 01:08 PM, Relph, Richard wrote:
> Oh my... Overlays... I'd completely forgotten that term... But that you mentioned it and that I recognized it shows something about our age, doesn't it... ;-)
> But you're right, this is very much like the old MS-DOS (or was it CP/M?) style overlays of too many years ago.
The Wikipedia entry for "overlay (programming)"  Seems to be unaware of 
anything
earlier than the IBM PC!           Jean Sammet mentions all the basic
characteristics of overlays in  her 1969 book "Programming Languages, 
History and Fundamentals"
in a few sentences, though without any detail.  Look  for "overlays" in 
the index.

John R. Levine, in his book "Linkers and Loaders", says (p 177) "The 
final topic of this chapter
is tree-structured overlays, an approach that was widely used in the 
days before there was virtual
memory to fit a program into a memory area smaller than the program.  
Overlays are another
technique that dates back to before 1960 and are still in use in some 
memory-constrained environments."
and  "The OS 360 overlay manager is only about 500 bytes." (IBM OS 360 
debuted in 1964 with
quite a bit of flair and pizazz).

Sorry, off-topic.   But I did want to mention prior-art!
DavidA.



From brobecker@adacore.com  Wed Jan 19 20:35:10 2011
From: brobecker@adacore.com (Joel Brobecker)
Date: Wed, 19 Jan 2011 15:35:10 -0500
Subject: [Dwarf-Discuss] question about [0x0..0xffffffff] location list entry
Message-ID: <20110119203510.GA28308@adacore.com>

Hello,

I was wondering if I could have your opinion on the following DWARF
data...

A user wrote some C code with a global variable defined in a unit.
This global variable is not static.  The DWARF debug info for it
looks like:

 <1><143010>: Abbrev Number: 135 (DW_TAG_variable)
    <143012>   DW_AT_decl_line   : 166
    <143014>   DW_AT_decl_file   : 26
    <143016>   DW_AT_type        : <0xe6e1>
    <14301a>   DW_AT_external    : 1
    <14301b>   DW_AT_location    : 0x90068      (location list)
    <14301f>   DW_AT_name        : blablabla

The location list entry, as dumped by readelf looks is:

    Offset   Begin    End      Expression
    00090068 0009f7c4 0009f7c3 (DW_OP_addr: 4000d120)
    00090068 <End of list>

A raw dump of the .debug_loc section at the offset where our
location list is located shows that begin and end offsets are
0x0 and 0xffffffff (address size is 32bits).

I believe that the intention behind this location list entry is
to say: variable lives at DW_OP_addr=4000d120 for all PC addresses
(infinite range).

But I also believe that this is not saying what they thing it is
expressing.  According to the way I understand the DWARF4 standard,
I believe that the above should mean that the variable is live
only for PC addresses starting at 0x0009f7c4 (the base address
of the unit containing that variable).  So the live range for
that address, as expressed, is [0x0009f7c4..+Inf[.

What do others think?

Thanks!
-- 
Joel



From roland@redhat.com  Wed Jan 19 20:50:34 2011
From: roland@redhat.com (Roland McGrath)
Date: Wed, 19 Jan 2011 12:50:34 -0800 (PST)
Subject: [Dwarf-Discuss] question about [0x0..0xffffffff] location list
	entry
In-Reply-To: Joel Brobecker's message of  Wednesday,
	19 January 2011 15:35:10 -0500
	<20110119203510.GA28308@adacore.com>
References: <20110119203510.GA28308@adacore.com>
Message-ID: <20110119205034.374401807D2@magilla.sf.frob.com>

> The location list entry, as dumped by readelf looks is:
> 
>     Offset   Begin    End      Expression
>     00090068 0009f7c4 0009f7c3 (DW_OP_addr: 4000d120)
>     00090068 <End of list>
> 
> A raw dump of the .debug_loc section at the offset where our
> location list is located shows that begin and end offsets are
> 0x0 and 0xffffffff (address size is 32bits).
> 
> I believe that the intention behind this location list entry is
> to say: variable lives at DW_OP_addr=4000d120 for all PC addresses
> (infinite range).

Any sensible compiler would just emit a direct DW_FORM_exprloc here instead
of using a location list at all.

> But I also believe that this is not saying what they thing it is
> expressing.  According to the way I understand the DWARF4 standard,
> I believe that the above should mean that the variable is live
> only for PC addresses starting at 0x0009f7c4 (the base address
> of the unit containing that variable).  So the live range for
> that address, as expressed, is [0x0009f7c4..+Inf[.

Assuming there is no previous base address entry in that location list,
then that's roughly correct.  What makes it "rough" is that I don't think
it's at all clear what overflowing/wrapping address computations are
supposed to mean.  It seems highly dubious that the producer should ever
emit values wherein the additions the consumer does would overflow like this.


Thanks,
Roland



From brobecker@adacore.com  Thu Jan 20 15:05:25 2011
From: brobecker@adacore.com (Joel Brobecker)
Date: Thu, 20 Jan 2011 10:05:25 -0500
Subject: [Dwarf-Discuss] question about [0x0..0xffffffff] location list
 entry
In-Reply-To: <20110119205034.374401807D2@magilla.sf.frob.com>
References: <20110119203510.GA28308@adacore.com>
	<20110119205034.374401807D2@magilla.sf.frob.com>
Message-ID: <20110120150525.GI710@adacore.com>

> Assuming there is no previous base address entry in that location list,
> then that's roughly correct.

I confirm that there was only 1 entry in that list, and that was the
normal location list entry.

> What makes it "rough" is that I don't think it's at all clear what
> overflowing/wrapping address computations are supposed to mean.  It
> seems highly dubious that the producer should ever emit values wherein
> the additions the consumer does would overflow like this.

Indeed.

I've trying to convince the producer that the debugging info they
generated is wrong on the basis that "it doesn't say what you want
it to say", but we also agree that it's peculiar at best. If we can
agree that overflow situation are erroneous in the sense that
consumers are allowed to do whatever they want, that would be good
enough for me.  However, if we decide to define what this means,
then I can adjust GDB to follow that decision...

Thanks for the feedback.
-- 
Joel



From eager@eagercon.com  Thu Jan 20 16:54:19 2011
From: eager@eagercon.com (Michael Eager)
Date: Thu, 20 Jan 2011 08:54:19 -0800
Subject: [Dwarf-Discuss] question about [0x0..0xffffffff] location list
 entry
In-Reply-To: <20110119203510.GA28308@adacore.com>
References: <20110119203510.GA28308@adacore.com>
Message-ID: <4D3868BB.4070406@eagercon.com>

Joel Brobecker wrote:

> The location list entry, as dumped by readelf looks is:
> 
>     Offset   Begin    End      Expression
>     00090068 0009f7c4 0009f7c3 (DW_OP_addr: 4000d120)
>     00090068 <End of list>

This is an invalid location list.

DWARF 4, Section 2.6.2, page 30:

   A location list entry consists of:

   1. A beginning address offset. This address offset has
      the size of an address and is relative to the applicable
      base address of the compilation unit referencing this
      location list. It marks the beginning of the address
      range over which the location is valid.

   2. An ending address offset. This address offset again has
      the size of an address and is relative to the applicable
      base address of the compilation unit referencing this
      location list. It marks the first address past the end
      of the address range over which the location is valid.
      The ending address must be greater than or equal to the
      beginning address.

In this case, the LocList violates the requirements for an
ending address.

DWARF does not mention overflow in address computations.
(Not that there is an address computation here.)  Nor does
it mention address ranges wrapping.

The best interpretation I can give for this LocList is that
the live range for the variable is empty.  But that would
be an attempt to give meaning to an illegal DWARF construct.

The correct LocList for this variable would be [0..0xffffffff].

-- 
Michael Eager	 eager at eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077



From brobecker@adacore.com  Thu Jan 20 17:17:09 2011
From: brobecker@adacore.com (Joel Brobecker)
Date: Thu, 20 Jan 2011 12:17:09 -0500
Subject: [Dwarf-Discuss] question about [0x0..0xffffffff] location list
 entry
In-Reply-To: <4D3868BB.4070406@eagercon.com>
References: <20110119203510.GA28308@adacore.com>
	<4D3868BB.4070406@eagercon.com>
Message-ID: <20110120171709.GK710@adacore.com>

Hi Michael,

Thanks a lot for your very well documented answer.

> In this case, the LocList violates the requirements for an
> ending address.
[...]
> The correct LocList for this variable would be [0..0xffffffff].

I am just a little confused by your last sentence.  In offsets,
in the debug_loc section, are 0..0xffffffff, but once we add
the base address, we get the strange empty range.  GDB treated
that range as empty, so were we correct? If we were, out of curiosity,
how would one get the LocList above?

-- 
Joel



From roland@redhat.com  Thu Jan 20 18:29:18 2011
From: roland@redhat.com (Roland McGrath)
Date: Thu, 20 Jan 2011 10:29:18 -0800 (PST)
Subject: [Dwarf-Discuss] question about [0x0..0xffffffff] location list
 entry
In-Reply-To: Michael Eager's message of  Thursday,
	20 January 2011 08:54:19 -0800 <4D3868BB.4070406@eagercon.com>
References: <20110119203510.GA28308@adacore.com>
	<4D3868BB.4070406@eagercon.com>
Message-ID: <20110120182918.62CED1807D1@magilla.sf.frob.com>

> DWARF does not mention overflow in address computations.
> (Not that there is an address computation here.)  Nor does
> it mention address ranges wrapping.

There are indeed two address computations here: adding the base address to
each of the two addresses in the location list entry.  I think it would be
helpful for DWARF to say explicitly that such calculations should not
overflow/wrap.  That is, if the base address plus a list entry address
exceeds the maximum address representable by the CU's address_size (as in
this example), the list entry is invalid.


Thanks,
Roland



From roland@redhat.com  Thu Jan 20 18:42:34 2011
From: roland@redhat.com (Roland McGrath)
Date: Thu, 20 Jan 2011 10:42:34 -0800 (PST)
Subject: [Dwarf-Discuss] question about [0x0..0xffffffff] location list
	entry
In-Reply-To: Joel Brobecker's message of  Thursday,
	20 January 2011 12:17:09 -0500 <20110120171709.GK710@adacore.com>
References: <20110119203510.GA28308@adacore.com>
	<4D3868BB.4070406@eagercon.com> <20110120171709.GK710@adacore.com>
Message-ID: <20110120184234.300161807D1@magilla.sf.frob.com>

> > In this case, the LocList violates the requirements for an
> > ending address.
> [...]
> > The correct LocList for this variable would be [0..0xffffffff].
> 
> I am just a little confused by your last sentence.  In offsets,
> in the debug_loc section, are 0..0xffffffff, but once we add
> the base address, we get the strange empty range.  GDB treated
> that range as empty, so were we correct? 

If we were to assume "normal wrapping arithmetic" in the CU address_size
(which I think should be called invalid too, but anyway), this entry is
invalid on its face.  As wrapped, the ending address is less than the
beginning address.  The paragraph Michael cited says that's invalid.
That's different from an empty range.

> If we were, out of curiosity, how would one get the LocList above?

If the relevant base address for that entry is B, then the location list
entry would contain a beginning address of (-B) and an ending address of
(0xffffffff-B).  As I just said above, nobody should be assuming that
expecting the consumer to do wrapping arithmetic like that is valid.  But,
in practice, if this entry had used those values, then the calculation that
GDB did would have come out so that the entry spanned [0,0xffffffff) and it
would have behaved as the producer apparently intended it to.  Of course,
as I mentioned before, when the intent is "applies to all addresses", the
proper thing to do is not to use a location list at all, but just a single
location description.


Thanks,
Roland



From eager@eagercon.com  Thu Jan 20 18:59:52 2011
From: eager@eagercon.com (Michael Eager)
Date: Thu, 20 Jan 2011 10:59:52 -0800
Subject: [Dwarf-Discuss] question about [0x0..0xffffffff] location list
 entry
In-Reply-To: <20110120182918.62CED1807D1@magilla.sf.frob.com>
References: <20110119203510.GA28308@adacore.com>	<4D3868BB.4070406@eagercon.com>
	<20110120182918.62CED1807D1@magilla.sf.frob.com>
Message-ID: <4D388628.6030701@eagercon.com>

Roland McGrath wrote:
>> DWARF does not mention overflow in address computations.
>> (Not that there is an address computation here.)  Nor does
>> it mention address ranges wrapping.
> 
> There are indeed two address computations here: adding the base address to
> each of the two addresses in the location list entry.  I think it would be
> helpful for DWARF to say explicitly that such calculations should not
> overflow/wrap.  That is, if the base address plus a list entry address
> exceeds the maximum address representable by the CU's address_size (as in
> this example), the list entry is invalid.

Please submit a comment at http://dwarfstd.org/Comment.php.

-- 
Michael Eager	 eager at eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077



From eager@eagercon.com  Thu Jan 20 19:11:49 2011
From: eager@eagercon.com (Michael Eager)
Date: Thu, 20 Jan 2011 11:11:49 -0800
Subject: [Dwarf-Discuss] question about [0x0..0xffffffff] location list
 entry
In-Reply-To: <20110120171709.GK710@adacore.com>
References: <20110119203510.GA28308@adacore.com>
	<4D3868BB.4070406@eagercon.com> <20110120171709.GK710@adacore.com>
Message-ID: <4D3888F5.5080509@eagercon.com>

Joel Brobecker wrote:

>> The correct LocList for this variable would be [0..0xffffffff].
> 
> I am just a little confused by your last sentence.  In offsets,
> in the debug_loc section, are 0..0xffffffff, but once we add
> the base address, we get the strange empty range.  GDB treated
> that range as empty, so were we correct? If we were, out of curiosity,
> how would one get the LocList above?

I had some second thoughts about this after I sent the email.

The end offset should be the highest address where the address
is valid, perhaps the end of the compilation unit.

To again quote the DWARF 4 standard:

   2.6.2 Location Lists

   Location lists are used in place of location expressions
   whenever the object whose location is being described can
   change location during its lifetime.

Since there is no intent that the object's location can change
during it's lifetime, use of a location list is inappropriate.
As this paragraph suggests, and as Roland said, a location
expression should be used when the location of an object
does not change during it's lifetime.

-- 
Michael Eager	 eager at eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077



From brobecker@adacore.com  Thu Jan 20 19:28:42 2011
From: brobecker@adacore.com (Joel Brobecker)
Date: Thu, 20 Jan 2011 14:28:42 -0500
Subject: [Dwarf-Discuss] question about [0x0..0xffffffff] location list
 entry
In-Reply-To: <4D388628.6030701@eagercon.com>
References: <20110119203510.GA28308@adacore.com>
	<4D3868BB.4070406@eagercon.com>
	<20110120182918.62CED1807D1@magilla.sf.frob.com>
	<4D388628.6030701@eagercon.com>
Message-ID: <20110120192842.GL710@adacore.com>

> >There are indeed two address computations here: adding the base address to
> >each of the two addresses in the location list entry.  I think it would be
> >helpful for DWARF to say explicitly that such calculations should not
> >overflow/wrap.  That is, if the base address plus a list entry address
> >exceeds the maximum address representable by the CU's address_size (as in
> >this example), the list entry is invalid.
> 
> Please submit a comment at http://dwarfstd.org/Comment.php.

Since I asked the questions that started this thread, I think it is
only fair if I submit the comment.

-- 
Joel



From louzonr@yahoo.com  Fri Jan 21 15:36:57 2011
From: louzonr@yahoo.com (Ron Louzon)
Date: Fri, 21 Jan 2011 07:36:57 -0800 (PST)
Subject: [Dwarf-Discuss] _dwarf_p_error(): Does it correctly handle negative
	error values?
Message-ID: <85126.41451.qm@web114709.mail.gq1.yahoo.com>

_dwarf_p_error() is defined in pro_error.c as
?void_dwarf_p_error(Dwarf_P_Debug dbg,
Dwarf_Error * error, Dwarf_Word errval)
{
Dwarf_Error errptr;
want to report the upper-level error, not this one. *//* Allow NULL dbg on 
entry, since sometimes that can happen and weprintf("ERROR VALUE: %ld - %s\n",
(
errptr = (Dwarf_Error)
_dwarf_p_get_alloc(dbg, 
fprintf(stderr,
"Could not allocate Dwarf_Error structure\n");
abort();
}
errptr->er_errval = (Dwarf_Sword) errval;
*error = errptr;
}
?
If?the supplied "errval" is negative, the conversion code "-errval-1" is used to 
convert "errval" to a positive index.?
?
Assume that?"errval" had a value of -5.? This conversion?generates an index of 4 
which is then used to retrieve a string from the _dwarf_errmsgs?table.? Was?it 
intended to have an index which is off by 1 or should the conversion be using 
the positive equivalent of "errval" as an index into the _dwarf_errmsgs?table?? 

?
It appears?to me that the conversion of errval to a positive number?should 
really be?
(
?
?I also could not find any usage in libdwarf of this function where a negative 
value was passed in through errval.
?
Thanks,
Ronunsigned)( -( (signed)errval ) );if((Dwarf_Sword) errval < 0)long) errval, 
_dwarf_errmsgs[-errval - 1]);if(error != NULL) 
{sizeof(structDwarf_Error_s));if(errptr == NULL) {return;


      



From louzonr@yahoo.com  Fri Jan 21 15:57:25 2011
From: louzonr@yahoo.com (Ron Louzon)
Date: Fri, 21 Jan 2011 07:57:25 -0800 (PST)
Subject: [Dwarf-Discuss] _dwarf_p_error(): Does it correctly handle
	negative error values?
In-Reply-To: <85126.41451.qm@web114709.mail.gq1.yahoo.com>
References: <85126.41451.qm@web114709.mail.gq1.yahoo.com>
Message-ID: <771643.10748.qm@web114718.mail.gq1.yahoo.com>


Sorry, somehow the original question became garbled.? Here is the same post in 
ungarbled form:
=======================================================================

_dwarf_p_error() is defined in pro_error.c as
void
_dwarf_p_error(Dwarf_P_Debug dbg,
?????????????? Dwarf_Error * error, Dwarf_Word errval)
{
??? Dwarf_Error errptr;
??? /* Allow NULL dbg on entry, since sometimes that can happen and we
?????? want to report the upper-level error, not this one. */
??? if ((Dwarf_Sword) errval < 0)
??????? printf("ERROR VALUE: %ld - %s\n",
??????????????? (long) errval, _dwarf_errmsgs[-errval - 1]);
??? if (error != NULL) {
??????? errptr = (Dwarf_Error)
??????????? _dwarf_p_get_alloc(dbg, sizeof(struct Dwarf_Error_s));
??????? if (errptr == NULL) {
??????????? fprintf(stderr,
??????????????????? "Could not allocate Dwarf_Error structure\n");
??????????? abort();
??????? }
??????? errptr->er_errval = (Dwarf_Sword) errval;
??????? *error = errptr;
??????? return;
??? }
?
If the supplied "errval" is negative, the conversion code "-errval-1" is used to 

convert "errval" to a positive index. 
?
Assume that "errval" had a value of -5.? This conversion generates an index of 4 

which is then used to retrieve a string from the _dwarf_errmsgs table.? Was it 
intended to have an index which is off by 1 or should the conversion be using 
the positive equivalent of "errval" as an index into the _dwarf_errmsgs table?? 
?
It appears to me that the conversion of errval to a positive number should 
really be 
(signed)( -( (signed)errval ) )
?
I also could not find any usage in libdwarf of this function where a negative 
value was passed in through errval.
?
Thanks,
Ron



      



From davea42@earthlink.net  Fri Jan 21 18:30:24 2011
From: davea42@earthlink.net (David Anderson)
Date: Fri, 21 Jan 2011 10:30:24 -0800
Subject: [Dwarf-Discuss] _dwarf_p_error(): Does it correctly
 handle	negative error values?
In-Reply-To: <771643.10748.qm@web114718.mail.gq1.yahoo.com>
References: <85126.41451.qm@web114709.mail.gq1.yahoo.com>
	<771643.10748.qm@web114718.mail.gq1.yahoo.com>
Message-ID: <4D39D0C0.5090001@earthlink.net>

On 01/21/2011 07:57 AM, Ron Louzon wrote:
> Sorry, somehow the original question became garbled.  Here is the same post in 
>
> ungarbled form:
>
...[ Thanks, that helps, deleted here to save space]
>   
> If the supplied "errval" is negative, the conversion code "-errval-1" is used to
>
> convert "errval" to a positive index.
>   
> Assume that "errval" had a value of -5.  This conversion generates an index of 4
>
> which is then used to retrieve a string from the _dwarf_errmsgs table.  Was it 
>
> intended to have an index which is off by 1 or should the conversion be using 
>
> the positive equivalent of "errval" as an index into the _dwarf_errmsgs table? 
>
>

I do not recall any intent to support negative numbers as error values, 
though
it is possible that an early version of libdwarf had negatives as error 
values.
Indeed I vaguely recall that libdwarf error numbers had negative values 
very early on
(meaning 1991-1994).     But now negative error values should not  be 
happening.

I cannot remember why  that libdwarf-internal function treats negatives 
as it does,
but it seems unimportant (unless there is something I am missing here).
Perhaps a comment in the source about the intent would be a good idea.

David Anderson

-- 
Maybe it's in the basement, let me go up stairs and check . --  M. C. Escher 




From louzonr@yahoo.com  Fri Jan 21 18:35:44 2011
From: louzonr@yahoo.com (Ron Louzon)
Date: Fri, 21 Jan 2011 10:35:44 -0800 (PST)
Subject: [Dwarf-Discuss] _dwarf_p_error(): Does it correctly handle
	negative error values?
In-Reply-To: <4D39D0C0.5090001@earthlink.net>
References: <85126.41451.qm@web114709.mail.gq1.yahoo.com>
	<771643.10748.qm@web114718.mail.gq1.yahoo.com>
	<4D39D0C0.5090001@earthlink.net>
Message-ID: <744950.68979.qm@web114710.mail.gq1.yahoo.com>

For me, the only issue with the code is that it causes Microsoft's C++ compilers 
to throw a warning about applying the unary "-" operator to an unsigned number.

- Ron


----- Original Message ----
From: David Anderson <davea42@earthlink.net>
To: dwarf-discuss at lists.dwarfstd.org
Sent: Fri, January 21, 2011 1:30:24 PM
Subject: Re: [Dwarf-Discuss] _dwarf_p_error(): Does it correctly handle negative 
error values?

...

I cannot remember why? that libdwarf-internal function treats negatives as it 
does,
but it seems unimportant (unless there is something I am missing here).
Perhaps a comment in the source about the intent would be a good idea.

David Anderson

-- Maybe it's in the basement, let me go up stairs and check . --? M. C. Escher 
_______________________________________________
Dwarf-Discuss mailing list
Dwarf-Discuss at lists.dwarfstd.org
http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org



      



From davea42@earthlink.net  Sun Jan 23 17:23:19 2011
From: davea42@earthlink.net (David Anderson)
Date: Sun, 23 Jan 2011 09:23:19 -0800
Subject: [Dwarf-Discuss] _dwarf_p_error(): Does it correctly
 handle	negative error values?
In-Reply-To: <744950.68979.qm@web114710.mail.gq1.yahoo.com>
References: <85126.41451.qm@web114709.mail.gq1.yahoo.com>	<771643.10748.qm@web114718.mail.gq1.yahoo.com>	<4D39D0C0.5090001@earthlink.net>
	<744950.68979.qm@web114710.mail.gq1.yahoo.com>
Message-ID: <4D3C6407.6030801@earthlink.net>

On 01/21/2011 10:35 AM, Ron Louzon wrote:
> For me, the only issue with the code is that it causes Microsoft's C++ compilers
> to throw a warning about applying the unary "-" operator to an unsigned number.
Ah. That's a  reason I understand. Can do.  Thanks!
david anderson

-- 
Maybe it's in the basement, let me go up stairs and check . --  M. C. Escher




From mankku86@yahoo.se  Thu Jan 27 20:43:13 2011
From: mankku86@yahoo.se (Magnus Sundell)
Date: Thu, 27 Jan 2011 20:43:13 +0000 (GMT)
Subject: [Dwarf-Discuss] Question about line number program
Message-ID: <651200.93279.qm@web26002.mail.ukl.yahoo.com>

Hello all!

First post to this list.

When processing the DWARF information of an ELF file, I ran into a line number program as clarified below (hope it shows up nice without too lousy formatting). I have tried reading the DWARF2, 3 and 4 specifications and had a quick peek at the libdwarf source code, but could not really determine how to interpret the actual line number program.

Thus, I would like to ask the following questions:

1) Could someone reveal to me, maybe through the below example, what the structure of the line program "entries" are?
2) How should the first entry, beginning with 0x00 0x05 be interpreted? The page "http://www.avrfreaks.net/index.php?name=PNphpBB2&file=viewtopic&t=34445" would make me want to interpret the 0x00 as DW_OP_xxx and 0x05 as DW_LNS_set_column. However this does not seem to make sense. I also considered the 0x00 belonging to the end of the file list, however 0x05 as a DW_OP_xxx does not make sense, either.


So, I thought that I'd try to avoid wasting time by posting to this list. I tried searching the net for a DWARF forum, but this was the closest that I got. I hope that no one will feel badly about me posting this kind of question here?

Thanks for reading the message,

// Magnus Sundell


Example follows:


=== Line Number Program Header ===
78 01 00 00 
02 00 
16 01 00 00 
02 
01 
fb 
0e 
0d 
00 01 01 01 01 00 00 00 01 00 00 01 
=== Include dirs ===
65 3a 2f 79 61 67 61 72 74 6f 2f 6c 69 62 2f 67 63 63 2f 2e 2e 2f 2e 2e 2f 61 72 6d 2d 6e 6f 6e 65 2d 65 61 62 69 2f 73 79 73 2d 69 6e 63 6c 75 64 65 00 
2e 2f 48 41 4c 00 
2e 2f 43 6f 6d 6d 6f 6e 00 
2e 2f 44 72 69 76 65 72 73 2f 4c 45 44 00 
2e 2f 52 54 4f 53 2f 46 72 65 65 52 54 4f 53 2f 69 6e 63 6c 75 64 65 00 
2e 2f 52 54 4f 53 2f 4f 53 41 62 73 74 72 61 63 74 69 6f 6e 00 
2e 2f 50 72 6f 74 6f 63 6f 6c 73 2f 55 54 50 00 
00 
=== File names ===
6d 61 69 6e 2e 63 00 00 00 00 
73 74 64 69 6e 74 2e 68 00 01 00 00 
73 74 6d 33 32 66 31 30 78 2e 68 00 02 00 00 
43 6f 6d 6d 6f 6e 54 79 70 65 73 2e 68 00 03 00 00 
4c 45 44 44 72 69 76 65 72 2e 68 00 04 00 00 
74 61 73 6b 2e 68 00 05 00 00 
4f 53 41 62 73 74 72 61 63 74 2e 68 00 06 00 00 
55 54 50 2e 68 00 07 00 00 
63 6f 72 65 5f 63 6d 33 2e 68 00 02 00 00 
00 
=== Line Number Program begins here ===
00 05 02 0c 01 00 
08 03 85 01 01 7a 59 68 00 
02 04 01 b8 7c 68 00 
02 04 01 80 7c 2f 85 2f 30 30 30 30 2f 4d 30 4d 30 4d 32 ae 32 ae 30 32 00 
02 04 01 4b 03 0f 2e 5a 5b 2f 5a 03 0d 2e 3f 3d 08 59 3f 00 
02 04 01 4d 00 
02 04 01 4c 5a 76 30 30 30 4b 67 5a 4c 4c 51 02 04 00 
01 01
=== End of LNP, the next octet belongs to next LNP header ===





From davea42@earthlink.net  Thu Jan 27 22:50:59 2011
From: davea42@earthlink.net (David Anderson)
Date: Thu, 27 Jan 2011 14:50:59 -0800
Subject: [Dwarf-Discuss] Question about line number program
In-Reply-To: <651200.93279.qm@web26002.mail.ukl.yahoo.com>
References: <651200.93279.qm@web26002.mail.ukl.yahoo.com>
Message-ID: <4D41F6D3.1010105@earthlink.net>

On 01/27/2011 12:43 PM, Magnus Sundell wrote:
> Hello all!
>
> First post to this list.
>
> When processing the DWARF information of an ELF file, I ran into a line number program as clarified below (hope it shows up nice without too lousy formatting). I have tried reading the DWARF2, 3 and 4 specifications and had a quick peek at the libdwarf source code, but could not really determine how to interpret the actual line number program.

Both readelf and dwarfdump are quite capable of showing you all the 
details of the DWARF information.
I suggest you get one or the other.

David Anderson

-- 
Free as in lawsuit.




From eager@eagercon.com  Thu Jan 27 23:03:50 2011
From: eager@eagercon.com (Michael Eager)
Date: Thu, 27 Jan 2011 15:03:50 -0800
Subject: [Dwarf-Discuss] Question about line number program
In-Reply-To: <651200.93279.qm@web26002.mail.ukl.yahoo.com>
References: <651200.93279.qm@web26002.mail.ukl.yahoo.com>
Message-ID: <4D41F9D6.7010704@eagercon.com>

Magnus Sundell wrote:
> Hello all!

Hi Magnus!

> When processing the DWARF information of an ELF file, I ran into a line number program as
> clarified below (hope it shows up nice without too lousy formatting). I have tried reading the
> DWARF2, 3 and 4 specifications and had a quick peek at the libdwarf source code, but could not
> really determine how to interpret the actual line number program.

To help understand what you are doing, please tell us a bit more:

   What processor are you using?
   What compiler are you using?
   What version of DWARF does it generate?
   What are you using to process the DWARF info?
   What are you trying to achieve?

> 1) Could someone reveal to me, maybe through the below example, what the structure of the line
> program "entries" are? 

My first suggestion is that you use a tool like readelf (in the GNU Binutils
package) or dwarfdump (in libdwarf).  (Why spend time trying to parse the
DWARF data when there are programs which do this for you?)

> 2) How should the first entry, beginning with 0x00 0x05 be interpreted?
> The page "http://www.avrfreaks.net/index.php?name=PNphpBB2&file=viewtopic&t=34445" would make me
> want to interpret the 0x00 as DW_OP_xxx and 0x05 as DW_LNS_set_column. However this does not seem
> to make sense. I also considered the 0x00 belonging to the end of the file list, however 0x05 as
> a DW_OP_xxx does not make sense, either.

The best aid to interpreting DWARF is the standard, available
at http://dwarfstd.org, not an email thread from 2005.  Take
a look at Chapter 7. Data Representation.

-- 
Michael Eager	 eager at eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077




