From tharrison@eci-usa.net  Thu Jul 14 15:36:06 2016
From: tharrison@eci-usa.net (Trevor Harrison)
Date: Thu, 14 Jul 2016 11:36:06 -0400
Subject: [Dwarf-Discuss] DWARF expression evaluation
Message-ID: <CAEaoM9dJKHpS5Vj8idcOAEGCMbD3_n8RNUi1sisWFnuHwPaZxQ@mail.gmail.com>

Hi,
  I'm trying to evaluate DWARF expressions and I'm looking for a hint
or some documentation about the state of the expression evaluator's
stack prior to beginning the expression.

  The reason I'm asking about this is I'm seeing expressions (a
DW_AT_data_member_location for example) that start with DW_OP_dup,
which implies that there is something on the stack for it to dup.

-Trevor


From jakub@redhat.com  Thu Jul 14 15:46:19 2016
From: jakub@redhat.com (Jakub Jelinek)
Date: Thu, 14 Jul 2016 17:46:19 +0200
Subject: [Dwarf-Discuss] DWARF expression evaluation
In-Reply-To: <CAEaoM9dJKHpS5Vj8idcOAEGCMbD3_n8RNUi1sisWFnuHwPaZxQ@mail.gmail.com>
References: <CAEaoM9dJKHpS5Vj8idcOAEGCMbD3_n8RNUi1sisWFnuHwPaZxQ@mail.gmail.com>
Message-ID: <20160714154619.GJ7387@tucnak.redhat.com>

On Thu, Jul 14, 2016 at 11:36:06AM -0400, Trevor Harrison wrote:
> Hi,
>   I'm trying to evaluate DWARF expressions and I'm looking for a hint
> or some documentation about the state of the expression evaluator's
> stack prior to beginning the expression.
> 
>   The reason I'm asking about this is I'm seeing expressions (a
> DW_AT_data_member_location for example) that start with DW_OP_dup,
> which implies that there is something on the stack for it to dup.

Of course there is, just read e.g. in DWARF4 p. 87 in the first paragraph:
"Otherwise, the value must be a location description. In this latter case,
the beginning address of the derived class is pushed on the expression stack
before the location description is evaluated and the result of the evaluation
is the location of the inherited type."
Similarly on p. 88.

	Jakub


From simon@codeplay.com  Thu Jul 21 08:36:06 2016
From: simon@codeplay.com (Simon Brand)
Date: Thu, 21 Jul 2016 09:36:06 +0100
Subject: [Dwarf-Discuss] DWARF and online-compiled programs (Simon Brand)
In-Reply-To: <14d26d48-30c2-9d08-9813-16454e294347@codeplay.com>
References: <8AA83D0D74B8AF4EB4D90F34FA7563D3586CBBBC@ORSMSX114.amr.corp.intel.com>
	<E3B07FDB86BFF041819DC057DEED8FEAD2D52A3C@USCULXMSG08.am.sony.com>
	<CANSbVThr4BFasKvCnbejVZR8fHgR3W-nGrmHFSToJZMdNehb+w@mail.gmail.com>
	<cedf009d-93c6-956e-0822-cd51ebf15213@codeplay.com>
	<9294d7f9-2251-80ec-e6a9-35ef27d27cb1@redhat.com>
	<14d26d48-30c2-9d08-9813-16454e294347@codeplay.com>
Message-ID: <804e01ec-2645-3e60-722e-5fbaf080e51c@codeplay.com>

Apologies for the delay, but I have updated the original proposal to 
take Pedro's suggestion into account.

Instead of using DW_AT_location and DW_LNCT_location, it uses 
DW_AT_source and DW_LNCT_source so the source is embedded in the DWARF 
and doesn't need to be loaded. These are described below.

=========================================================================
Changes to compile unit sections:
I added a DW_AT_source attribute, which is a string attribute 
identifying the source code. The intention is for implementations to use 
DW_FORM_strp so that the string is held in the .debug_str section and 
referenced from both the compile unit DIE and line table.

Section 3.1.1:
Replace bullet 2 with this:
A DW_AT_name or DW_AT_source attribute identifying the primary source 
from which the compilation unit was derived. If a DW_AT_name attribute 
is used, its value is a null-terminated string containing the full or 
relative path name of the source file. If a DW_AT_source attribute is 
used, its value is a null-terminated string containing the full contents 
of the source code from which the compilation unit was derived is 
stored. The source code string is UTF-8 encoded and encodes line endings 
with `\n`.

Figure 2:
Add DW_AT_source, which identifies "Embedded source code".

Figure 20:
Add DW_AT_source, whose class is "string"

Figure 42:
Add DW_AT_source to DW_TAG_compile_unit and DW_TAG_partial_unit entries.

=========================================================================


=========================================================================
Changes to line table sections:
I have based my modifications off of issue 140724.1. I don't know if 
this has since been modified, so there may be some inconsistencies.

These changes are a bit more complex, as there is currently the 
assumption that a given .debug_line section will only have a single 
file_name_entry_format. This would not support having a mix of usual 
source files and source-in-memory in the same program.

One solution would be to add the concept of a file name entry set, of 
which there can be more than one in a given header, and each can have 
its own file_name_entry_format. The header would contain a field 
specifying the number of file_name_entry_sets, then fields 17-21 would 
be repeated for each set. Another possibility would be to encode the 
sets in the same file_name_entry_format and file_names fields, but 
specify the sizes of each set. This is not quite as clear, but it seems 
desirable to avoid repeating the fields. I've sketched out the second 
option below.

------------------------------------------------------------------------
     Field  Field Name Value(s)
     Number
     1        Same as in Version 4                         ...
     2        version 5
     3        Not present in Version 4  -
     4        Not present in Version 4  -
     5-12   Same as in Version 4                         ...
     13      directory_entry_format_count           1
     14      directory_entry_format DW_LNCT_path, DW_FORM_string
     15      directories_count <n+1>
     16      directories <n+1>*<null terminated string>
     17      file_name_entry_set_count                2
     18      file_name_entry_format_set_counts  4,2
     19      file_name_entry_format DW_LNCT_path, DW_FORM_string,
        DW_LNCT_directory_index, DW_FORM_udata,
       DW_LNCT_timestamp, DW_FORM_udata,
       DW_LNCT_size, DW_FORM_udata,
       DW_LNCT_source, DW_FORM_strp,
       DW_LNCT_size, DW_FORM_udata

     20      file_name_set_count <m>, <n>
     21      file_names <m>*{<null terminated string>,
<index>, <timestamp>, <size>},
                                             <n>*{<source offset>, <size>}
------------------------------------------------------------------------

Section 6.2.4:
Add bullets after "16. directories"
17. file_name_entry_set_count (ubyte)
A count of the number of file name entry sets that occur in the 
following fields. If this field is zero, then the 
file_name_entry_format_set_sizes field (see below) must also be zero.

18. file_name_entry_format_set_counts (sequence of ubytes)
A sequence of counts of the number of entry formats for each file name 
entry set.

Section 6.2.4.1:
Add bullet after "5. DW_LNCT_MD5"
6. DW_LNCT_source
    The component is an offset into the .debug_str section where a 
null-terminated string contains the source code from which the 
compilation unit was derived. The string is UTF-8 encoded and encodes 
line endings using '\n'. Only one of DW_LNCT_path and DW_LNCT_source 
will be specified for a given file_name_entry_format. This content code 
is paired with the form DW_FORM_strp.

Append paragraph to bullet 1:
Only one of DW_LNCT_path and DW_LNCT_source will be specified for a 
given file_name_entry_format.

Add paragraph after the first paragraph of bullet 2:
The index is 0 if the source is identified by a memory location.

Table 7.25:
Add DW_LNCT_source 0x6 to the table

The description for DW_LNE_define_file will also need updating with 
similar text.

=========================================================================

Thanks,

Simon

On 15/06/16 15:01, Simon Brand wrote:
>
> This approach seems promising; it's probably easier to specify and 
> keeps all the necessary information within DWARF. We'd definitely need 
> to allow the line table and compile unit information to reference the 
> same string so that there's no duplication.
>
> Thanks,
>
> Simon
>
>
> On 13/06/16 11:22, Pedro Alves wrote:
>> On 06/13/2016 09:07 AM, Simon Brand wrote:
>>
>>> Also, the source might not be "in memory", as it might be placed in a
>>> binary segment which is not loaded. The debugger should be interpreting
>>> the location as something like the ELF virtual address to locate the
>>> source in the object file.
>> In that case, wouldn't it be better to instead represent the source
>> as another debug string, with DW_FORM_string, or
>> .debug_str/DW_FORM_strp, or .debug_str_offsets/DW_FORM_strx, etc. ?
>>
>> For example, add a new attribute, e.g., DW_AT_source, and in 3.1.1
>> where DW_AT_name is described:
>>
>>   Compilation unit entries may have the following attributes:
>>   ...
>>   2. A DW_AT_name attribute whose value is a null-terminated string containing
>>   the full or relative path name of the primary source file from which the
>>   compilation unit was derived.
>>
>> Change it to say either DW_AT_name or DW_AT_source.  Something like:
>>
>>   2. Either a DW_AT_name attribute whose value is a null-terminated string
>>   containing the full or relative path name of the primary source file from
>>   which the compilation unit was derived, or a DW_AT_source attribute whose
>>   value is a null-terminated string containing the full contents of the source
>>   code from which the compilation unit was derived.
>>
>> Thanks,
>> Pedro Alves
>>
>
> -- 
> Simon Brand
> Staff Software Engineer
> Codeplay Software Ltd
> Level C, Argyle House, 3 Lady Lawson St, Edinburgh, EH3 9DR
> Tel: 0131 466 0503
> Fax: 0131 557 6600
> Website:http://www.codeplay.com
> Twitter:https://twitter.com/codeplaysoft
>
> This email and any attachments may contain confidential and /or privileged information and is for use by the addressee only. If you are not the intended recipient, please notify Codeplay Software Ltd immediately and delete the message from your computer. You may not copy or forward it, or use or disclose its contents to any other person. Any views or other information in this message which do not relate to our business are not authorized by Codeplay software Ltd, nor does this message form part of any contract unless so stated.
> As internet communications are capable of data corruption Codeplay Software Ltd does not accept any responsibility for any changes made to this message after it was sent. Please note that Codeplay Software Ltd does not accept any liability or responsibility for viruses and it is your responsibility to scan any attachments.
> Company registered in England and Wales, number: 04567874
> Registered office: 81 Linkfield Street, Redhill RH1 6BY
>
>
> _______________________________________________
> Dwarf-Discuss mailing list
> Dwarf-Discuss at lists.dwarfstd.org
> http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org

-- 
Simon Brand
Staff Software Engineer
Codeplay Software Ltd
Level C, Argyle House, 3 Lady Lawson St, Edinburgh, EH3 9DR
Tel: 0131 466 0503
Fax: 0131 557 6600
Website: http://www.codeplay.com
Twitter: https://twitter.com/codeplaysoft

This email and any attachments may contain confidential and /or privileged information and is for use by the addressee only. If you are not the intended recipient, please notify Codeplay Software Ltd immediately and delete the message from your computer. You may not copy or forward it, or use or disclose its contents to any other person. Any views or other information in this message which do not relate to our business are not authorized by Codeplay software Ltd, nor does this message form part of any contract unless so stated.
As internet communications are capable of data corruption Codeplay Software Ltd does not accept any responsibility for any changes made to this message after it was sent. Please note that Codeplay Software Ltd does not accept any liability or responsibility for viruses and it is your responsibility to scan any attachments.
Company registered in England and Wales, number: 04567874
Registered office: 81 Linkfield Street, Redhill RH1 6BY

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/private.cgi/dwarf-discuss-dwarfstd.org/attachments/20160721/518accc3/attachment.htm>


From Tony.Tye@amd.com  Wed Jul 27 23:39:54 2016
From: Tony.Tye@amd.com (Tye, Tony)
Date: Wed, 27 Jul 2016 23:39:54 +0000
Subject: [Dwarf-Discuss] How to represent address space information in DWARF
Message-ID: <CY1PR1201MB103321B132C9640FD1C7DA7C970F0@CY1PR1201MB1033.namprd12.prod.outlook.com>

Another question that has been raised as part of the HSA Foundation (http://www.hsafoundation.com/) tools working group relates to the manner that address spaces should be represented in DWARF.

HSA defines segments in which variables can be allocated. These are basically the same as the address spaces of OpenCL. HSA defines kernels that are basically the same as OpenCL kernels. A kernel is a grid launch of separate threads of execution (termed work-items). These work-items are grouped into work-groups. The work-items can access one of three main memory segments:

1. The global segment is accessible by all work-items. In hardware it is typically just the global memory.
2. The group segment (corresponding to the local address space of OpenCL) is accessible only to the work-items in the same work-group. Each work-group has its own copy of variables allocated in the group segment. On GPU hardware this can be implemented as special hardware managed scratch pad memory (not part of globally addressable memory), with special hardware instructions to access it.
3. The private segment is accessible to a single work-item. Each work-item has its own copy of variables allocated in the private segment. On GPU hardware this could also involve special hardware instructions.

HSA also defines the concept of a flat address (similar to OpenCL generic addresses). It is essentially a linearization of the addresses of the 3 address spaces. For example, one range of a flat address maps to the group segment, another range maps to the private segment, and the rest map directly to the global segment. However, it is target specific what exact method is used to achieve the linearization.

The following was the conclusion we reached from reading the DWARF standard and looking at how gdb and lldb would use the information. We are currently working on creating a patch for LLVM to support address spaces and would appreciate any feedback on if this matches the intended usage of DWARF features to support this style of address space.

1. Use the DW_AT_address_class to specify that the value of a pointer-like value is the address within a specific address space. Pointer-like values include pointers, references, functions and function types. For HSA we are really only concerned with pointer/reference values currently. It would apply to a pointer-like type DIE, or a variable with a pointer-like type. In the case of a variable it does not specify the address space of the variable's location, but specifies how to treat the address value stored in the variable.

2. Use DW_OP_xderef in the location expression of a variable to specify the address space in which the variable is located. Since location expressions can specify different locations depending on the PC, this allows the variable to be optimized to have multiple locations. For example, sometimes in a memory location in the group address space, sometimes in a register, sometimes in a memory location in the private address space (maybe due to spilling of the register), etc.

Attempting to use DW_AT_address_class on variables to specify their address space location conflicts with DWARF stating that it applies to the pointee as described in #1. It also breaks the flexibility of location expressions allowing the location to change according to PC.

When a debugger evaluates a DWARF location expression it can generate a flat address to encode the address space. It can do this by implementing the XDEREF as a target specific conversion from a segment address into a flat address. Similarly when using a value as an address that has a pointer-like type with an address class, the value can be converted to a flat address. When accessing addresses the debugger would have to provide the "current thread" so that the correct group/private address space instance can be accessed when given a flat address that maps to the group or private segments. It appears both gdb and lldb provide this.

Thanks,
Tony Tye
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/private.cgi/dwarf-discuss-dwarfstd.org/attachments/20160727/234afcc4/attachment.htm>


From todd.allen@ccur.com  Thu Jul 28 18:46:03 2016
From: todd.allen@ccur.com (Todd Allen)
Date: Thu, 28 Jul 2016 12:46:03 -0600
Subject: [Dwarf-Discuss] How to represent address space information in
 DWARF
In-Reply-To: <CY1PR1201MB103321B132C9640FD1C7DA7C970F0@CY1PR1201MB1033.namprd12.prod.outlook.com>
References: <CY1PR1201MB103321B132C9640FD1C7DA7C970F0@CY1PR1201MB1033.namprd12.prod.outlook.com>
Message-ID: <20160728184603.GA13701@toad.ccur.com>

On Wed, Jul 27, 2016 at 07:39:54PM -0400, Tye, Tony wrote:
>    Another question that has been raised as part of the HSA Foundation
>    ([1]http://www.hsafoundation.com/) tools working group relates to the
>    manner that address spaces should be represented in DWARF.
> 
> 
> 
>    HSA defines segments in which variables can be allocated. These are
>    basically the same as the address spaces of OpenCL. HSA defines kernels
>    that are basically the same as OpenCL kernels. A kernel is a grid launch
>    of separate threads of execution (termed work-items). These work-items are
>    grouped into work-groups. The work-items can access one of three main
>    memory segments:
> 
> 
> 
>    1. The global segment is accessible by all work-items. In hardware it is
>    typically just the global memory.
> 
>    2. The group segment (corresponding to the local address space of OpenCL)
>    is accessible only to the work-items in the same work-group. Each
>    work-group has its own copy of variables allocated in the group segment.
>    On GPU hardware this can be implemented as special hardware managed
>    scratch pad memory (not part of globally addressable memory), with special
>    hardware instructions to access it.
> 
>    3. The private segment is accessible to a single work-item. Each work-item
>    has its own copy of variables allocated in the private segment. On GPU
>    hardware this could also involve special hardware instructions.
> 
> 
> 
>    HSA also defines the concept of a flat address (similar to OpenCL generic
>    addresses). It is essentially a linearization of the addresses of the 3
>    address spaces. For example, one range of a flat address maps to the group
>    segment, another range maps to the private segment, and the rest map
>    directly to the global segment. However, it is target specific what exact
>    method is used to achieve the linearization.
> 
> 
> 
>    The following was the conclusion we reached from reading the DWARF
>    standard and looking at how gdb and lldb would use the information. We are
>    currently working on creating a patch for LLVM to support address spaces
>    and would appreciate any feedback on if this matches the intended usage of
>    DWARF features to support this style of address space.
> 
> 
> 
>    1. Use the DW_AT_address_class to specify that the value of a pointer-like
>    value is the address within a specific address space. Pointer-like values
>    include pointers, references, functions and function types. For HSA we are
>    really only concerned with pointer/reference values currently. It would
>    apply to a pointer-like type DIE, or a variable with a pointer-like type.
>    In the case of a variable it does not specify the address space of the
>    variable's location, but specifies how to treat the address value stored
>    in the variable.
> 
> 
> 
>    2. Use DW_OP_xderef in the location expression of a variable to specify
>    the address space in which the variable is located. Since location
>    expressions can specify different locations depending on the PC, this
>    allows the variable to be optimized to have multiple locations. For
>    example, sometimes in a memory location in the group address space,
>    sometimes in a register, sometimes in a memory location in the private
>    address space (maybe due to spilling of the register), etc.
> 
> 
> 
>    Attempting to use DW_AT_address_class on variables to specify their
>    address space location conflicts with DWARF stating that it applies to the
>    pointee as described in #1. It also breaks the flexibility of location
>    expressions allowing the location to change according to PC.
> 
> 
> 
>    When a debugger evaluates a DWARF location expression it can generate a
>    flat address to encode the address space. It can do this by implementing
>    the XDEREF as a target specific conversion from a segment address into a
>    flat address. Similarly when using a value as an address that has a
>    pointer-like type with an address class, the value can be converted to a
>    flat address. When accessing addresses the debugger would have to provide
>    the "current thread" so that the correct group/private address space
>    instance can be accessed when given a flat address that maps to the group
>    or private segments. It appears both gdb and lldb provide this.
> 

FWIW, the use of DW_AT_address_class on pointer & reference times also is how
Nvidia's CUDA compiler describes its various segments.  I haven't encountered
any uses of DW_OP_xderef* operators, but it probably is just because it hasn't
been necessary.

-- 
Todd Allen
Concurrent Computer Corporation


