From jan.kratochvil@redhat.com  Tue Jun 13 21:22:44 2017
From: jan.kratochvil@redhat.com (Jan Kratochvil)
Date: Tue, 13 Jun 2017 23:22:44 +0200
Subject: [Dwarf-Discuss] .debug_names: Preallocation of DW_IDX_* for
	static/extern symbols
Message-ID: <20170613212244.GA30078@host1.jankratochvil.net>

Hello,

filed:
	http://dwarfstd.org/ShowIssue.php?issue=170527.1

When a debugger wants to print 'somename' it logically tries to find first
'somename' as an external symbol in all available libraries.  Only if none
such external symbol is found the debugger starts searching for a static
'somename' symbol in those libraries.

This requires to know whether a symbol in .debug_names index has
DW_AT_external or not.  Otherwise a lot of needless CU expansions happen.
This extension improves performance gain of the .debug_names index.

Discovered in an original fix by Doug Evans - GDB Bug 14125:
	https://sourceware.org/bugzilla/show_bug.cgi?id=14125

Proposing and asking for pre-allocation:
  DW_IDX_static   = 6 = DW_FORM_flag_present = DIE's DW_AT_external is not present
  DW_IDX_external = 7 = DW_FORM_flag_present = DIE's DW_AT_external is present

Is it OK to start using these numbers to be in DWARF-6 or should stay in the
DW_IDX_lo_user..DW_IDX_hi_user range?


Thanks,
Jan Kratochvil


From paul.robinson@sony.com  Tue Jun 13 22:13:40 2017
From: paul.robinson@sony.com (Robinson, Paul)
Date: Tue, 13 Jun 2017 22:13:40 +0000
Subject: [Dwarf-Discuss] .debug_names: Preallocation of DW_IDX_*
	for	static/extern symbols
In-Reply-To: <20170613212244.GA30078@host1.jankratochvil.net>
References: <20170613212244.GA30078@host1.jankratochvil.net>
Message-ID: <E3B07FDB86BFF041819DC057DEED8FEA0133370B20@USCULXMSG08.am.sony.com>



> -----Original Message-----
> From: Dwarf-Discuss [mailto:dwarf-discuss-bounces at lists.dwarfstd.org] On
> Behalf Of Jan Kratochvil
> Sent: Tuesday, June 13, 2017 2:23 PM
> To: dwarf-discuss at lists.dwarfstd.org
> Subject: [Dwarf-Discuss] .debug_names: Preallocation of DW_IDX_* for
> static/extern symbols
> 
> Hello,
> 
> filed:
> 	http://dwarfstd.org/ShowIssue.php?issue=170527.1
> 
> When a debugger wants to print 'somename' it logically tries to find first
> 'somename' as an external symbol in all available libraries.  Only if none
> such external symbol is found the debugger starts searching for a static
> 'somename' symbol in those libraries.
> 
> This requires to know whether a symbol in .debug_names index has
> DW_AT_external or not.  Otherwise a lot of needless CU expansions happen.
> This extension improves performance gain of the .debug_names index.
> 
> Discovered in an original fix by Doug Evans - GDB Bug 14125:
> 	https://sourceware.org/bugzilla/show_bug.cgi?id=14125
> 
> Proposing and asking for pre-allocation:
>   DW_IDX_static   = 6 = DW_FORM_flag_present = DIE's DW_AT_external is not
> present
>   DW_IDX_external = 7 = DW_FORM_flag_present = DIE's DW_AT_external is
> present
> 
> Is it OK to start using these numbers to be in DWARF-6 or should stay in
> the
> DW_IDX_lo_user..DW_IDX_hi_user range?

There's no guarantee that your proposal would be accepted unmodified.
You should use the lo_user..hi_user range for anything you do now.
--paulr

> 
> 
> Thanks,
> Jan Kratochvil
> _______________________________________________
> Dwarf-Discuss mailing list
> Dwarf-Discuss at lists.dwarfstd.org
> http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org


From rupeshpotharla@gmail.com  Wed Jun 21 17:13:59 2017
From: rupeshpotharla@gmail.com (rupesh potharla)
Date: Wed, 21 Jun 2017 22:43:59 +0530
Subject: [Dwarf-Discuss] New Dwarf tags
Message-ID: <CAPLvmz6aLw3-rWGtU6WKzXxjM52LrmEjx=OdvyWQUtsfk3Gaxg@mail.gmail.com>

Hi,

I am looking for a document which has all the newly added dwarf tags for
Dwarf3 and Dwarf4 at one place.
Could someone help me  with this?

-- 
  Thanks & Regards
  Rupesh P
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/attachments/20170621/282565c0/attachment.htm>


From aprantl@apple.com  Wed Jun 21 17:18:41 2017
From: aprantl@apple.com (Adrian Prantl)
Date: Wed, 21 Jun 2017 10:18:41 -0700
Subject: [Dwarf-Discuss] New Dwarf tags
In-Reply-To: <CAPLvmz6aLw3-rWGtU6WKzXxjM52LrmEjx=OdvyWQUtsfk3Gaxg@mail.gmail.com>
References: <CAPLvmz6aLw3-rWGtU6WKzXxjM52LrmEjx=OdvyWQUtsfk3Gaxg@mail.gmail.com>
Message-ID: <917DA27B-0E1D-4E6E-A988-8592AB6E8F35@apple.com>

The LLVM source code has most tags annotated with the version number they were introduced in:

https://github.com/llvm-mirror/llvm/blob/master/include/llvm/BinaryFormat/Dwarf.def <https://github.com/llvm-mirror/llvm/blob/master/include/llvm/BinaryFormat/Dwarf.def>

-- adrian

> On Jun 21, 2017, at 10:13 AM, rupesh potharla <rupeshpotharla at gmail.com> wrote:
> 
> Hi,
> 
> I am looking for a document which has all the newly added dwarf tags for Dwarf3 and Dwarf4 at one place. 
> Could someone help me  with this?
> 
> -- 
>   Thanks & Regards
>   Rupesh P
>    
>    
> _______________________________________________
> Dwarf-Discuss mailing list
> Dwarf-Discuss at lists.dwarfstd.org
> http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/attachments/20170621/1089fc62/attachment.htm>


From arnez@linux.vnet.ibm.com  Wed Jun 21 17:46:33 2017
From: arnez@linux.vnet.ibm.com (Andreas Arnez)
Date: Wed, 21 Jun 2017 19:46:33 +0200
Subject: [Dwarf-Discuss] New Dwarf tags
In-Reply-To: <CAPLvmz6aLw3-rWGtU6WKzXxjM52LrmEjx=OdvyWQUtsfk3Gaxg@mail.gmail.com>
	(rupesh potharla's message of "Wed, 21 Jun 2017 22:43:59 +0530")
References: <CAPLvmz6aLw3-rWGtU6WKzXxjM52LrmEjx=OdvyWQUtsfk3Gaxg@mail.gmail.com>
Message-ID: <m3injpjksm.fsf@oc1027705133.ibm.com>

On Wed, Jun 21 2017, rupesh potharla wrote:

> Hi,
>
> I am looking for a document which has all the newly added dwarf tags for Dwarf3 and Dwarf4 at one place. 
> Could someone help me with this?

GNU Binutils defines DWARF tags in a simple format, arranged by DWARF
version/extension:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=include/dwarf2.def;hb=HEAD

--
Andreas



From davea42@linuxmail.org  Wed Jun 21 17:47:15 2017
From: davea42@linuxmail.org (David Anderson)
Date: Wed, 21 Jun 2017 10:47:15 -0700
Subject: [Dwarf-Discuss] New Dwarf tags
In-Reply-To: <CAPLvmz6aLw3-rWGtU6WKzXxjM52LrmEjx=OdvyWQUtsfk3Gaxg@mail.gmail.com>
References: <CAPLvmz6aLw3-rWGtU6WKzXxjM52LrmEjx=OdvyWQUtsfk3Gaxg@mail.gmail.com>
Message-ID: <496adbac-183b-b0ff-6408-72ec12c808db@linuxmail.org>

On 06/21/2017 10:13 AM, rupesh potharla wrote:
> I am looking for a document which has all the newly added dwarf tags for
> Dwarf3 and Dwarf4 at one place. 
> Could someone help me  with this?

Last email: wrong url.

https://www.prevanders.net/dwarf.h

(not dwarf.html).
DavidA
-- 
We already have an industrial policy on energy--it's just that it's an
industrial policy designed to subsidize and encourage the use of fossil
fuels.  -- James Surowiecki, The New Yorker, Oct 10 2011.


From eager@eagercon.com  Wed Jun 21 19:58:46 2017
From: eager@eagercon.com (Michael Eager)
Date: Wed, 21 Jun 2017 12:58:46 -0700
Subject: [Dwarf-Discuss] New Dwarf tags
In-Reply-To: <CAPLvmz6aLw3-rWGtU6WKzXxjM52LrmEjx=OdvyWQUtsfk3Gaxg@mail.gmail.com>
References: <CAPLvmz6aLw3-rWGtU6WKzXxjM52LrmEjx=OdvyWQUtsfk3Gaxg@mail.gmail.com>
Message-ID: <594ACFF6.3000600@eagercon.com>

On 06/21/2017 10:13 AM, rupesh potharla wrote:
> Hi,
>
> I am looking for a document which has all the newly added dwarf tags for Dwarf3 and Dwarf4 at one
> place.
> Could someone help me  with this?

Why are you looking for "newly added" tags for DWARF v3 and DWARF v4?
The latest version is DWARF v5.


-- 
Michael Eager	 eager at eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077


From dblaikie@gmail.com  Tue Jun 27 18:46:45 2017
From: dblaikie@gmail.com (David Blaikie)
Date: Tue, 27 Jun 2017 18:46:45 +0000
Subject: [Dwarf-Discuss] Fission + cross-CU references (ref_addr)
In-Reply-To: <CAENS6EvCObQKPDR=8ZBA3fa4wu5kNyRNj5taO18YTu+-J8wWdg@mail.gmail.com>
References: <CAENS6EuSZ+8__YTXdPgbTfKgs819sVPW972nRFcMF=pB_XLuSw@mail.gmail.com>
	<CAENS6EvCObQKPDR=8ZBA3fa4wu5kNyRNj5taO18YTu+-J8wWdg@mail.gmail.com>
Message-ID: <CAENS6EsL5S6Ec1amQQ=R3C8aHZm-Vg9cnpPPpdZJGe3etZo+3g@mail.gmail.com>

ping

On Wed, May 31, 2017 at 1:36 PM David Blaikie <dblaikie at gmail.com> wrote:

> Pinging this thread & bringing it to the attention of Cary & Doug in case
> they've got some thoughts.
>
> I'd love to have some kind of ideas about this to work on an
> implementation in GDB (as an extension/prototype).
>
> (don't mean to dismiss Paul's suggestion - and I generally agree with it,
> except where it gets a bit difficult when type units are folded into
> debug_info (Paul's suggestion requiring some kind of grouping within a
> section ("contributions") which seems like a new constraint/detail I
> wouldn't think to have here))
>
> On another thread Cary talked about the idea of avoiding ref_addr and
> expanding the functionality of DW_FORM_sig8/signature and perhaps
> generalizing the type unit header to allow multiple hash+offset pairs. This
> is a feature Adrian & I had talked about previously (when looking at the
> overhead of type units - the need to duplicate member function declarations
> in any unit that has a definition of the member function, since the
> specific DIE in the type unit couldn't be referred to from the definition
> in the CU) & I think would be great (& would further generalize the unit
> header - no need for a difference between TU and CU header, instead have a
> "count" & then that many {hash, offset} pairs for referring to any DIEs
> from elsewhere). Might not be the perfect solution to the CU->TU
> references, since signatures are large-ish. So might require a bit more
> care/data/etc.
>
> I'm also not sure/don't think I'd have the time to push that ^ feature, as
> much as it is a nice one. Paul's cu_index column extension wouldn't be too
> hard to implement, I think.
>
>
> On Tue, May 2, 2017 at 12:09 PM David Blaikie <dblaikie at gmail.com> wrote:
>
>> I've recently been trying to resolve the use of Fission in LLVM's ThinLTO
>> mode (though this would apply to plain LTO too).
>>
>> One of the things that happens here is that cross-CU DIE references
>> (DW_FORM_ref_addr) are used to describe inlining a function in one CU into
>> another CU.
>>
>> This format has been implemented in LLVM and GCC for ~years and seems to
>> work well outside of Fission.
>>
>> So the question is: what to do with Fission?
>>
>> It seemed to me that a good representation would be to produce multiple
>> CUs into a single DWO file, which GDB can't yet consume, but I'm working on
>> patches to help there. DW_FORM_ref_addr would not use any ELF relocation,
>> but be assumed to be "relative to the chunk of debug_info it was in"
>> (within the .dwo file)
>>
>> But what about DWP files? Currently binutils dwp produces records like
>> this:
>>
>> (this dwp contains 3 CUs, two from one LTO compile, and one from a
>> standalone compile linked in for comparison):
>>
>> Index Signature          INFO     ABBR     LINE     STR_OFF
>> ----- ------------------ -------- -------- -------- --------
>>     2 0x7bd765349b7e7631 [2d, 65) [38, ae) [11, 22) [14, 3c)
>>     8 0x66f4e160661d2687 [00, 2d) [00, 38) [00, 11) [00, 14)
>>    11 0x32dd6d7121dd1d9a [65, 98) [38, ae) [11, 22) [14, 3c)
>>
>> So the ABBR/LINE/STR_OFF sections are kept as-is (no analysis is done to
>> find which portions of the dwo file are used by which CUs, etc), but the
>> INFO section is fragmented on the CU boundaries. Fragmenting the TYPES
>> section on the TU boundaries is necessary/useful for deduplication of
>> types, but this fragmenting of the CU makes it impossible (I think) to use
>> ref_addr in a dwp file.
>>
>> If this fragmenting were not done - consumers (GDB, etc) would need to
>> change to account for this - searching through the INFO range to find the
>> CU matching the signature, rather than knowing it starts at the start of
>> the INFO range. This could have a noticeable performance impact especially
>> in a full LTO build (where /all/ the CUs were in the same .dwo - so the
>> index would be entirely unhelpful, I think).
>>
>> Does all this sound right/sane - anyone have ideas/perspectives/thoughts
>> on how this should work?
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/attachments/20170627/2573c0b4/attachment.htm>


From dblaikie@gmail.com  Tue Jun 27 18:47:06 2017
From: dblaikie@gmail.com (David Blaikie)
Date: Tue, 27 Jun 2017 18:47:06 +0000
Subject: [Dwarf-Discuss] Including the DWO name in the CU hash
In-Reply-To: <CAENS6EuyxBh4csERgvsDReo3P1HQCRoe9iWv72mrATXSkmnF5Q@mail.gmail.com>
References: <CAENS6EvGfeOr32N5Yzkkj7pYS+LM-yytO5ROEWdsvOLOC2-4gw@mail.gmail.com>
	<CAENS6EuyxBh4csERgvsDReo3P1HQCRoe9iWv72mrATXSkmnF5Q@mail.gmail.com>
Message-ID: <CAENS6Etn58h6VKL-rUW2hVg5NdX+J652f28jU27PLNzYf8t66w@mail.gmail.com>

ping, in case anyone's got some ideas here

On Wed, May 31, 2017 at 1:28 PM David Blaikie <dblaikie at gmail.com> wrote:

> Ping?
>
> I did end up hacking around this by hashing in the DWO name into the CU
> hash if LLVM's producing more than one CU. It's not perfect (really it's
> more about the ThinLTO importing stage - normal LTO doesn't need this sort
> of mangling) but suffices for now:
> http://llvm.org/viewvc/llvm-project?rev=304119&view=rev
>
> But it seems to suffice for now.
>
>
> On Fri, May 19, 2017 at 3:51 PM David Blaikie <dblaikie at gmail.com> wrote:
>
>> some context:
>>
>> 1) A little while ago, I added the dwo_name to the dwo CU to improve
>> diagnostic quality on CU hash collisions in during a dwp action (previously
>> it could only report which input files (possibly other DWP files) contained
>> the duplicates/collision which could be very manual to track back to the
>> original input DWO files - having the original DWO names in the diagnostic
>> made it relatively easy to track down).
>>
>> 2) LLVM's new ThinLTO presents a high chance of duplicate DWO CUs - it
>> does this by creating effectively "new" CUs containing a stripped down
>> version of an existing CU - containing only a handful of functions that may
>> be relevant to optimizing some other CU. (imagine two CUs both using a
>> single inline function from a 3rd CU - the 3rd CU's inline function and the
>> basic CU itself is imported into the compilation steps of the other two CUs
>> - so in the end you get two DWO files, each with two CUs, where one CU
>> contains only an abstract definition of the inline function).
>>
>> My initial thinking here was that I could cross-pollinate the CU hash
>> from each CU within a single compilation, since the primary CU would have
>> enough uniqueness (hash all the CUs, then cross-hash them).
>>
>> But then I realized the CUs should already be unique because they include
>> the dwo_name which will be different between the two stripped down CU
>> clones. But the dwo_name isn't included in the hash - so I prototyped
>> including it & it does what you'd expect.
>>
>> Extra wrinkle: Once the dwo_name is in the hash, then it defeats my
>> original motivation for including it in the DWO CU in the first place: such
>> CUs will never collide, so the name would never be useful for diagnostic
>> quality.
>>
>> Should I drop the dwo_name from the DWO CU and manually/explicitly
>> include it in the hash? Does cross pollination sound better? Should I only
>> do either of these when dealing with more than one CU in a DWO? (in which
>> case the diagnostic improvement would still be valid - it catches some
>> interesting cases, but they're not /very/ interesting like major bugs (&
>> does DWO ID collisions have some false positives too, which hashing the
>> dwo_id would fix), etc... and the mechanism wasn't built for bug catching
>> in any case)
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/attachments/20170627/30b2ab6f/attachment.htm>


From dblaikie@gmail.com  Tue Jun 27 18:48:03 2017
From: dblaikie@gmail.com (David Blaikie)
Date: Tue, 27 Jun 2017 18:48:03 +0000
Subject: [Dwarf-Discuss] address pool + offset representation
In-Reply-To: <CAENS6Ev7ZPLSvtGZpcP81q=jhW18yR97A1LFq36aehQaiUuC5A@mail.gmail.com>
References: <CAENS6EvgnkgVgZ11JeagDtXE1i6s14nEnT_jNvZpOiQov1=ERA@mail.gmail.com>
	<CAENS6Ev7ZPLSvtGZpcP81q=jhW18yR97A1LFq36aehQaiUuC5A@mail.gmail.com>
Message-ID: <CAENS6EtKFJ+bZBRi-+hkW4H9v-01Un4bRTPnc657SQRfKgPfVA@mail.gmail.com>

Ping

This'd be a great file size saving, especially in optimized builds - and
probably link time optimization by having /many/ fewer relocations.

On Wed, May 31, 2017 at 1:25 PM David Blaikie <dblaikie at gmail.com> wrote:

> Ping - any ideas?
>
>
> On Wed, May 17, 2017 at 4:17 PM David Blaikie <dblaikie at gmail.com> wrote:
>
>> A big part of Fission debug info in object files an optimized build, are
>> unique address relocation in debug_addr and debug_ranges. I have an example
>> binary where for N bytes of .text, there are ~2N bytes of .rela.debug_addr
>> and >2N bytes of .rela.debug_ranges.
>>
>> Given that .rela.debug_line is about ~1% of the size of .text - that
>> gives a sense of the lower bound - there should be only a few more
>> relocations needed in .debug_addr than in .debug_line (relocs for global
>> variables).
>>
>> This arises because things like low_pc (for each subprogram and then each
>> lexical block inside it) use distinct addresses in the address pool, when
>> they could use an offset relative to some other known address in the pool
>> (basically one address for each .text section - and everything would use
>> that + offset).
>>
>> The new *x (startx*, base_addressx) forms in the debug_rnglists format
>> address the relocations for the debug_ranges section - allowing it to reuse
>> addresses in debug_addr.
>>
>> But to address debug_addr's redundancy, it'd would be nice to have an
>> abbreviation form to represent "address in the pool, plus a constant
>> offset".
>>
>> What form should this take?
>>
>> Currently there's addrx{,1-4} - a LEB128, or 1-4 byte fixed-length
>> representation.
>> Similarly the offset between addresses could be a variety of lengths.
>> high_pc for example allows for any form of the constant class.
>>
>> Should this support the combination of all of these forms
>> (addrx{,1-4}*data{,1-4}). Or is there some better option?
>>
>> I hope to try prototyping this as a GNU extension form if/when people
>> have some idea of what might be best for the representation.
>>
>>
>> Bonus points: It'd be great to move the address pool into the line table
>> - then it'd be really one relocation per section. That'd need a new
>> assembly directive, though. Anyone got ideas on that too? (this would be
>> orthogonal to the above improvement - both things are good to do)
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/attachments/20170627/2e21877f/attachment-0001.htm>


