From eager@eagercon.com  Fri Jul 21 23:58:48 2017
From: eager@eagercon.com (Michael Eager)
Date: Fri, 21 Jul 2017 16:58:48 -0700
Subject: [Dwarf-Discuss] Large posts to DWARF mailing list
Message-ID: <82e3a4cd-b606-6f58-29f6-c1169e64e07c@eagercon.com>

Hi Folks --

There have been several recent posts to the mailing list which are over 100K bytes.

Please submit posts in plain text rather than HTML and be judicious about the amount of backquoting.


-- 
Michael Eager	 eager at eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077


From dblaikie@gmail.com  Wed Jul 26 18:03:43 2017
From: dblaikie@gmail.com (David Blaikie)
Date: Wed, 26 Jul 2017 18:03:43 +0000
Subject: [Dwarf-Discuss] address pool + offset representation
In-Reply-To: <CADPb22QMVA7Hq3_1Ao8jbD=Xz58xgm43x0y2UUaKfjdgsCDvag@mail.gmail.com>
References: <CAENS6EvgnkgVgZ11JeagDtXE1i6s14nEnT_jNvZpOiQov1=ERA@mail.gmail.com>
	<CAENS6Ev7ZPLSvtGZpcP81q=jhW18yR97A1LFq36aehQaiUuC5A@mail.gmail.com>
	<CAENS6EtKFJ+bZBRi-+hkW4H9v-01Un4bRTPnc657SQRfKgPfVA@mail.gmail.com>
	<CADPb22STNsqPd+HU0GV6v+h6e4MkQ48k6iAn=xSE5JPVzNsOVg@mail.gmail.com>
	<CAENS6EtSJVAGPdYijZ6d0uE4koZ5R6bbBmUd6EL190QKbmk4bA@mail.gmail.com>
	<CADPb22QMVA7Hq3_1Ao8jbD=Xz58xgm43x0y2UUaKfjdgsCDvag@mail.gmail.com>
Message-ID: <CAENS6EuPud9YP5mx_S7VCb-euv1XHvqZ-nd+t2covQ-rXfpQqQ@mail.gmail.com>

On Wed, Jul 26, 2017 at 10:51 AM Doug Evans <dje at google.com> wrote:

> How about a new form that specifies two things: index in .debug_addr +
> offset, where index and offset are any current constant representation
> (fixed size or leb) ?
>

Roughly what I had in mind - but wasn't sure how to encode that.

Would the type of the representation for each component be encoded at the
use? So this would be a variable length form (even when both the
index+offset use fixed length forms - because the form would be at the use,
so you'd have to read the bytes to figure out that they were using fixed
length forms)?

A two-value version of DW_FORM_indirect, I suppose?

I know the LLDB folks seem to find that fixed-width forms especially help
with parsing - not sure if GDB has a similar tendency/preference?


>
>
>
> On Tue, Jul 25, 2017 at 9:01 PM, David Blaikie <dblaikie at gmail.com> wrote:
>
>>
>>
>> On Tue, Jun 27, 2017 at 5:04 PM Doug Evans <dje at google.com> wrote:
>>
>>> Heya. Again I'm not sure what help I can be here so I wasn't planning on
>>> entering the thread.
>>>
>>
>> Certainly open to ideas from you or anyone else - mostly with both these
>> threads I'm more interested in something you'd think was OK for GDB, since
>> I'd like to implement these as extensions in GDB now, rather than waiting
>> for a whole DWARF release cycle (good to have implementation experience
>> anyway)
>>
>> For opt builds this would make a huge difference, since they have many
>> relocations (many ranges, etc).
>>
>> But not a huge priority or anything.
>>
>>
>>>
>>> [just making sure you're not waiting on me]
>>>
>>> On Tue, Jun 27, 2017 at 11:48 AM, David Blaikie <dblaikie at gmail.com>
>>> wrote:
>>>
>>>> Ping
>>>>
>>>> This'd be a great file size saving, especially in optimized builds -
>>>> and probably link time optimization by having /many/ fewer relocations.
>>>>
>>>> On Wed, May 31, 2017 at 1:25 PM David Blaikie <dblaikie at gmail.com>
>>>> wrote:
>>>>
>>>>> Ping - any ideas?
>>>>>
>>>>>
>>>>> On Wed, May 17, 2017 at 4:17 PM David Blaikie <dblaikie at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> A big part of Fission debug info in object files an optimized build,
>>>>>> are unique address relocation in debug_addr and debug_ranges. I have an
>>>>>> example binary where for N bytes of .text, there are ~2N bytes of
>>>>>> .rela.debug_addr and >2N bytes of .rela.debug_ranges.
>>>>>>
>>>>>> Given that .rela.debug_line is about ~1% of the size of .text - that
>>>>>> gives a sense of the lower bound - there should be only a few more
>>>>>> relocations needed in .debug_addr than in .debug_line (relocs for global
>>>>>> variables).
>>>>>>
>>>>>> This arises because things like low_pc (for each subprogram and then
>>>>>> each lexical block inside it) use distinct addresses in the address pool,
>>>>>> when they could use an offset relative to some other known address in the
>>>>>> pool (basically one address for each .text section - and everything would
>>>>>> use that + offset).
>>>>>>
>>>>>> The new *x (startx*, base_addressx) forms in the debug_rnglists
>>>>>> format address the relocations for the debug_ranges section - allowing it
>>>>>> to reuse addresses in debug_addr.
>>>>>>
>>>>>> But to address debug_addr's redundancy, it'd would be nice to have an
>>>>>> abbreviation form to represent "address in the pool, plus a constant
>>>>>> offset".
>>>>>>
>>>>>> What form should this take?
>>>>>>
>>>>>> Currently there's addrx{,1-4} - a LEB128, or 1-4 byte fixed-length
>>>>>> representation.
>>>>>> Similarly the offset between addresses could be a variety of lengths.
>>>>>> high_pc for example allows for any form of the constant class.
>>>>>>
>>>>>> Should this support the combination of all of these forms
>>>>>> (addrx{,1-4}*data{,1-4}). Or is there some better option?
>>>>>>
>>>>>> I hope to try prototyping this as a GNU extension form if/when people
>>>>>> have some idea of what might be best for the representation.
>>>>>>
>>>>>>
>>>>>> Bonus points: It'd be great to move the address pool into the line
>>>>>> table - then it'd be really one relocation per section. That'd need a new
>>>>>> assembly directive, though. Anyone got ideas on that too? (this would be
>>>>>> orthogonal to the above improvement - both things are good to do)
>>>>>>
>>>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/attachments/20170726/01f42799/attachment.htm>


From paul.robinson@sony.com  Wed Jul 26 19:00:17 2017
From: paul.robinson@sony.com (Robinson, Paul)
Date: Wed, 26 Jul 2017 19:00:17 +0000
Subject: [Dwarf-Discuss] address pool + offset representation
In-Reply-To: <CAENS6EuPud9YP5mx_S7VCb-euv1XHvqZ-nd+t2covQ-rXfpQqQ@mail.gmail.com>
References: <CAENS6EvgnkgVgZ11JeagDtXE1i6s14nEnT_jNvZpOiQov1=ERA@mail.gmail.com>
	<CAENS6Ev7ZPLSvtGZpcP81q=jhW18yR97A1LFq36aehQaiUuC5A@mail.gmail.com>
	<CAENS6EtKFJ+bZBRi-+hkW4H9v-01Un4bRTPnc657SQRfKgPfVA@mail.gmail.com>
	<CADPb22STNsqPd+HU0GV6v+h6e4MkQ48k6iAn=xSE5JPVzNsOVg@mail.gmail.com>
	<CAENS6EtSJVAGPdYijZ6d0uE4koZ5R6bbBmUd6EL190QKbmk4bA@mail.gmail.com>
	<CADPb22QMVA7Hq3_1Ao8jbD=Xz58xgm43x0y2UUaKfjdgsCDvag@mail.gmail.com>
	<CAENS6EuPud9YP5mx_S7VCb-euv1XHvqZ-nd+t2covQ-rXfpQqQ@mail.gmail.com>
Message-ID: <E3B07FDB86BFF041819DC057DEED8FEA01333E9658@USCULXMSG08.am.sony.com>

> From: Dwarf-Discuss [mailto:dwarf-discuss-bounces at lists.dwarfstd.org] On Behalf Of David Blaikie
> Sent: Wednesday, July 26, 2017 11:04 AM
> To: Doug Evans; dwarf-discuss at lists.dwarfstd.org
> Subject: Re: [Dwarf-Discuss] address pool + offset representation
>
>
> On Wed, Jul 26, 2017 at 10:51 AM Doug Evans <dje at google.com> wrote:
>> How about a new form that specifies two things: index in .debug_addr +
>> offset, where index and offset are any current constant representation
>> (fixed size or leb) ?
>
> Roughly what I had in mind - but wasn't sure how to encode that.
>
> Would the type of the representation for each component be encoded at
> the use? So this would be a variable length form (even when both the
> index+offset use fixed length forms - because the form would be at the
> use, so you'd have to read the bytes to figure out that they were using
> fixed length forms)?
>
> A two-value version of DW_FORM_indirect, I suppose?

This discussion is about reducing the number of .debug_addr entries you
need, so the place to look would be the "class address" attributes.
There aren't all that many of them, and the one most likely to benefit
would be DW_AT_low_pc.

We made it okay to have DW_AT_high_pc be a constant, which is implicitly
the sum of that constant with the same DIE's DW_AT_low_pc value.
How about if DW_AT_low_pc could also be a constant, which would be added
to the containing block/subprogram's DW_AT_low_pc?

>
> I know the LLDB folks seem to find that fixed-width forms especially
> help with parsing - not sure if GDB has a similar tendency/preference?

The producer can pick a fixed-size constant form to accommodate this.
It would have to be a big-enough form, as the assembler would be
computing the delta, but that's a QOI problem for the producer.
--paulr



From dblaikie@gmail.com  Wed Jul 26 19:17:32 2017
From: dblaikie@gmail.com (David Blaikie)
Date: Wed, 26 Jul 2017 19:17:32 +0000
Subject: [Dwarf-Discuss] address pool + offset representation
In-Reply-To: <E3B07FDB86BFF041819DC057DEED8FEA01333E9658@USCULXMSG08.am.sony.com>
References: <CAENS6EvgnkgVgZ11JeagDtXE1i6s14nEnT_jNvZpOiQov1=ERA@mail.gmail.com>
	<CAENS6Ev7ZPLSvtGZpcP81q=jhW18yR97A1LFq36aehQaiUuC5A@mail.gmail.com>
	<CAENS6EtKFJ+bZBRi-+hkW4H9v-01Un4bRTPnc657SQRfKgPfVA@mail.gmail.com>
	<CADPb22STNsqPd+HU0GV6v+h6e4MkQ48k6iAn=xSE5JPVzNsOVg@mail.gmail.com>
	<CAENS6EtSJVAGPdYijZ6d0uE4koZ5R6bbBmUd6EL190QKbmk4bA@mail.gmail.com>
	<CADPb22QMVA7Hq3_1Ao8jbD=Xz58xgm43x0y2UUaKfjdgsCDvag@mail.gmail.com>
	<CAENS6EuPud9YP5mx_S7VCb-euv1XHvqZ-nd+t2covQ-rXfpQqQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9658@USCULXMSG08.am.sony.com>
Message-ID: <CAENS6EuTV-X7i0hrnXth83+X=+8_sqXb0bP3EO1E2Ug2u9Y-nQ@mail.gmail.com>

On Wed, Jul 26, 2017 at 12:00 PM Robinson, Paul <paul.robinson at sony.com>
wrote:

> > From: Dwarf-Discuss [mailto:dwarf-discuss-bounces at lists.dwarfstd.org]
> On Behalf Of David Blaikie
> > Sent: Wednesday, July 26, 2017 11:04 AM
> > To: Doug Evans; dwarf-discuss at lists.dwarfstd.org
> > Subject: Re: [Dwarf-Discuss] address pool + offset representation
> >
> >
> > On Wed, Jul 26, 2017 at 10:51 AM Doug Evans <dje at google.com> wrote:
> >> How about a new form that specifies two things: index in .debug_addr +
> >> offset, where index and offset are any current constant representation
> >> (fixed size or leb) ?
> >
> > Roughly what I had in mind - but wasn't sure how to encode that.
> >
> > Would the type of the representation for each component be encoded at
> > the use? So this would be a variable length form (even when both the
> > index+offset use fixed length forms - because the form would be at the
> > use, so you'd have to read the bytes to figure out that they were using
> > fixed length forms)?
> >
> > A two-value version of DW_FORM_indirect, I suppose?
>
> This discussion is about reducing the number of .debug_addr entries you
> need, so the place to look would be the "class address" attributes.
> There aren't all that many of them, and the one most likely to benefit
> would be DW_AT_low_pc.
>
> We made it okay to have DW_AT_high_pc be a constant, which is implicitly
> the sum of that constant with the same DIE's DW_AT_low_pc value.
> How about if DW_AT_low_pc could also be a constant, which would be added
> to the containing block/subprogram's DW_AT_low_pc?
>

It'd catch a fair few, maybe even most, of the cases - though still
potentially many more relocations than needed: eg: two (or more) functions
in the same section would end up using separate relocations (assuming they
were in a CU with at least one function in some other section - such as an
inline function in a comdat).

Certainly having a low_pc works like high_pc, but uses the nearest
enclosing low_pc would be a handy encoding size win as well, though, but I
was partly motivated by looking at what the actual minimum number of
relocations would be and looking at possible features/implementations that
would get all the way there.


>
> >
> > I know the LLDB folks seem to find that fixed-width forms especially
> > help with parsing - not sure if GDB has a similar tendency/preference?
>
> The producer can pick a fixed-size constant form to accommodate this.
>

Not sure I quite follow - if the form is "pair of implicit values" then
that's the only form here. Inside the debug_info data, then there could be
constant forms for the two elements of the pair - but it's still a
dynamically sized form from the perspective of a parser.

Am I picturing this wrong?


> It would have to be a big-enough form, as the assembler would be
> computing the delta, but that's a QOI problem for the producer.
> --paulr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/attachments/20170726/cced7637/attachment.htm>


From paul.robinson@sony.com  Wed Jul 26 19:56:21 2017
From: paul.robinson@sony.com (Robinson, Paul)
Date: Wed, 26 Jul 2017 19:56:21 +0000
Subject: [Dwarf-Discuss] address pool + offset representation
In-Reply-To: <CAENS6EuTV-X7i0hrnXth83+X=+8_sqXb0bP3EO1E2Ug2u9Y-nQ@mail.gmail.com>
References: <CAENS6EvgnkgVgZ11JeagDtXE1i6s14nEnT_jNvZpOiQov1=ERA@mail.gmail.com>
	<CAENS6Ev7ZPLSvtGZpcP81q=jhW18yR97A1LFq36aehQaiUuC5A@mail.gmail.com>
	<CAENS6EtKFJ+bZBRi-+hkW4H9v-01Un4bRTPnc657SQRfKgPfVA@mail.gmail.com>
	<CADPb22STNsqPd+HU0GV6v+h6e4MkQ48k6iAn=xSE5JPVzNsOVg@mail.gmail.com>
	<CAENS6EtSJVAGPdYijZ6d0uE4koZ5R6bbBmUd6EL190QKbmk4bA@mail.gmail.com>
	<CADPb22QMVA7Hq3_1Ao8jbD=Xz58xgm43x0y2UUaKfjdgsCDvag@mail.gmail.com>
	<CAENS6EuPud9YP5mx_S7VCb-euv1XHvqZ-nd+t2covQ-rXfpQqQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9658@USCULXMSG08.am.sony.com>
	<CAENS6EuTV-X7i0hrnXth83+X=+8_sqXb0bP3EO1E2Ug2u9Y-nQ@mail.gmail.com>
Message-ID: <E3B07FDB86BFF041819DC057DEED8FEA01333E9724@USCULXMSG08.am.sony.com>

>> This discussion is about reducing the number of .debug_addr entries you
>> need, so the place to look would be the "class address" attributes.
>> There aren't all that many of them, and the one most likely to benefit
>> would be DW_AT_low_pc.
>>
>> We made it okay to have DW_AT_high_pc be a constant, which is implicitly
>> the sum of that constant with the same DIE's DW_AT_low_pc value.
>> How about if DW_AT_low_pc could also be a constant, which would be added
>> to the containing block/subprogram's DW_AT_low_pc?
>
> It'd catch a fair few, maybe even most, of the cases - though still
> potentially many more relocations than needed: eg: two (or more)
> functions in the same section would end up using separate relocations
> (assuming they were in a CU with at least one function in some other
> section - such as an inline function in a comdat).

Oh, because the unit wouldn't have a low_pc, it would have ranges?

Well... why not just use ranges, in that case?  .debug_rnglists is
already tuned to reduce relocations.  A range list with only one
entry is the same as a contiguous range, and DW_RLE_offset_pair is
basically the same as using constant low_pc and high_pc.
--paulr



From dblaikie@gmail.com  Wed Jul 26 21:31:53 2017
From: dblaikie@gmail.com (David Blaikie)
Date: Wed, 26 Jul 2017 21:31:53 +0000
Subject: [Dwarf-Discuss] address pool + offset representation
In-Reply-To: <E3B07FDB86BFF041819DC057DEED8FEA01333E9724@USCULXMSG08.am.sony.com>
References: <CAENS6EvgnkgVgZ11JeagDtXE1i6s14nEnT_jNvZpOiQov1=ERA@mail.gmail.com>
	<CAENS6Ev7ZPLSvtGZpcP81q=jhW18yR97A1LFq36aehQaiUuC5A@mail.gmail.com>
	<CAENS6EtKFJ+bZBRi-+hkW4H9v-01Un4bRTPnc657SQRfKgPfVA@mail.gmail.com>
	<CADPb22STNsqPd+HU0GV6v+h6e4MkQ48k6iAn=xSE5JPVzNsOVg@mail.gmail.com>
	<CAENS6EtSJVAGPdYijZ6d0uE4koZ5R6bbBmUd6EL190QKbmk4bA@mail.gmail.com>
	<CADPb22QMVA7Hq3_1Ao8jbD=Xz58xgm43x0y2UUaKfjdgsCDvag@mail.gmail.com>
	<CAENS6EuPud9YP5mx_S7VCb-euv1XHvqZ-nd+t2covQ-rXfpQqQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9658@USCULXMSG08.am.sony.com>
	<CAENS6EuTV-X7i0hrnXth83+X=+8_sqXb0bP3EO1E2Ug2u9Y-nQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9724@USCULXMSG08.am.sony.com>
Message-ID: <CAENS6Et-LZfoa06og5uyMVH=t_7UWu8iRh3UhAQKObcr3Y1m-Q@mail.gmail.com>

On Wed, Jul 26, 2017 at 12:56 PM Robinson, Paul <paul.robinson at sony.com>
wrote:

> >> This discussion is about reducing the number of .debug_addr entries you
> >> need, so the place to look would be the "class address" attributes.
> >> There aren't all that many of them, and the one most likely to benefit
> >> would be DW_AT_low_pc.
> >>
> >> We made it okay to have DW_AT_high_pc be a constant, which is implicitly
> >> the sum of that constant with the same DIE's DW_AT_low_pc value.
> >> How about if DW_AT_low_pc could also be a constant, which would be added
> >> to the containing block/subprogram's DW_AT_low_pc?
> >
> > It'd catch a fair few, maybe even most, of the cases - though still
> > potentially many more relocations than needed: eg: two (or more)
> > functions in the same section would end up using separate relocations
> > (assuming they were in a CU with at least one function in some other
> > section - such as an inline function in a comdat).
>
> Oh, because the unit wouldn't have a low_pc, it would have ranges?
>

Right


> Well... why not just use ranges, in that case?  .debug_rnglists is
> already tuned to reduce relocations.  A range list with only one
> entry is the same as a contiguous range, and DW_RLE_offset_pair is
> basically the same as using constant low_pc and high_pc.
>

Fair point/question (I was going to say I wasn't sure if gdb supported
rnglists yet, but grepping the source it looks like it probably does -
though LLVM can't/doesn't produce them yet, it would be a good thing, even
aside from this particular issue).

Certainly would be a thing to prototype/a valid representation that would
reduce the number of address relocations/size of object files (under split
DWARF at least).

But I imagine that representation might be a bit larger overall/more
expensive to parse?

- Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/attachments/20170726/d2b96988/attachment.htm>


From paul.robinson@sony.com  Wed Jul 26 23:27:19 2017
From: paul.robinson@sony.com (Robinson, Paul)
Date: Wed, 26 Jul 2017 23:27:19 +0000
Subject: [Dwarf-Discuss] address pool + offset representation
In-Reply-To: <CAENS6Et-LZfoa06og5uyMVH=t_7UWu8iRh3UhAQKObcr3Y1m-Q@mail.gmail.com>
References: <CAENS6EvgnkgVgZ11JeagDtXE1i6s14nEnT_jNvZpOiQov1=ERA@mail.gmail.com>
	<CAENS6Ev7ZPLSvtGZpcP81q=jhW18yR97A1LFq36aehQaiUuC5A@mail.gmail.com>
	<CAENS6EtKFJ+bZBRi-+hkW4H9v-01Un4bRTPnc657SQRfKgPfVA@mail.gmail.com>
	<CADPb22STNsqPd+HU0GV6v+h6e4MkQ48k6iAn=xSE5JPVzNsOVg@mail.gmail.com>
	<CAENS6EtSJVAGPdYijZ6d0uE4koZ5R6bbBmUd6EL190QKbmk4bA@mail.gmail.com>
	<CADPb22QMVA7Hq3_1Ao8jbD=Xz58xgm43x0y2UUaKfjdgsCDvag@mail.gmail.com>
	<CAENS6EuPud9YP5mx_S7VCb-euv1XHvqZ-nd+t2covQ-rXfpQqQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9658@USCULXMSG08.am.sony.com>
	<CAENS6EuTV-X7i0hrnXth83+X=+8_sqXb0bP3EO1E2Ug2u9Y-nQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9724@USCULXMSG08.am.sony.com>
	<CAENS6Et-LZfoa06og5uyMVH=t_7UWu8iRh3UhAQKObcr3Y1m-Q@mail.gmail.com>
Message-ID: <E3B07FDB86BFF041819DC057DEED8FEA01333E9C21@USCULXMSG08.am.sony.com>

>> Well... why not just use ranges, in that case?? .debug_rnglists is
>> already tuned to reduce relocations.? A range list with only one
>> entry is the same as a contiguous range, and DW_RLE_offset_pair is
>> basically the same as using constant low_pc and high_pc.
>												
> Fair point/question (I was going to say I wasn't sure if gdb supported
> rnglists yet, but grepping the source it looks like it probably does -
> though LLVM can't/doesn't produce them yet, it would be a good thing,
> even aside from this particular issue).

The new range lists are on my list of DWARF 5 features to do in LLVM.

> Certainly would be a thing to prototype/a valid representation that
> would reduce the number of address relocations/size of object files 
> (under split DWARF at least).
>
> But I imagine that representation might be a bit larger overall/more
> expensive to parse?

Well, if you want to minimize relocations, you can have:
- DW_AT_ranges with a LEB index into the rnglists section;
- one 4-byte entry in the rnglists header to point to the actual
  range list; (or 8-byte entry in DWARF-64 format);
- the range list itself which would be:
  o DW_RLE_offset_pair (1 byte entry code plus 2 LEBs)
  o DW_RLE_end_of_list (1 byte entry code)
  So 2 bytes plus 2 LEBs. This assumes using the unit's base address.
  If we're not using the unit's base address, we add 1 byte and 1 LEB
  to select the base address in the .debug_addr section.
Grand total is 6 bytes plus 3 LEBs (or 7 bytes and 4 LEBs if we can't
use the unit's base address).

Versus DW_AT_low_pc, one address-sized value and a relocation, or
one index into .debug_addr and the same relocation.

Or DW_AT_low_pc-as-constant, which is likely the smallest but does
have the drawback that to compute this DIE's low_pc you need to 
find the parent's low_pc (or the nearest parent that has a low_pc).

I *really* don't think a two-operand kind of FORM will ever fly.

So the range-list approach is a bit bigger, but one .debug_addr entry
can act as the base for everything in the section so essentially
relocation costs are eliminated (both size in .o files and linker
execution time).
It might also be cost-effective to pre-parse the rnglists section,
could easily be done in parallel with reading other info sections.
The low_pc solutions really don't feel like they could be done in
parallel with anything else.

--paulr



From dblaikie@gmail.com  Wed Jul 26 23:40:39 2017
From: dblaikie@gmail.com (David Blaikie)
Date: Wed, 26 Jul 2017 23:40:39 +0000
Subject: [Dwarf-Discuss] address pool + offset representation
In-Reply-To: <E3B07FDB86BFF041819DC057DEED8FEA01333E9C21@USCULXMSG08.am.sony.com>
References: <CAENS6EvgnkgVgZ11JeagDtXE1i6s14nEnT_jNvZpOiQov1=ERA@mail.gmail.com>
	<CAENS6Ev7ZPLSvtGZpcP81q=jhW18yR97A1LFq36aehQaiUuC5A@mail.gmail.com>
	<CAENS6EtKFJ+bZBRi-+hkW4H9v-01Un4bRTPnc657SQRfKgPfVA@mail.gmail.com>
	<CADPb22STNsqPd+HU0GV6v+h6e4MkQ48k6iAn=xSE5JPVzNsOVg@mail.gmail.com>
	<CAENS6EtSJVAGPdYijZ6d0uE4koZ5R6bbBmUd6EL190QKbmk4bA@mail.gmail.com>
	<CADPb22QMVA7Hq3_1Ao8jbD=Xz58xgm43x0y2UUaKfjdgsCDvag@mail.gmail.com>
	<CAENS6EuPud9YP5mx_S7VCb-euv1XHvqZ-nd+t2covQ-rXfpQqQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9658@USCULXMSG08.am.sony.com>
	<CAENS6EuTV-X7i0hrnXth83+X=+8_sqXb0bP3EO1E2Ug2u9Y-nQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9724@USCULXMSG08.am.sony.com>
	<CAENS6Et-LZfoa06og5uyMVH=t_7UWu8iRh3UhAQKObcr3Y1m-Q@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9C21@USCULXMSG08.am.sony.com>
Message-ID: <CAENS6Es7GhE6q-3WJC5PRPoCiJ72KRjst6R1h8MwGKzyO1uUbg@mail.gmail.com>

On Wed, Jul 26, 2017 at 4:27 PM Robinson, Paul <paul.robinson at sony.com>
wrote:

> >> Well... why not just use ranges, in that case?  .debug_rnglists is
> >> already tuned to reduce relocations.  A range list with only one
> >> entry is the same as a contiguous range, and DW_RLE_offset_pair is
> >> basically the same as using constant low_pc and high_pc.
> >
> > Fair point/question (I was going to say I wasn't sure if gdb supported
> > rnglists yet, but grepping the source it looks like it probably does -
> > though LLVM can't/doesn't produce them yet, it would be a good thing,
> > even aside from this particular issue).
>
> The new range lists are on my list of DWARF 5 features to do in LLVM.
>
> > Certainly would be a thing to prototype/a valid representation that
> > would reduce the number of address relocations/size of object files
> > (under split DWARF at least).
> >
> > But I imagine that representation might be a bit larger overall/more
> > expensive to parse?
>
> Well, if you want to minimize relocations, you can have:
> - DW_AT_ranges with a LEB index into the rnglists section;
>

That's necessarily a ULEB (reading the spec, there's no equivalent of
addrx1/2/3/4?)?

I don't really know just how much LLDB cares about fixed-size forms/DIEs,
but rumor has it it's important to some degree, so I continue to have a
slight preference towards fixed size representations (or at least having
the option to do so, even if there are variable length forms too - as with
addrx).


> - one 4-byte entry in the rnglists header to point to the actual
>   range list; (or 8-byte entry in DWARF-64 format);
> - the range list itself which would be:
>   o DW_RLE_offset_pair (1 byte entry code plus 2 LEBs)
>   o DW_RLE_end_of_list (1 byte entry code)
>   So 2 bytes plus 2 LEBs. This assumes using the unit's base address.
>   If we're not using the unit's base address, we add 1 byte and 1 LEB
>   to select the base address in the .debug_addr section.
> Grand total is 6 bytes plus 3 LEBs (or 7 bytes and 4 LEBs if we can't
> use the unit's base address).
>

Seems like a lot - if it's not possible to do a fixed size representation,
at that point I wouldn't mind a form that was two ULEBs (addrx + offset).
(at least I think the offset, computed as a label difference in the
assembler, can be done as a ULEB (I mean I think there's assembler syntax
for label difference as ULEB)).


> Versus DW_AT_low_pc, one address-sized value and a relocation, or
> one index into .debug_addr and the same relocation.
>
> Or DW_AT_low_pc-as-constant, which is likely the smallest but does
> have the drawback that to compute this DIE's low_pc you need to
> find the parent's low_pc (or the nearest parent that has a low_pc).
>

*nod* Wouldn't be /too/ hard to keep track of.


> I *really* don't think a two-operand kind of FORM will ever fly.
>

Oh? Why's that? I realize starting out that it wasn't ideal - but mostly
around the idea of having the fixed-size forms, and thus having
combinatorial explosion. If fixed sized forms aren't a priority/aren't
possible in the other representations anyway (& aren't done for rnglistx,
for example) - a two ULEB doesn't seem totally crazy to me.


> So the range-list approach is a bit bigger, but one .debug_addr entry
> can act as the base for everything in the section so essentially
> relocation costs are eliminated (both size in .o files and linker
> execution time).
>

Right - that's the goal (& eventually fold the addr pool into the line
table - since it'll contain all the same addresses/relocations over again -
be nice to have them literally /once/)


> It might also be cost-effective to pre-parse the rnglists section,
> could easily be done in parallel with reading other info sections.
> The low_pc solutions really don't feel like they could be done in
> parallel with anything else.
>

Maybe - either you can't do it in parallel because there are non-fixed-size
forms, in which case you have to walk through them anyway (& so you pay a
little extra time to track the current low_pc as you do the walk) or you
have fixed size forms and use them to skip over DIEs - in which case in
parallel you could have other threads reading thoes skipped over DIEs to
find/compute low_pcs (& a merge step to resolve which one is the base of
which other one, etc). Doubt anyone would go to quite such lengths, though.

- Dave


>
> --paulr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/attachments/20170726/8bc50519/attachment.htm>


From paul.robinson@sony.com  Thu Jul 27 18:54:24 2017
From: paul.robinson@sony.com (Robinson, Paul)
Date: Thu, 27 Jul 2017 18:54:24 +0000
Subject: [Dwarf-Discuss] address pool + offset representation
In-Reply-To: <CAENS6Es7GhE6q-3WJC5PRPoCiJ72KRjst6R1h8MwGKzyO1uUbg@mail.gmail.com>
References: <CAENS6EvgnkgVgZ11JeagDtXE1i6s14nEnT_jNvZpOiQov1=ERA@mail.gmail.com>
	<CAENS6Ev7ZPLSvtGZpcP81q=jhW18yR97A1LFq36aehQaiUuC5A@mail.gmail.com>
	<CAENS6EtKFJ+bZBRi-+hkW4H9v-01Un4bRTPnc657SQRfKgPfVA@mail.gmail.com>
	<CADPb22STNsqPd+HU0GV6v+h6e4MkQ48k6iAn=xSE5JPVzNsOVg@mail.gmail.com>
	<CAENS6EtSJVAGPdYijZ6d0uE4koZ5R6bbBmUd6EL190QKbmk4bA@mail.gmail.com>
	<CADPb22QMVA7Hq3_1Ao8jbD=Xz58xgm43x0y2UUaKfjdgsCDvag@mail.gmail.com>
	<CAENS6EuPud9YP5mx_S7VCb-euv1XHvqZ-nd+t2covQ-rXfpQqQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9658@USCULXMSG08.am.sony.com>
	<CAENS6EuTV-X7i0hrnXth83+X=+8_sqXb0bP3EO1E2Ug2u9Y-nQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9724@USCULXMSG08.am.sony.com>
	<CAENS6Et-LZfoa06og5uyMVH=t_7UWu8iRh3UhAQKObcr3Y1m-Q@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9C21@USCULXMSG08.am.sony.com>
	<CAENS6Es7GhE6q-3WJC5PRPoCiJ72KRjst6R1h8MwGKzyO1uUbg@mail.gmail.com>
Message-ID: <E3B07FDB86BFF041819DC057DEED8FEA01333EB4A1@USCULXMSG08.am.sony.com>

												|
> I don't really know just how much LLDB cares about fixed-size forms/DIEs,
> but rumor has it it's important to some degree, so I continue to have a
> slight preference towards fixed size representations (or at least having
> the option to do so, even if there are variable length forms too - as
> with addrx).

Greg Clayton has told me it's a performance win for loading .debug_info.
They can index fixed-size DIEs without actually parsing them.  When I
proposed the fixed-size strx/addrx forms to the committee, I did some
data collection on the effect of strx and addrx.  Converting to strx 
meant fixed size DIEs went from 90-ish to 55-ish percent of all DIEs,
so fixed-size strxN forms were definitely worthwhile.  The equivalent
analysis for addrx showed a difference of more like 4%, which was
enough to persuade the committee that would be worthwhile also.  If 
we go with the low_pc-as-range idea then adding fixed-size versions of 
rnglistx in DWARF 6 would not be a problem.

The advantage of what the ranges idea is it can all be done using
exiting DWARF 5 features.

> at that point I wouldn't mind a form that was two ULEBs (addrx + offset).

Might as well just allow DW_AT_low_pc to use a location expression?
  DW_OP_addrx <n> DW_OP_constx <offset> DW_OP_plus
This would be way more acceptable than a new special-purpose form.
Still bigger than a new form, of course, but as I noted above only 4%
of DIEs become variable size so that penalty doesn't seem excessive.
3 bytes plus 3 LEBs, so only 3 bytes + 1 LEB more than the new form,
and still cheaper than the ranges idea.
--paulr



From dblaikie@gmail.com  Thu Jul 27 19:58:12 2017
From: dblaikie@gmail.com (David Blaikie)
Date: Thu, 27 Jul 2017 19:58:12 +0000
Subject: [Dwarf-Discuss] address pool + offset representation
In-Reply-To: <E3B07FDB86BFF041819DC057DEED8FEA01333EB4A1@USCULXMSG08.am.sony.com>
References: <CAENS6EvgnkgVgZ11JeagDtXE1i6s14nEnT_jNvZpOiQov1=ERA@mail.gmail.com>
	<CAENS6Ev7ZPLSvtGZpcP81q=jhW18yR97A1LFq36aehQaiUuC5A@mail.gmail.com>
	<CAENS6EtKFJ+bZBRi-+hkW4H9v-01Un4bRTPnc657SQRfKgPfVA@mail.gmail.com>
	<CADPb22STNsqPd+HU0GV6v+h6e4MkQ48k6iAn=xSE5JPVzNsOVg@mail.gmail.com>
	<CAENS6EtSJVAGPdYijZ6d0uE4koZ5R6bbBmUd6EL190QKbmk4bA@mail.gmail.com>
	<CADPb22QMVA7Hq3_1Ao8jbD=Xz58xgm43x0y2UUaKfjdgsCDvag@mail.gmail.com>
	<CAENS6EuPud9YP5mx_S7VCb-euv1XHvqZ-nd+t2covQ-rXfpQqQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9658@USCULXMSG08.am.sony.com>
	<CAENS6EuTV-X7i0hrnXth83+X=+8_sqXb0bP3EO1E2Ug2u9Y-nQ@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9724@USCULXMSG08.am.sony.com>
	<CAENS6Et-LZfoa06og5uyMVH=t_7UWu8iRh3UhAQKObcr3Y1m-Q@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333E9C21@USCULXMSG08.am.sony.com>
	<CAENS6Es7GhE6q-3WJC5PRPoCiJ72KRjst6R1h8MwGKzyO1uUbg@mail.gmail.com>
	<E3B07FDB86BFF041819DC057DEED8FEA01333EB4A1@USCULXMSG08.am.sony.com>
Message-ID: <CAENS6Es3KJ3=5dn_auqaS-+fsNoMmUW28RaKgyEs7uRfc6t4=w@mail.gmail.com>

On Thu, Jul 27, 2017 at 11:54 AM Robinson, Paul <paul.robinson at sony.com>
wrote:

>
>                       |
> > I don't really know just how much LLDB cares about fixed-size forms/DIEs,
> > but rumor has it it's important to some degree, so I continue to have a
> > slight preference towards fixed size representations (or at least having
> > the option to do so, even if there are variable length forms too - as
> > with addrx).
>
> Greg Clayton has told me it's a performance win for loading .debug_info.
> They can index fixed-size DIEs without actually parsing them.  When I
> proposed the fixed-size strx/addrx forms to the committee, I did some
> data collection on the effect of strx and addrx.  Converting to strx
> meant fixed size DIEs went from 90-ish to 55-ish percent of all DIEs,
> so fixed-size strxN forms were definitely worthwhile.  The equivalent
> analysis for addrx showed a difference of more like 4%, which was
> enough to persuade the committee that would be worthwhile also.


I imagine the same would be true of ranges in an optimized build? (Maybe a
little lower, but not by much) 'spose, as you say, we can always add
rnglistxN in the future.


>   If
> we go with the low_pc-as-range idea then adding fixed-size versions of
> rnglistx in DWARF 6 would not be a problem.
>
> The advantage of what the ranges idea is it can all be done using
> exiting DWARF 5 features.
>

Yeah, for sure. Might well be the place for me to start - and certainly
something I could ship today (probably under a flag).

I was thinking that this optimization was an obvious enough win that there
might be interest in having a dedicated form for it - but OK proving it out
with another method like this first & then standardizing a
convenience/compact representation.


>
> > at that point I wouldn't mind a form that was two ULEBs (addrx + offset).
>
> Might as well just allow DW_AT_low_pc to use a location expression?
>   DW_OP_addrx <n> DW_OP_constx <offset> DW_OP_plus
> This would be way more acceptable than a new special-purpose form.
> Still bigger than a new form, of course, but as I noted above only 4%
> of DIEs become variable size so that penalty doesn't seem excessive.
> 3 bytes plus 3 LEBs, so only 3 bytes + 1 LEB more than the new form,
> and still cheaper than the ranges idea.
>

Oh, heh - funnily enough, this is /exactly/ how I prototyped it:

+  addUInt(Die, dwarf::DW_FORM_data1, dwarf::DW_OP_GNU_addr_index);
+  addUInt(Die, dwarf::DW_FORM_GNU_addr_index, IndexEntry.second);
+  if (IndexEntry.first != Label) {
+    addLabelDelta(Die, (dwarf::Attribute)0, Label, IndexEntry.first);
+    addUInt(Die, dwarf::DW_FORM_data1, dwarf::DW_OP_plus);
+  }

(Can certainly provide the full patch if you're curious)

I'm open to that - though it seemed like, while it'd be a small spec change
& obxviously low cost to producers (if they don't use the feature, costs
them nothing, and encoding this particular expression isn't
complicated/hard) - but might be really expensive for consumers to support
the full diversity of expressions in a new context? I could be wrong,
though.

- Dave

--paulr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/attachments/20170727/f4c68c44/attachment-0001.htm>


