DWARF Standard


HOME
SPECIFICATIONS
FAQ
ISSUES



181223.1 Jordan Frost Add Microsoft SourceLink support Enhancement Open Tony Tye


Section , pg 

Microsoft's PDB format supports a feature, SourceLink. It allows embedding a 
JSON blob inside the PDB file that their debuggers can use to find source.

This feature is sorely missing from code compiled via GCC that has debug 
information.

The format does not have not be SourceLink at all, but just any kind of blob 
sitting in the debug file.


DISCUSSION

It is unclear if the request is seeking a standard way to put arbitrary blobs
into DWARF information, or a standard way to provide functionality similar 
to SourceLink.

If the former, existing code object formats already provide such ability. For
example, ELF already provides the ability to add arbitrary sections or note
records. It would then require agreement on the name, format, and meaning of
such sections or note records so that producers and consumers could communicate.

If the latter, then a first step is to look at the SourceLink definition 
which can be found at:

  https://github.com/dotnet/designs/blob/main/accepted/2020/diagnostics/source-link.md

It is intended to provide a means to specify a URL that can be used to locate
the source files from an online server. It uses a JSON textual representation. 
The representation supports limited wildcards to allow a single entry to
describe the location of many files.

DWARF has a file table that provides information about the source files. So one
approach is to extend it to allow the specification of a URL. This would provide
a standard place that produces and consumers could use to communicate the URL
that could be used to find the source online. A consumer could use this if the
source is not locally available using the file path. The file table already has
a means to add fields to each file entry, so an additional one could be defined.

A compiler/linker producer could accept a SourceLink JSON file and use it while
generating the DWARF file table. Standard linking would be able to preserve 
the information as it is just part of the existing DWARF file table.

The proposal below describes this alternative. It uses URL for the name of the
field, but should it use URI to be more general?

ALTERNATIVES

The above approach does not benefit from the compaction offered by the
SourceLink JSON use of wildcards. However, is this considered an issue
in DWARF size?

If it is then a more elaborate DWARF approach could be devised. For example,
a per CU “url table” could be added with entries for a file root and a URL
root (essentially what is in the JSON file). File entries could either
reference entries in the url table, or the url table could simply be an
independent table that a consumer could consult to determine the URL of a
file using the same rules defined by SourceLink.

Another approach to address this issue that does not involve any DWARF changes 
is the use of debuginfo files
(see https://fedoraproject.org/wiki/Releases/FeatureBuildId). An executable that 
contains DWARF debug information can be split into a stripped ELF executable that 
contains no debug information and a debuginfo file that is an ELF file that only 
contains the DWARF debug information. The stripped executable has an ELF note record
added that has a Build-ID 
(see https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/developer_guide/compiling-build-id)
which is a hash of the stable parts of the stripped executable that is also
saved in the debuginfo file. The stripped executable is included in the distribute
package. In addition a debug package can be made that includes the debuginfo file
together with the sources. 

The debugedit (http://manpages.ubuntu.com/manpages/bionic/man8/debugedit.8.html) 
tool can be used to split an executable, compute and insert the Build-ID, and
change the prefix of all the source file paths in the DWARF to use a fixed
location as opposed to the one used by the build system.

To debug the executable the debuginfo file and sources can then be installed using
the debug packages (see dwarf-workgroup@lists.dwarfstd.org). The debuginfod server
(https://developers.redhat.com/blog/2019/10/14/introducing-debuginfod-the-elfutils-debuginfo-server)
can be used to retrieve the debuginfo files in an automated fashion using a URL.
A debugger can use the Build-ID in the executable to request the debuginfo and 
sources from the debuginfod server and cache them locally
(see https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html).

PROPOSED RESOLUTION

This augments DWARF Version 5 section 6.2.4.1.

6.  DW_LNCT_URL
 
    The component is a null-terminated UTF-8 URL text string. This content code
    is paired with the same forms as DW_LNCT_path. It can be used for file
    name entries.  If DW_LNCT_URL content kind is not present, or is present 
    with an empty null-terminated string value, no URL is available.  Otherwise,
    the null-terminated string value provides a URL for a location that can be
    used to access the source text.

    When the URL is available, consumers can use it to obtain the source
    text if the file path provided by the DW_LNCT_path field is
    not accessible, or if the timestamp, size or hash of the contents of that
    file does not match that provided by the DW_LNCT_timestamp, 
    DW_LNCT_size or DW_LNCT_MD5 fields respectively.

This augments DWARF Version 5 section 7.22 and Table 7.27.

The following table gives the encoding of the additional line number header
entry format.

  Table 7.27: Line number header entry format encodings

  ====================================  ====================
  Line number header entry format name  Value
  ====================================  ====================
  DW_LNCT_is_URL #                      0x6
  ====================================  ====================

--
2021-08-06:  Revised: Added discussion, alternatives, proposal.


All logos and trademarks in this site are property of their respective owner.
The comments are property of their posters, all the rest © 2007-2021 by DWARF Standards Committee.