Issue 231110.4: DWARF Extension Expiration

Author:	Ben Woodard
Champion:	Mark Wielaard
Date submitted:	2023-11-10
Date revised:	2024-03-25
Date closed:	2024-06-10
Type:	Enhancement
Status:	Rejected
DWARF Version:	6

Background

This is an Enhancement issue split off from Issue 231110.1 to expire DWARF extensions when a new DWARF standard version is released.

The DWARF standard has always had the wisdom to acknowledge the need for Vendor Extensibility. Section 1.3.13 describes this policy. For the producers it explicitly reserves some of the valid values for encoding various constructs and promises not to use those values in future versions of the standard. Consumers are given the freedom and expected to skip over data that they do not recognize.

The original intent of these vendor extensions was that they would be a private agreement between a particular producer and a cooperating consumer. However, the range of tools expanded beyond a toolchain provided by a single vendor and so the concept of a private agreement between a specific producer and a cooperating consumer came to be regarded as a registry of vendor extensions that every tool must be aware of for the sake of compatibility.

Because compatibility between tools was prioritized, the extensions in these de facto registries were effectively considered permanent unofficial DWARF encodings. This presumptive permanence is in spite of the fact that in some cases:

Producers that made use of these extensions were never implemented or publicly released.
Producers that made use of these extensions are so obsolete that not only does the tool chain no longer exist but the hardware that this toolchain was designed for no longer exists except as historical artifacts.
Official versions of DWARF encodings that accomplish the same thing have been added to the standard making the vendor extension obsolete.

There are several problems that the accumulation of these vendor extensions over the years has led to:

Some DWARF constructs are running out of available encodings that do not overlap with the ones used by or reserved by the standard. In particular the range of vendor extension encoding is running particularly short in the range of DW_OP encoding. This has led to several informal proposals within the vendor community along the lines of an official proposal 230324.1 Expression Operation Vendor Extensibility Opcode. The basic idea in most of these proposals is to extend the range of available vendor extension encodings. While DWARF expression operations, Section 2.5, has the most extreme shortage of vendor encodings, other DWARF constructs are also running out of usable encodings.
While producers do not incur cost for legacy producer provided operations and tags because producers can simply not use them, consumers do incur a cost. Consumers which are focused on compatibility often times need to interpret them. This forces consumers who want to be compatible to continue to include code needed to support those old elements while handling new structures. Since few if any producers use these, tools that produce these elements must be maintained in an operable state or the consumer risks that the code remains untested and subject to bit rot.
Extreme backward compatibility works against code maintainability. The code which inteprets the old structures must be integration tested with support for new structures.
For consumers it increases the size of the required test matrices that are needed to ensure compatibility.
It makes it harder to design new proposals to address new problems. The requirement that everything needs to be backward compatible for all time introduces design constraints which make it difficult for people to solve new problems. The conditions that existed at the time when a particular proposal was adopted may not continue to apply at the time when a new problem is addressed.

Overview

This proposal is in addition to 230324.1 and similar informal approaches to increase the available encoding space.

It makes explicit that Vendor Extensions can only be interpreted in the context of a particular producer and a DWARF standard version. This allows obsolete vendor extensions to be retired when a new version of the standard is released.

A registry would be published on the DWARF standard’s web site so that consumers can easily find it. Each extension would be listed along with the producers that emit it and the range of DWARF standard versions that it applies to. When new DWARF standard versions are released, each extension will be reconsidered to see if it continues to add utility in the context of the new version of the DWARF standard.

It is assumed that many extensions in this registry will automatically carried over between versions of the standard but it allows for obsolete extensions to be retired and their encoding to be reused when a new version of the standard is released.

Proposed Changes

Section 1.3.13

Replace the following:

Vendors may also use debugging information entries and attributes deﬁned here in new situations. Future versions of this document will not use names or values reserved for vendor speciﬁc additions. All names and values not reserved for vendor additions, however, are reserved for future versions of this document

With:

Producers may also use debugging information entries and attributes defined here in new situations. Future versions of this document will not use names or values reserved for these specific additions but the meanings of these encoding may change from DWARF version to version. Therefore the values of these extensions must be interpreted in the context of the DWARF standard versions for which they apply. All names and values not explicitly reserved for producer specific additions, however, are reserved for future versions of this document.

Section 7.1 Vendor Extensibility

Add a new rule 1 at the end of the section:

To ensure that extensions added by one producer may be safely ignored by consumers that do not understand those extensions, the following rules must be followed:

Producer defined extensions must be evaluated in the context of a DWARF standard version.

And renumber the existing rule numbers to 2..5.

2024-03-25: Split from 231110.1.

2024-06-01: Update proposal with additional rational.

2024-06-10: Rejected. This suggestion was made mostly in reaction to the scarcity of DW_OP encoding space, which has been resolved by 230324.1.