[Feature] support abi_check, Identify potential ABI incompatibility issues

[anjiahao1]

ABI Check Tool

Overview

abi_check.py is a Python tool designed for checking binary compatibility between different versions of ELF files and static libraries. It helps ensure that Application Binary Interface (ABI) compatibility is maintained when updating firmware components, preventing runtime errors caused by incompatible changes.

What is ABI Compatibility?

Application Binary Interface (ABI) compatibility refers to the binary compatibility between compiled code and libraries. When you have static libraries (.a files) that were compiled against one version of a kernel/ELF, they must continue to work correctly when linked against a new version. ABI compatibility ensures that:

• Function signatures remain unchanged
• Structure layouts (size, member offsets, member types) stay the same
• Data types maintain their size and alignment
• Calling conventions remain consistent

Why ABI Check is Needed

Problem Scenario

In embedded systems, it's common to have:

1. Static libraries compiled against one version of the kernel/RTOS
2. Kernel/ELF files that get updated independently
3. Firmware updates where only certain components are updated

If the kernel/ELF changes its internal structures or function signatures without proper versioning, previously compiled static libraries may:

• Read wrong memory offsets when accessing structures
• Call functions with incorrect parameters
• Cause crashes, memory corruption, or undefined behavior

Example Issue

Consider this structure change in a semaphore implementation:

Old Version:

struct sem_s {
    volatile int16_t semcount;  /* Offset: 0, Size: 2 */
    uint8_t flags;              /* Offset: 2, Size: 1 */
    /* 1 byte padding */
    dq_queue_t waitlist;         /* Offset: 4, Size: 8 */
    /* Total size: 12 bytes */
};

New Version:

struct sem_s {
    volatile int16_t semcount;  /* Offset: 0, Size: 2 */
    /* Total size: 2 bytes */
};

A static library compiled against the old version would try to access waitlist at offset 4, but in the new version, accessing that memory would be undefined behavior. This is exactly the kind of issue abi_check helps detect.

Features

1. API Signature Extraction

Extract external API function signatures used by static libraries from an ELF file.

abi_check.py -e nuttx.elf -a libapp.a libdriver.a -j api_signatures.json

What it does:

• Parses static libraries to find undefined symbols (external functions they call)
• Finds these functions in the ELF file
• Extracts function signatures including:
  • Return type and size
  • Parameter types, sizes, and names
  • Structure layout if parameters are structures
• Outputs results to a JSON file

2. Version Comparison

Compare function signatures between two different ELF versions.

# Generate signatures for old version
abi_check.py -e nuttx_old.elf -a libapp.a -j old_api.json

# Generate signatures for new version
abi_check.py -e nuttx_new.elf -a libapp.a -j new_api.json

# Compare them
abi_check.py -i old_api.json new_api.json

What it detects:

• Functions that disappeared between versions
• Functions that appeared (new APIs)
• Return type changes
• Parameter count changes
• Parameter type/size changes
• Structure layout changes (size, offsets, member types)

3. Structure Consistency Check

Check if structures with the same name have different member definitions within a single ELF file.

abi_check.py -s -e nuttx.elf

What it detects:

• Multiple definitions of the same structure name
• Different member layouts for same-named structures
• This can happen when different compilation units have different structure definitions

Installation

Prerequisites

The tool requires:

1. Python 3 with standard library
2. pyelftools for ELF parsing:

   pip install pyelftools

3. pahole (dwarves package) for structure layout analysis:

   # Ubuntu/Debian
   sudo apt-get install dwarves
   
   # Fedora/RHEL
   sudo dnf install dwarves
   
   # Alpine
   sudo apk add dwarves

4. GNU ar for extracting static libraries (usually pre-installed)

Usage

Command-Line Options

usage: abi_check.py [-h] [-a LIB [LIB ...]] [-e ELF] [-c] [-d] [-j JSON]
                   [-s] [-i INPUT_JSON [INPUT_JSON ...]]

optional arguments:
  -h, --help            Show help message and exit
  -a LIB [LIB ...], --lib LIB [LIB ...]
                        Path to static libraries (.a files)
  -e ELF, --elf ELF     Path to ELF file
  -c, --check           If static library contains debug information,
                        try to find function in static library,
                        and output to lib_<json> file
  -d, --dump            Dump result to console
  -j JSON, --json JSON  Save result to json file (default: out.json)
  -s, --struct_check    Check for struct differences within ELF
  -i INPUT_JSON [INPUT_JSON ...]
                        Compare two JSON files (diff mode)

Use Case 1: Extract API Signatures

Extract all external APIs used by static libraries and find their signatures in the ELF file.

abi_check.py \
  -e nuttx.elf \
  -a libapps.a libdrivers.a libnetwork.a \
  -j api_signatures.json

Output:

[
  [
    [
      "sem_wait",
      {
        "return": {
          "type": "int",
          "size": 4,
          "field": []
        },
        "parameters": [
          {
            "type": "FAR sem_t *",
            "size": 4,
            "field": []
          }
        ]
      }
    ],
    [
      "open",
      {
        "return": {
          "type": "int",
          "size": 4,
          "field": []
        },
        "parameters": [
          {
            "type": "FAR const char *",
            "size": 4,
            "field": []
          },
          {
            "type": "int",
            "size": 4,
            "field": []
          }
        ]
      }
    ]
  ],
  [
    "undefined_function_1",
    "undefined_function_2"
  ]
]

The JSON output contains:

1. Array of function signatures (function name, return type, parameters)
2. Array of undefined functions that couldn't be found in the ELF

With dump option:

abi_check.py -e nuttx.elf -a libapp.a -j api.json -d

Output to console:

int sem_wait(FAR sem_t *sem);
int open(FAR const char *path, int oflags, ...);
Function undefined_function not found in elf file
Function another_undefined not found in elf file

Use Case 2: Compare Two Versions

Compare API signatures between old and new firmware versions.

# Step 1: Generate signatures for old version
abi_check.py -e nuttx_v1.0.elf -a libapp.a libdriver.a -j v1_signatures.json

# Step 2: Generate signatures for new version
abi_check.py -e nuttx_v2.0.elf -a libapp.a libdriver.a -j v2_signatures.json

# Step 3: Compare
abi_check.py -i v1_signatures.json v2_signatures.json

Example output showing incompatibilities:

Function sem_wait return type is different

  int have Different:
    Field int size is different  4 != 8

Function open parameters count is different

Function read parameter 2 is different

  FAR size_t * have Different:
    Field size_t * type is different
    Field size_t * size is different  8 != 4

Use Case 3: Structure Consistency Check

Check for structure definition inconsistencies within a single ELF file.

abi_check.py -s -e nuttx.elf

Example output showing structure mismatch:

struct sem_s {
        volatile int16_t           semcount;             /*     0     2 */
        uint8_t                    flags;                /*     2     1 */

        /* XXX 1 byte hole, try to pack */

        dq_queue_t                 waitlist;             /*     4     8 */

        /* size: 12, cachelines: 1, members: 3 */
        /* sum members: 11, holes: 1, sum holes: 1 */
        /* last cacheline: 12 bytes */

}; at /home/xxx/include/semaphore.h:99

struct sem_s {
        volatile int16_t           semcount;             /*     0     2 */

        /* size: 2, cachelines: 1, members: 1 */
        /* last cacheline: 2 bytes */

}; at /home/xxx/include/semaphore.h:98

------

This output shows that struct sem_s has two different definitions at different source locations, which is a potential ABI issue.

Use Case 4: Verify Static Library Compatibility

When static libraries contain debug information, you can verify if they match the ELF file.

abi_check.py \
  -e nuttx.elf \
  -a libapp.a \
  -j api_signatures.json \
  --check

This generates two files:

• api_signatures.json - API signatures from ELF
• lib_api_signatures.json - API signatures from static libraries

You can then compare these to ensure compatibility.

Workflow Examples

Firmware Update Validation

When updating firmware while keeping static libraries unchanged:

#!/bin/bash
# validate_firmware_update.sh

OLD_ELF="nuttx_v1.0.elf"
NEW_ELF="nuttx_v2.0.elf"
LIBS="libapp.a libdriver.a libnetwork.a"

# Extract old API signatures
abi_check.py -e $OLD_ELF -a $LIBS -j old_signatures.json

# Extract new API signatures
abi_check.py -e $NEW_ELF -a $LIBS -j new_signatures.json

# Compare for compatibility
echo "Checking ABI compatibility..."
abi_check.py -i old_signatures.json new_signatures.json

# Check exit code
if [ $? -eq 0 ]; then
    echo "✓ ABI compatibility maintained"
    echo "Safe to proceed with firmware update"
else
    echo "✗ ABI incompatibilities detected"
    echo "Firmware update would break existing libraries"
    exit 1
fi

CI/CD Integration

Integrate ABI checking into your build pipeline:

# .github/workflows/abi-check.yml
name: ABI Compatibility Check

on: [pull_request, push]

jobs:
  abi-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Install dependencies
        run: |
          sudo apt-get update
          sudo apt-get install -y python3-pip dwarves
          pip3 install pyelftools

      - name: Build firmware
        run: |
          cmake -B build
          cmake --build build

      - name: Extract baseline signatures
        run: |
          abi_check.py -e build/baseline.elf -a lib/*.a -j baseline.json

      - name: Extract new signatures
        run: |
          abi_check.py -e build/nuttx.bin -a lib/*.a -j new.json

      - name: Compare signatures
        run: |
          abi_check.py -i baseline.json new.json

      - name: Check for differences
        run: |
          # Fail if abi_check reports incompatibilities
          ! abi_check.py -i baseline.json new.json | grep -q "is different"

Troubleshooting

Error: "elf and lib must be provided"

You must provide both an ELF file and at least one static library:

# Correct
abi_check.py -e nuttx.elf -a libapp.a

# Incorrect
abi_check.py -e nuttx.elf

Error: "Error: elf must be provided"

When using --struct_check, you must provide an ELF file:

# Correct
abi_check.py -s -e nuttx.elf

# Incorrect
abi_check.py -s

Warning: "File already exists, will be overwritten"

The tool warns before overwriting existing JSON files:

# Specify different output file to avoid overwriting
abi_check.py -e nuttx.elf -a libapp.a -j new_signatures.json

Missing pahole

If you get an error about pahole, install the dwarves package:

# Ubuntu/Debian
sudo apt-get install dwarves

# Check installation
pahole --version

Function not found in ELF

If you see warnings about functions not found in the ELF file:

Function some_function not found in elf file

This means:

1. The static library references a function not present in the ELF
2. This could be intentional (weak symbols, optional features)
3. Or it could indicate a missing dependency

Technical Details

How It Works

1. Symbol Collection:

   • Extracts .obj files from static libraries using ar x
   • Parses ELF symbol tables to find undefined symbols (external references)

2. Signature Extraction:

   • Uses DWARF debug information in the ELF file
   • Traverses DIE (Debugging Information Entry) structures
   • Extracts function prototypes including return types and parameters

3. Structure Analysis:

   • Uses pahole to dump structure layouts
   • Parses structure definitions including member offsets and types
   • Compares structures across different compilation units

4. Comparison:

   • Compares function signatures name-by-name
   • Detects changes in types, sizes, and offsets
   • Reports incompatibilities in human-readable format

Output Format

The JSON output structure:

[
  [  // Array of function signatures
    [  // Single function
      "function_name",
      {
        "return": {
          "type": "return_type_name",
          "size": size_in_bytes,
          "field": [
            // If return type is a struct, fields go here
            {
              "type": "field_type",
              "size": field_size,
              "offset": field_offset
            }
          ]
        },
        "parameters": [
          {
            "type": "parameter_type",
            "size": parameter_size,
            "field": []  // Empty unless parameter is a struct
          }
        ]
      }
    ]
  ],
  [  // Array of undefined functions
    "undefined_function_1",
    "undefined_function_2"
  ]
]

Best Practices

1. Baseline Early: Establish ABI signatures early in your project lifecycle
2. Automate Checks: Integrate into CI/CD pipeline to catch changes early
3. Document APIs: Keep clear documentation of public APIs and their stability guarantees
4. Version Libraries: Use versioning for libraries when breaking changes are necessary
5. Test Thoroughly: Combine ABI checking with integration testing
6. Review Output: Don't ignore warnings about undefined functions or structure mismatches

Limitations

1. Requires Debug Info: The tool depends on DWARF debug information; compile with -g flag
2. Static Libraries Only: Currently designed for static library analysis, not shared libraries
3. Python 3: Requires Python 3.x
4. pahole Dependency: External tool dependency for structure analysis
5. No Runtime Checks: Only checks compile-time ABI, not runtime behavior

[xiaoxiang781216]

@anjiahao1 please fix typo:

1 file would be left unchanged.
Used config files:
    1: .codespellrc
/home/runner/work/nuttx/nuttx/nuttx/tools/abi_check.py:279: unkown ==> unknown
/home/runner/work/nuttx/nuttx/nuttx/tools/abi_check.py:573: Erorr ==> Error
Some checks failed. For contributing guidelines, see: