Rust in Lumberyard

Amazon Lumberyard, like CryEngine, is mostly C++. Some tools are done in Python, and Lua can additionally be used for scripting in-game. But, it being primarily C++ opens up the possibility of using languages that support native code bindings- like Rust.

As an experiment, I wanted to look at how feasible it would be to use Rust, starting with the simplest case- integrating a Rust static library.

Bindgen

bindgen uses clang/LLVM to generate Rust FFI bindings from C/C++ header files. This makes it easy to call functions defined in native libraries and work with native types and data. The user guide provides a good introduction.

In Cargo.toml:

[package]
name = "lmbr_sys"
version = "0.1.0"
edition = "2018"

[build-dependencies]
bindgen = "0.51"

In the build script, build.rs:

use std::{env, path::PathBuf};

fn main() {
    let builder = bindgen::Builder::default()
        .header("wrapper.hpp")
        .clang_arg("-I<Lumberyard root>/dev/Code/Framework/AzCore")
        .enable_cxx_namespaces()
        .generate_inline_functions(true)
        .whitelist_type("AZ::Debug::.*")
        ;
    let bindings = builder.generate().expect("Unable to generate bindings");
    let out_path = PathBuf::from(env::var("OUT_DIR").unwrap());
    bindings
        .write_to_file(out_path.join("bindings.rs"))
        .expect("Couldn't write bindings");
}

Builder provides a number of methods to control what and how bindings are generated. clang_arg() can pass options used when building a C++ program.

OUT_DIR is an environment variable set in build scripts to the path in the target/ directory containing intermediate files.

wrapper.hpp has declarations we want to generate bindings for. For starters, it only contains:

#include <AzCore/Debug/Trace.h>

AZ::Debug::Trace contains static methods used by the tracing macros to output text to the Lumberyard console/log.

Unfortunately, cargo build resulted in a Segmentation Fault.

Debugging Bindgen

The end of CONTRIBUTING.md has some good information on working with and debugging bindgen.

creduce is used to help produce minimal repro cases. It’s pretty neat, what starts as a 3.4MB pre-processed C++ source file ends up 4 lines that fail the same way.

Key to this is a “predicate script” that determines if the behavior you’re trying to isolate has occurred. My first predicate.sh:

#!/usr/bin/env bash

# Exit the script with a nonzero exit code if:
# * any individual command finishes with a nonzero exit code, or
# * we access any undefined variable.
set -eu

~/projects/rust-bindgen/csmith-fuzzing/predicate.py \
    --expect-bindgen-fail \
    --bindgen-args "-- -std=c++14" \
    ./wrapper.hpp

Running creduce on our header file produces:

creduce ./predicate.sh ./wrapper.hpp
# Output
}

Indeed, an invalid source file is the shortest possible way to get bindgen to fail. I need bindgen to partially work, but not output bindings:

~/projects/rust-bindgen/csmith-fuzzing/predicate.py \
    --expect-bindgen-fail \
    --bindgen-args "-- -std=c++14" \
    --bindgen-grep "Unhandled cursor kind 24" \
    ./wrapper.hpp

Produces:

template <class d> struct g {
  d e;
  static const long f = __alignof__(e);
};

Using a Custom LLVM

Turns out this problem was the same as an already reported issue and requires a fix in llvm.

Follow the “Getting Started” guide to build llvm trunk. The requirements don’t seem to mention this, but make sure you’ve got plenty of memory. A Linux VM with 8GB RAM and 1.5GB of swap space runs out of memory linking.

See if there’s any build options that interest you:

git clone https://github.com/llvm/llvm-project.git
cd llvm-project && mkdir build && cd build

# On Linux/MacOS
cmake -DLLVM_TARGETS_TO_BUILD="X86" -DLLVM_ENABLE_PROJECTS=clang -DCMAKE_BUILD_TYPE=Release ../llvm
make -j4

# On Windows
# OPTIONAL, specify install path: -DCMAKE_INSTALL_PREFIX="d:\llvm"
cmake -DLLVM_TARGETS_TO_BUILD="X86" -DLLVM_ENABLE_PROJECTS=clang -Thost=x64 -G "Visual Studio 16 2019" -A x64 ..\llvm
cmake --build . --config Release

Once the build succeeds, using this custom version of llvm/clang is as simple as setting the LIBCLANG_PATH environment variable:

export LIBCLANG_PATH=<llvm-project>/build/lib
# OR
LIBCLANG_PATH=<llvm-project>/build/lib cargo build

Or, if using Visual Studio and PowerShell:

$env:LIBCLANG_PATH="<llvm-project>/build/Debug/bin"
cargo build

If it fails with:

Caused by:
  process didn't exit successfully: `D:\projects\lmbr\target\debug\build\lmbr_sys-3651b1b85579815e\build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "Cannot find clang executable" }', src\libcore\result.rs:1165:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

You need to add the clang executable to your PATH environment variable:

 $env:PATH+=";d:\projects\llvm\build\release\bin"
 # OR, if you ran `cmake --install`
 $env:PATH+=";d:\llvm\bin"

This gets us past the seg fault and generates the first version of our Rust FFI bindings. However, now Rust compilation fails.

Conflicting Types

The initial generated bindings fail to compile with rustc:

error[E0391]: cycle detected when processing `root::AZ::u32`
  --> /XXX/lmbr/target/debug/build/lmbr_sys-4d14e371f0340e58/out/bindings.rs:91:24
   |
91 |         pub type u32 = u32;
   |                        ^^^
   |
   = note: ...which again requires processing `root::AZ::u32`, completing the cycle
note: cycle used when processing `root::AZ::Debug::ProfilerRegister::m_systemId`
  --> /XXX/lmbr/target/debug/build/lmbr_sys-4d14e371f0340e58/out/bindings.rs:547:33
   |
547|                 pub m_systemId: root::AZ::u32,
   |                                 ^^^^^^^^^^^^^

The issue here being Lumberyard defines u32 and others that conflict with Rust’s built-in types. In AzCore/base.h:

namespace AZ
{
#if AZ_TRAIT_COMPILER_INCLUDE_CSTDINT // Defined on Windows/Mac/Linux/Android
    typedef int8_t    s8;
    typedef uint8_t   u8;
    typedef int16_t   s16;
    typedef uint16_t  u16;
    typedef int32_t   s32;
    typedef uint32_t  u32;
#   if AZ_TRAIT_COMPILER_INT64_T_IS_LONG // int64_t is long
    typedef signed long long        s64;
    typedef unsigned long long      u64;
#   else
    typedef int64_t   s64;
    typedef uint64_t  u64;
#   endif
    //...

Causes bindgen to generate:

pub mod AZ {
        //...
        pub type s8 = i8;
        pub type u8 = u8; // error[E0391]: cycle detected
        pub type s16 = i16;
        pub type u16 = u16; // error[E0391]: cycle detected
        pub type s32 = i32;
        pub type u32 = u32; // error[E0391]: cycle detected
        pub type s64 = i64;
        pub type u64 = u64; // error[E0391]: cycle detected

One possible work-around:

.blacklist_type(r"AZ::u\d{2,3}") // Ban AZ::u32, etc. generated by C typedefs
.raw_line("type U32 = u32;") // Create top-level type aliases
.raw_line("type U64 = u64;")
// Define AZ::u32 in terms of our top-level aliases
.module_raw_lines("root::AZ", ["pub type u32 = crate::U32;", "pub type u64 = crate::U64;"].iter().map(|s| *s))

In short, for u32 (or another conflicting type):

1. Blacklist bindgen generated AZ::u32
2. Create type U32 = u32 alias in crate root
3. Output mod AZ { pub type u32 = crate::U32 } in place of blacklisted AZ::u32

The resulting generated bindings.rs becomes:

/* automatically generated by rust-bindgen */

type U32 = u32; // Our top-level type aliases
type U64 = u64;

#[allow(non_snake_case, non_camel_case_types, non_upper_case_globals)]
pub mod root {

    //...

    #[allow(unused_imports)]
    use self::super::root;
    pub mod AZ {
        #[allow(unused_imports)]
        use self::super::super::root;
        pub type u32 = crate::U32; // C typedefs defined via top-level aliases
        pub type u64 = crate::U64;
        //...

When primitive types are added to std, bindgen could use those and everything would be fine (without the work-around):

pub mod AZ {
    //...
    pub type u32 = std::primitive::u32;

If you get “expected syntax” or “unknown type” errors, make sure you pass -x c++ to clang or name the header file *.hpp (instead of *.h- see this issue).

As usual, Macs may require a bit more love. Depending on the version of macOS and Xcode installed, you may need some more header files (see this forum post and the relevant release notes):

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Likewise, to use rust-lldb:

ps aux | grep build
# Get PID
sudo env "PATH=$PATH" rust-lldb -p <PID>

Static Library

With our bindings generated, we can put them to use and create our static library as a package example called “staticlib”:

[[example]]
name = "staticlib"
crate-type = ["staticlib"] # Examples are executables by default

[dev-dependencies]
log = "0.4"

We’ll just create a simple function to call from C/C++:

use log::{info};

#[no_mangle]
pub extern fn example_static_lib() {
    lmbr_logger::init().unwrap();
    info!("RUST!!!!");
}

lmbr_logger is an implementation of the venerable log crate for Lumberyard:

[package]
name = "lmbr_logger"
version = "0.1.0"
edition = "2018"

[dependencies]
lmbr_sys = { version = "0.1", path = "../lmbr_sys" }
log = "0.4"

Following the example from the log docs:

use log::{Level, LevelFilter, Metadata, Record, SetLoggerError};
use std::{ffi::CString, os::raw::c_char};

struct LmbrLogger;

static LOGGER: LmbrLogger = LmbrLogger;

pub fn init() -> Result<(), SetLoggerError> {
    log::set_logger(&LOGGER)
        .map(|()| log::set_max_level(LevelFilter::Info))
}

impl log::Log for LmbrLogger {
    fn enabled(&self, metadata: &Metadata) -> bool {
        metadata.level() <= Level::Info
    }

    fn log(&self, record: &Record) {
        if self.enabled(record.metadata()) {
            let message = format!("{}", record.args());
            log("RUST", &message);
        }
    }

    fn flush(&self) {}
}

pub fn log(window: &str, message: &str) {
    let window = CString::new(window).unwrap();
    let window = window.as_bytes_with_nul().as_ptr() as *const c_char;
    let message = CString::new(message).unwrap();
    let message = message.as_bytes_with_nul().as_ptr() as *const c_char;
    unsafe {
        lmbr_sys::root::AZ::Debug::Trace::Output(window, message);
    }
}

Rust strings aren’t null-terminated, so we use CString to convert them as expected by native code.

cargo build --examples should produce our library in target/debug/examples/. We can verify our test function will be callable from C/C++:

dumpbin /symbols C:\XXX\lmbr\target\debug\examples\staticlib.lib | Select-String -Pattern example_static_lib
# Output
008 00000000 SECT4  notype ()    External     | example_static_lib

Per dumpbin /symbols docs, the third column value SECTx shows it is defined in the object file, and the fifth column value External shows it is externally visible.

Adding a library to Waf module

We previously introduced Lumberyard’s Waf-based build system.

We might look at integrating this as a 3rd-party library when we turn this into a Gem, for now we’ll just hard-code the path. In Sandbox/Editor/wscript:

# ...
hw = dict(
    # ...
    # OLD: win_lib = ['version'],
    win_lib = ['version', 'staticlib', 'Ws2_32', 'userenv'],
    libpath = ['c:/XXX/lmbr/target/debug/examples/'],

If you get “undefined symbol” link errors check out this issue.

Our example_static_lib() method is then usable from C/C++:

extern "C" void example_static_lib();

//...
example_static_lib();

Launch Editor and the output is visible in the console: