195 lines
8 KiB
Markdown
195 lines
8 KiB
Markdown
---
|
||
permalink: "/{{ year }}/{{ month }}/{{ day }}/build-your-project"
|
||
title: "Build your project"
|
||
published_date: "2020-06-19 13:46:00 +0200"
|
||
layout: post.liquid
|
||
data:
|
||
route: blog
|
||
tags:
|
||
- rust
|
||
---
|
||
|
||
What if you could build a Rust project without Cargo?
|
||
It's not an actual problem I encountered.
|
||
It's still a problem I looked into.
|
||
|
||
> Cargo is the Rust package manager.
|
||
> Cargo downloads your Rust package’s dependencies, compiles your packages, makes distributable packages,
|
||
> and uploads them to crates.io, the Rust community’s package registry.
|
||
>
|
||
> _(from the [Cargo Book][cargo])_
|
||
|
||
That's a lot of work this tool does and it's very good at doing it.
|
||
|
||
I wanted to understand just the bit where it takes your Rust code and builds that into a binary that you can execute and run on your machine.
|
||
I didn't really want to read all of Cargo, so instead I turned to what Cargo itself tells me.
|
||
So I took the output of `cargo build --verbose` and turned that into simple build instructions.
|
||
|
||
What does it look like?
|
||
|
||
```
|
||
$ cargo build --verbose
|
||
[...]
|
||
Compiling smallvec v1.4.0
|
||
Compiling matches v0.1.8
|
||
Compiling libc v0.2.71
|
||
Running `rustc --crate-name smallvec --edition=2018 /Users/jer/.cargo/registry/src/github.com-1ecc6299db9ec823/smallvec-1.4.0/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C debuginfo=2 -C metadata=d85954c5ff803feb -C extra-filename=-d85954c5ff803feb --out-dir /Users/jer/projects/rust/bygge/target/debug/deps -L dependency=/Users/jer/projects/rust/bygge/target/debug/deps --cap-lints allow`
|
||
Running `rustc --crate-name libc /Users/jer/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.71/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=bbaef975a82234c8 -C extra-filename=-bbaef975a82234c8 --out-dir /Users/jer/projects/rust/bygge/target/debug/deps -L dependency=/Users/jer/projects/rust/bygge/target/debug/deps --cap-lints allow --cfg freebsd11 --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_core_cvoid --cfg libc_packedN`
|
||
[...]
|
||
Running `rustc --crate-name bygge --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=c1c12e33a54c5b24 -C extra-filename=-c1c12e33a54c5b24 --out-dir /Users/jer/projects/rust/bygge/target/debug/deps -C incremental=/Users/jer/projects/rust/bygge/target/debug/incremental -L dependency=/Users/jer/projects/rust/bygge/target/debug/deps --extern cargo_lock=/Users/jer/projects/rust/bygge/target/debug/deps/libcargo_lock-43ce509885ad1aff.rlib --extern cargo_toml=/Users/jer/projects/rust/bygge/target/debug/deps/libcargo_toml-5b6fff58e6c9a546.rlib --extern dirs=/Users/jer/projects/rust/bygge/target/debug/deps/libdirs-4eb870f2ccf6ed43.rlib --extern petgraph=/Users/jer/projects/rust/bygge/target/debug/deps/libpetgraph-7cb995dcfae4c1f7.rlib --extern pico_args=/Users/jer/projects/rust/bygge/target/debug/deps/libpico_args-03badf8072a773d9.rlib`
|
||
Finished dev [unoptimized + debuginfo] target(s) in 30.82s
|
||
```
|
||
|
||
I removed a bunch of the output, but it all boils down to the same.
|
||
You can re-run these commands and you should end up with the same binary as the one Cargo puts into `target/debug/`.
|
||
|
||
You could put them all in a Makefile and use that to re-run them, but you would have gained nothing.
|
||
Changing your own code or adding, removing or updating dependencies won't be picked up unless you carefully specify all these dependencies in the Makefile
|
||
and update them when something changes.
|
||
|
||
We can automate that part.
|
||
|
||
Lately (again) I've been also reading about other build systems, such as [Ninja][ninja-essay] and was intrigued to rebuild that.
|
||
I haven't done that yet, but I decided to generate Ninja configuration to see how it behaves.
|
||
|
||
Let's combine these two things.
|
||
|
||
## The basic manual version
|
||
|
||
Let's work from a basic binary crate, with no dependencies and a single file.
|
||
|
||
```
|
||
$ cargo new --bin hello-world
|
||
Created binary (application) `hello-world` package
|
||
$ cat
|
||
fn main() {
|
||
println!("Hello, world!");
|
||
}
|
||
```
|
||
|
||
Building just that is easy:
|
||
|
||
```
|
||
$ rustc src/main.rs -o hello-world
|
||
$ ./hello-world
|
||
Hello, world!
|
||
```
|
||
|
||
If we rerun `rustc` it will re-build the code and generate a new file `hello-world` (that should be the same as the one from the first invocation).
|
||
That's a bit of extra work; if nothing changed we shouldn't need to rebuild.
|
||
Let's create a Ninja configuration file `build.ninja`:
|
||
|
||
```
|
||
# The build rule to invoke rustc.
|
||
# $in and $out are variables provided by ninja.
|
||
rule rustc
|
||
command = rustc $in -o $out
|
||
description = RUSTC $out
|
||
|
||
# Specifying dependencies:
|
||
# For `hello-world` to be build we invoke the above `rustc` rule with `src/main.rs` as $in
|
||
build hello-world: rustc src/main.rs
|
||
```
|
||
|
||
With [Ninja] installed we can build and run this:
|
||
|
||
```
|
||
$ ninja
|
||
[1/1] RUSTC hello-world
|
||
$ ./hello-world
|
||
Hello, world!
|
||
```
|
||
|
||
Re-running Ninja won't build it again:
|
||
|
||
```
|
||
$ ninja
|
||
ninja: no work to do.
|
||
```
|
||
|
||
Ninja knows that nothing changed and thus skips extra work.
|
||
If we modify the source it rebuilds:
|
||
|
||
```
|
||
$ echo >> src/main.rs
|
||
$ ninja
|
||
[1/1] RUSTC hello-world
|
||
$ ./hello-world
|
||
Hello, world!
|
||
```
|
||
|
||
We don't gain anything if we keep continuing writing a Ninja build configuration from hand.
|
||
Especially not once we add more files and some external crates.
|
||
|
||
## Bygge - build your project
|
||
|
||
Instead of writing out the whole Ninja file,
|
||
or coming up with some rules for Make that expand to the right files,
|
||
we can generate the build configuration once and then not touch it again until something changes.
|
||
That's exactly what Ninja was designed for:
|
||
|
||
> it is designed to have its input files generated by a higher-level build system, [...]
|
||
>
|
||
> _(from the [Ninja Website][ninja])_
|
||
|
||
After some more exploration with Cargo and the command lines it produces and the files it creates I managed to more-or-less auto-generate a Ninja configuration for a crate.
|
||
|
||
I've published this experiment as [bygge], the crate to build your crates (and itself).
|
||
|
||
### What it does
|
||
|
||
`bygge create` generates a Ninja build configuration (in `build.ninja` by default),
|
||
listing all the targets a binary crate depends on, including all crate dependencies.
|
||
`ninja` can then take this configuration and assemble the final binary.
|
||
The result should be about the same as an invocation of `cargo build`.
|
||
|
||
### What it doesn't
|
||
|
||
`bygge` is and never will be an alternative to Cargo.
|
||
Cargo is a full-fledged build system, aware of different build targets, allowing to enable features per dependency,
|
||
easily cross-compile to different targets and run the built programs as well as tests and generate documentation.
|
||
|
||
`bygge` ... builds.
|
||
|
||
### Features
|
||
|
||
* Builds cargo dependencies as listed in a project's Cargo.toml
|
||
* Can build only crates with a single binary target
|
||
* Runs on (at least) macOS and Linux
|
||
* No support for `build.rs` files
|
||
* No support for linking non-Rust libraries
|
||
* Hard-coded Cargo features for its dependencies
|
||
* Uses `cargo fetch` to download your project's dependencies
|
||
|
||
### Building bygge
|
||
|
||
Bygge is able to create a Ninja build configuration to build itself.
|
||
But first you need a compiled `bygge`.
|
||
It comes with a pre-generated configuration for that, that only works on my machine unless you change the paths to the local checkouts of the dependencies.
|
||
|
||
```
|
||
$ ninja -f manual.ninja
|
||
[29/29] RUSTC build/bygge
|
||
```
|
||
|
||
It builds a `build/bygge` file that is able to create the build configuration and run Ninja:
|
||
|
||
```
|
||
$ build/bygge create
|
||
==> Creating build file: build.ninja
|
||
==> Package: bygge
|
||
$ build/bygge build
|
||
[29/29] RUSTC build/bygge
|
||
$ build/bygge -V
|
||
bygge v0.1.0
|
||
```
|
||
|
||
You can also build it using `cargo build` and use `cargo run create` (or `target/debug/bygge create`) to create the `build.ninja` file.
|
||
|
||
And there you have it: a 300-line Ninja build configuration to build bygge.
|
||
|
||
[bygge]: https://github.com/badboy/bygge
|
||
[cargo]: https://doc.rust-lang.org/cargo/index.html
|
||
[ninja]: https://ninja-build.org/
|
||
[ninja-essay]: https://www.aosabook.org/en/posa/ninja.html
|