Merge ~blkerby/ubuntu/+source/rustc:std-vendor into ~canonical-foundations/ubuntu/+source/rustc:merge-1.91

[Brent Kerby]

This adds support for vendoring the standard library dependencies as part of the "rust-X.Y-src" package, enabling the "build-std" unstable Cargo feature to be used in offline environments. This includes:
- A new bash script "vendor-std-deps" to vendor the standard library dependencies into a temporary directory.
- A new Python script to scan the temporary output and autogenerate a section of "rust-X.Y-src.install.in", listing each vendored crate.
- A new target "vendor-std-deps" in "debian/rules" to invoke the new bash script and Python script.
- Addition of "vendor-std-deps" as a dependent to the "vendor-tarball" target; so going forward the vendoring of standard library dependencies should be handled automatically, without the maintainer needing to do any additional steps in the rustc upgrade process.

While working on this, I had some trouble reproducing the existing vendor tarball for rustc-1.91. One possible cause of differences would be a different toolchain version being used. To prevent this in the future, a check is added to ensure that the toolchain version used to create the vendor tarball matches the version of the toolchain being built.

I also had an issue where the vendored crate versions ended up not being the same versions as the ones actually required by the "build-std" option in the end. This appears to have been caused by the "dh_auto_configure" step removing (or rather renaming) the Cargo.lock files. I'm not aware of the context of why this was done, but leaving the original Cargo.lock files intact appears to work and resolves the issue. Leaving the Cargo.lock intact also seems ideal in order to match the upstream behavior as close as possible? While troubleshooting, I also set "locked-deps" to true in the "bootstrap.toml.in", to make sure the versions in the Cargo.lock files are exactly what are used; this isn't necessarily needed but may be worth keeping as a check, since if something were to cause the versions to change it would be unexpected and we may want to see an error.

[Brent Kerby (blkerby)]

PPA: https://launchpad.net/~blkerby/+archive/ubuntu/rustc-1.91-resolute-b

Steps to validate that this solution will work:
> add-apt-repository -y ppa:blkerby/rustc-1.91-resolute-b
> apt update
> apt install -y rustc-1.91 cargo-1.91 rust-1.91-src
> alias cargo=cargo-1.91
> cargo new hello
> cd hello
> cargo add serde # add dummy dependency so "cargo vendor" doesn't complain
> cargo vendor --versioned-dirs
> ln -s /usr/src/rustc-1.91.1/vendor/* vendor/
> mkdir .cargo
> cat >.cargo/config.toml
[source.crates-io]
replace-with = "vendored-sources"

[source.vendored-sources]
directory = "vendor"
> RUSTC_BOOTSTRAP=1 cargo run -Z build-std --offline

[Max Gilmour (maxgmr)]

The changes look good. I've also verified that this new toolchain is able to build itself successfully via autopkgtests. Merging.

review: Approve

diff --git a/debian/bootstrap.toml.in b/debian/bootstrap.toml.in
index 11c7214..ff8d267 100644
--- a/debian/bootstrap.toml.in
+++ b/debian/bootstrap.toml.in
@@ -3,7 +3,7 @@ change-id = 131075
 [build]
 submodules = false
 vendor = true
-locked-deps = false
+locked-deps = true
 verbose = VERBOSITY
 profiler = true
 
diff --git a/debian/changelog b/debian/changelog
index d5182fa..1694aa6 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,11 @@
+rustc-1.91 (1.91.1+dfsg-0ubuntu3) resolute; urgency=medium
+
+  * Add standard library vendored dependencies to rust-X.Y-src package
+  * Restrict toolchain used to make vendor tarball, for reproducibility
+  * Use frozen Cargo.lock dependencies
+
+ -- Brent Kerby <[e-mail]>  Fri, 02 Jan 2026 12:59:35 -0700
+
 rustc-1.91 (1.91.1+dfsg-0ubuntu2) resolute; urgency=medium
 
   * Set Architecture Level (ALS) to z15 on s390x (LP: #2126577)
diff --git a/debian/rules b/debian/rules
index 71a2bcc..a1c1f47 100755
--- a/debian/rules
+++ b/debian/rules
@@ -30,6 +30,8 @@ VENDOR_TARBALL := $(PKG_NAME_CHANGELOG)_$(DEB_VERSION_UPSTREAM).orig-vendor.tar.
 # This is a temporary directory used to store the filtered vendored dependencies.
 # It is packed into the vendor tarball.
 FILTERED_DEPS := $(CURDIR)/../vendor
+# Another temporary directory used to store the standard library dependencies.
+STD_DEPS := $(CURDIR)/../std-vendor
 
 include /usr/share/dpkg/buildflags.mk
 # needed for cross-compilation to avoid passing host CFLAGS to the BUILD
@@ -106,19 +108,32 @@ ifneq (,$(RUST_TEST_SELFBUILD))
 RUST_BOOTSTRAP_DIR = /usr/lib/rust-$(RUST_VERSION)
 endif
 
+BOOTSTRAP_VERSION := $(shell $(RUST_BOOTSTRAP_DIR)/bin/rustc --version | awk '{print $$2}')
+
 .PHONY: vendor-tarball-quick-check
 vendor-tarball-quick-check:
 	if [ -e $(CURDIR)/../$(VENDOR_TARBALL) ]; then \
 		echo "../$(VENDOR_TARBALL) already exists" >&2; \
 		exit 1; \
 	fi
+ifneq ($(BOOTSTRAP_VERSION),$(DEFINED_SRC_VERSION))
+	$(error The bootstrap rustc version does not match the upstream version: \
+	  Found "$(BOOTSTRAP_VERSION)", expected "$(DEFINED_SRC_VERSION)". \
+	  Please set RUST_BOOTSTRAP_DIR to point to a toolchain matching the upstream version.)
+endif
 
 .PHONY: vendor-deps
 vendor-deps:
 	debian/vendor-deps $(FILTERED_DEPS)
 
+.PHONY: vendor-std-deps
+vendor-std-deps:
+	debian/vendor-std-deps $(STD_DEPS)
+	python3 debian/scripts/update-rust-src-install $(STD_DEPS)
+	rm -rf $(STD_DEPS)
+
 .PHONY: vendor-tarball
-vendor-tarball: vendor-tarball-quick-check vendor-deps
+vendor-tarball: vendor-tarball-quick-check vendor-std-deps vendor-deps
 	tar --sort=name --mtime=@0 --owner=0 --group=0 --numeric-owner --pax-option=exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime -caf $(CURDIR)/../$(VENDOR_TARBALL) $(FILTERED_DEPS)
 	mk-origtargz --rename -c vendor $(CURDIR)/../$(VENDOR_TARBALL)
 	rm -rf $(FILTERED_DEPS)
@@ -339,11 +354,6 @@ debian/dh_auto_configure.stamp: debian/bootstrap.toml check-no-old-llvm
 	# v="$(DEB_VERSION)"; test "$$v" = "$${v%~exp*}" -o "$(DEB_DISTRIBUTION)" = "experimental" -o "$(DEB_DISTRIBUTION)" = "UNRELEASED"
 	$(PRECONFIGURE_CHECK)
 	if [ -d stage0 ]; then mkdir -p build && ln -sfT ../stage0 build/cache; fi
-	# don't care about lock changes
-	# keep in sync with d/vendor-deps
-	for ws in . src/bootstrap library src/tools/clippy/clippy_test_deps src/tools/rust-analyzer src/tools/cargo src/tools/rustbook; do \
-		if [ -f "$$ws/Cargo.lock" ]; then mv "$$ws/Cargo.lock" "$$ws/Cargo.lock.orig"; fi \
-	done
 	# We patched some crates so have to rm the checksums
 	find vendor -name .cargo-checksum.json -execdir "$(CURDIR)/debian/prune-checksums" "{}" +
 	# Link against system liblzma, see https://github.com/alexcrichton/xz2-rs/issues/16
@@ -575,6 +585,8 @@ endif
 
 
 override_dh_install-indep:
+	# Touch vendored crates to avoid upload failing with "time stamp too far in the past"
+	find vendor -exec touch {} +
 	dh_install
 	$(RM) -rf $(SRC_CLEAN:%=debian/rust-$(RUST_VERSION)-src/src/usr/src/rustc-$(RUST_LONG_VERSION)/%)
 	# Get rid of lintian warnings
diff --git a/debian/rust-X.Y-src.install.in b/debian/rust-X.Y-src.install.in
index 0dc17f4..1740a7f 100644
--- a/debian/rust-X.Y-src.install.in
+++ b/debian/rust-X.Y-src.install.in
@@ -13,3 +13,23 @@ Cargo.toml      usr/src/rustc-${env:RUST_LONG_VERSION}
 src             usr/src/rustc-${env:RUST_LONG_VERSION}
 library         usr/src/rustc-${env:RUST_LONG_VERSION}
 compiler        usr/src/rustc-${env:RUST_LONG_VERSION}
+# Do not edit: AUTOGENERATED
+vendor/libc-0.2.175 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/object-0.37.3 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/memchr-2.7.5 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/hashbrown-0.15.5 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/rand_core-0.9.3 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/cc-1.2.0 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/rand-0.9.2 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/cfg-if-1.0.1 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/miniz_oxide-0.8.9 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/adler2-2.0.1 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/unwinding-0.2.8 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/getopts-0.2.24 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/rustc-literal-escaper-0.0.5 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/rustc-demangle-0.1.26 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/addr2line-0.25.0 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/rand_xorshift-0.4.0 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/shlex-1.3.0 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+vendor/gimli-0.32.0 usr/src/rustc-${env:RUST_LONG_VERSION}/vendor
+# AUTOGENERATED END
diff --git a/debian/scripts/update-rust-src-install b/debian/scripts/update-rust-src-install
new file mode 100644
index 0000000..92f997b
--- /dev/null
+++ b/debian/scripts/update-rust-src-install
@@ -0,0 +1,46 @@
+#!/usr/bin/python3
+# Update the autogenerated part of rust-X.Y-src.install.in, by including the vendored
+# dependencies of the standard library.
+
+import argparse
+import os
+
+parser = argparse.ArgumentParser()
+parser.add_argument("vendor", help="path to vendor directory of standard library dependencies")
+args = parser.parse_args()
+
+install_path = "debian/rust-X.Y-src.install.in"
+install_list = []
+
+for x in os.listdir(args.vendor):
+    install_list.append(f"vendor/{x}" + " usr/src/rustc-${env:RUST_LONG_VERSION}" + f"/vendor")
+
+# Read the existing file
+with open(install_path, "r") as f:
+    lines = f.readlines()
+
+start_marker = "# Do not edit: AUTOGENERATED"
+end_marker = "# AUTOGENERATED END"
+
+start_idx = None
+end_idx = None
+
+for i, line in enumerate(lines):
+    if start_marker in line:
+        start_idx = i
+    if end_marker in line:
+        end_idx = i
+        break
+
+if start_idx is not None and end_idx is not None and start_idx < end_idx:
+    new_lines = (
+        lines[:start_idx + 1]
+        + [line + "\n" for line in install_list]
+        + lines[end_idx:]
+    )
+    with open(install_path, "w") as f:
+        f.writelines(new_lines)
+else:
+    raise RuntimeError(f"Could not find AUTOGENERATED markers in {install_path}")
+
+# print(args.vendor)
\ No newline at end of file
diff --git a/debian/vendor-deps b/debian/vendor-deps
index 1f91338..89118a0 100755
--- a/debian/vendor-deps
+++ b/debian/vendor-deps
@@ -7,7 +7,7 @@
 # It is typically invoked by d/rules vendor-deps.
 #
 # Manual invocation example for 1.89:
-# RUST_BOOTSTRAP_DIR=~/.rustup/toolchains/1.89.0-x86_64-unknown-linux-gnu/bin/rustc debian/vendor-deps ./filtered-vendor
+# RUST_BOOTSTRAP_DIR=~/.rustup/toolchains/1.89.0-x86_64-unknown-linux-gnu/ debian/vendor-deps ./filtered-vendor
 
 set -e
 
@@ -127,7 +127,7 @@ for forced_std_dep in "${forced_std_deps[@]}"; do
     std_dep_make_non_optional "$forced_std_dep"
 done
 
-cargo +nightly vendor-filterer "$vendored_deps" \
+RUSTC_BOOTSTRAP=1 ${RUST_BOOTSTRAP_DIR}/bin/cargo vendor-filterer "$vendored_deps" \
     --versioned-dirs \
     "${vendor_filterer_args[@]}"
 
diff --git a/debian/vendor-std-deps b/debian/vendor-std-deps
new file mode 100755
index 0000000..0375443
--- /dev/null
+++ b/debian/vendor-std-deps
@@ -0,0 +1,40 @@
+#!/bin/bash
+# Vendor the standard library dependencies. Currently this is needed
+# in order to support use of the unstable "-Z build-std" rustc feature in offline
+# environments. Once the feature is stabilized it should become no longer necessary 
+# (pending resolution of https://github.com/rust-lang/wg-cargo-std-aware/issues/23
+# and related issues).
+
+std_vendored_deps="$1"
+if [ -d "$std_vendored_deps" ]; then
+    echo "directory $std_vendored_deps already exists" >&2
+    exit 1
+fi
+
+# List of supported platforms; edit as needed
+supported_platforms=(
+    "x86_64-unknown-linux-gnu"
+    "aarch64-unknown-linux-gnu"
+    "i686-unknown-linux-gnu"
+    "armv7-unknown-linux-gnueabihf"
+    "powerpc64le-unknown-linux-gnu"
+    "powerpc-unknown-linux-gnu"
+    "s390x-unknown-linux-gnu"
+    "riscv64a23-unknown-linux-gnu"
+)
+
+vendor_filterer_args=()
+for platform in "${supported_platforms[@]}"; do
+    vendor_filterer_args+=("--platform=$platform")
+done
+
+quilt push -a
+
+echo "Vendoring standard library dependencies to ${std_vendored_deps}..."
+RUSTC_BOOTSTRAP=1 ${RUST_BOOTSTRAP_DIR}/bin/cargo vendor-filterer \
+  --manifest-path library/Cargo.toml \
+  --versioned-dirs \
+  "${vendor_filterer_args[@]}" \
+  ${std_vendored_deps}
+
+quilt pop -a