<p>fanquake answered this on IRC:</p>
<blockquote>
<p>There are a number of benefits that come with using Guix as a release
build environment (or just as a build system in general).</p>
<p>Using Gitian, you have to pick a Linux distro to build on. This
defines your toolchains, available packages, the glibc your building
against etc.</p>
<p>This also means you're subject to choices of the upstream package
maintainers. They control how/when package versions are updated, what
patches are applied to them, how they are compiled etc. This has
downsides.</p>
<p>As an example, our security-check tests rely on a number of -no-*
options being present in mingw-w64 ld. These options were patched into
the ld in the Ubuntu Bionic binutils-mingw-w64 package, however, the
binutils maintainer chose to stop doing that in Ubuntu Focal, which
meant our security-check tests would not have been able to run (they
have since decided to re-patch them in Hirsute). See #18629 for more
details.</p>
<p>In Guix, because we're in full control of our toolchains, we don't
have to worry about issues like this, and can just directly apply the
patches we want to use, and this is exactly what we've done in #22381,
where we started patching the -no-* linker options into our mingw-w64
toolchain. Compiler / linker options or defaults that we may rely on,
can no-longer be randomly changing out from underneath us.</p>
<p>To that, some might say &quot;well just don't change the Ubuntu base image,
and things won't change&quot;. To that, I'd say, I don't want the project
to be in a position where we are &quot;stuck&quot;, and can't update our base
image to use new tools (i.e compilers), because by doing so we'll just
break something else, like our security tests.</p>
<p>It would seem smarter to just remove the potential for those kinds of
problem entirely, by constructing and using exactly the release build
environment we want.</p>
<p>&quot;Well why not just patch / change things in gitian&quot; - I wont say much
on this, except for, the gitian environment is not at all setup to do
the same kind of patching that we can fairly trivially achieve in
Guix. Trying to patch and compile, for example, the mingw-w64
toolchain in a gitian descriptor, would also just turn into a terrible
mess.</p>
<p>The same can be said about trying to gitian build anything using a
glibc different than what is already available on that version of
Ubuntu.</p>
<p>This has backward-compatibility implications, as the version of glibc
you're building against, essentially determines your runtime glibc
compatiblity. However this can be extended by using the sorts of
&quot;workarounds&quot; that we currently have in our glibc-back-compat code.</p>
<p>Now, similar to the situation where you might want to use a newer
Ubuntu release (for any number of reasons), doing so will mean that
you have to build against a newer glibc (you just get whatever version
comes with that release of Ubuntu).</p>
<p>This means that as you use newer versions of glibc, the number of
&quot;workarounds&quot; you need to maintain backwards compatibility pile up,
get continually more complicated, and even start to leak out of
Bitcoin Core code, and into our dependency system. See all the PRs
linked in this comment:
<a href="https://github.com/bitcoin/bitcoin/pull/22418#issuecomment-876379846" rel="nofollow noreferrer">https://github.com/bitcoin/bitcoin/pull/22418#issuecomment-876379846</a>.</p>
<p>Changes like that leaking into depends are bad, because now normals
builders, using depends, are now being subject to the side-effects of
patches that are only really necessary in our release build
environment. Maybe you'd argue that in that case we should only apply
the depends patches when building releases, however then you've got a
situation where release builds are even more divergent from
&quot;normal&quot;/developer builds, and this means either maintaining an even
more complex CI / testing routines, or they just end up less tested
(guess which one is more likely to happen).</p>
<p>If that all just sounds like a complicated mess, it's basically
because it is. However the solution, using Guix, is actually pretty
straight forward.</p>
<p>When we are in complete control of our release environment, we can
pick exactly the version of glibc we want to use (even at a per-HOST
level), something that we definitely couldn't achieve, in any sort of
straight forward fashion, if at all, using gitian.</p>
<p>This is something we've done recently, in #22365. We are now building
using glibc 2.27 for the RISC-V HOST, and using glibc 2.24 for all
others, while maintaining runtime compatibility with glibc 2.17.
You'll note this this is achieved without needing to use any of the
work arounds from our glibc-back-compat code (#22405), or any of the
other PRs / changes mentioned in the comment above.</p>
<p>These are just two very practical benefits that using Guix provides
(there are more), which ultimately all boil down to us being in much
greater control of our release build environment. Something I am very
happy about, and I think makes a lot of sense for a project like
Bitcoin Core.</p>
<p>The fact that there are also many people in the Guix space actively
working on bootstrapability is just another big bonus, and even if
that doesn't work exactly like some may want it too right now, it is
rapidly being improved, month on month.</p>
<p>Guix is also a much more likely pathway to fully bootstrapable Bitcoin
Core builds that what gitian could ever provide.</p>
</blockquote>
<p>dongcarl stated:</p>
<blockquote>
<p>I don't think we're going to run Guix inside Gitian for the releases,
but personally I think people are free to do whatever they think make
sense if they're committed to maintaining it</p>
</blockquote>
<p>luke-jr added:</p>
<blockquote>
<p>You don't lose those Guix benefits by running it inside gitian; gitian just provides the isolation needed to avoid regressing on security</p>
</blockquote>

CC BY-SA 4.0
2021-07-19T11:09:58.790Z
2021-07-19T11:09:58.790Z
Michael Folkson
<p>Gitian achieved reproducibility through the use of a virtual machine to isolate the toolchain used for building. This means that any and all unaudited and untrusted third party binaries execute in a contained environment where they are less likely to pollute and cause damage to the host system.</p>
<p>On the other hand, Guix is intended to be run directly on the host itself, so if there are any unaudited and untrusted third party binaries executed, they could cause damage to the host.</p>
<p>So some developers, notably luke-jr, want to perform Guix builds within a virtual machine to minimize this risk. This could be achieved by installing Guix inside of a virtual machine, but we already have Gitian and the infrastructure around it, so it is also possible to achieve this by running Guix within Gitian and avoid the need for such developers to figure out how to install Guix in a VM, and maintain the VM.</p>
<p>Furthermore, many people who have participated in the reproducible builds process may not have Guix setup yet. By doing Guix inside of Gitian, it will be possible for those people to participate in the new release process with minimal changes. Providing an easy way to transition from Gitian to Guix is the primary motivation for enabling Guix within Gitian.</p>
<p>Of course the recommended method is still to use Guix directly rather than through Gitian. There is a significant performance penalty for using Guix inside of Gitian. Additionally, running Guix directly allows for issues to be more easily debugged and worked through as the toolchain and extra binaries built will still be available to the builder.</p>

CC BY-SA 4.0
2021-07-19T16:54:02.470Z
2021-07-19T16:54:02.470Z
Ava Chow
<p>What's the purpose of using Guix within Gitian? Doesn't that reintroduce dependencies and security concerns that were the purpose of transitioning from Gitian to Guix?</p>
<p>My understanding was that Gitian was reproducible but not bootstrappable and that Guix is an improvement to bootstrappable and reproducible?</p>
<p>This question was asked by robertspigler on IRC.</p>

<compiling><gitian><guix>
What's the purpose of using Guix within Gitian? Doesn't that reintroduce dependencies and security concerns?
Michael Folkson
# What's the purpose of using Guix within Gitian? Doesn't that reintroduce dependencies and security concerns?

(Asked by: username_0 on 2021-07-19)

What's the purpose of using Guix within Gitian? Doesn't that reintroduce dependencies and security concerns that were the purpose of transitioning from Gitian to Guix?

My understanding was that Gitian was reproducible but not bootstrappable and that Guix is an improvement to bootstrappable and reproducible?

This question was asked by robertspigler on IRC.

**Tags:** compiling, gitian, guix


---

## 2 answers

### Answer by username_1 on 2021-07-19. Score: 4 | Accepted answer

Gitian achieved reproducibility through the use of a virtual machine to isolate the toolchain used for building. This means that any and all unaudited and untrusted third party binaries execute in a contained environment where they are less likely to pollute and cause damage to the host system.

On the other hand, Guix is intended to be run directly on the host itself, so if there are any unaudited and untrusted third party binaries executed, they could cause damage to the host.

So some developers, notably luke-jr, want to perform Guix builds within a virtual machine to minimize this risk. This could be achieved by installing Guix inside of a virtual machine, but we already have Gitian and the infrastructure around it, so it is also possible to achieve this by running Guix within Gitian and avoid the need for such developers to figure out how to install Guix in a VM, and maintain the VM.

Furthermore, many people who have participated in the reproducible builds process may not have Guix setup yet. By doing Guix inside of Gitian, it will be possible for those people to participate in the new release process with minimal changes. Providing an easy way to transition from Gitian to Guix is the primary motivation for enabling Guix within Gitian.

Of course the recommended method is still to use Guix directly rather than through Gitian. There is a significant performance penalty for using Guix inside of Gitian. Additionally, running Guix directly allows for issues to be more easily debugged and worked through as the toolchain and extra binaries built will still be available to the builder.


---

### Answer by username_0 on 2021-07-19. Score: 0

fanquake answered this on IRC:

> There are a number of benefits that come with using Guix as a release
> build environment (or just as a build system in general).
>
> Using Gitian, you have to pick a Linux distro to build on. This
> defines your toolchains, available packages, the glibc your building
> against etc.
>
> This also means you're subject to choices of the upstream package
> maintainers. They control how/when package versions are updated, what
> patches are applied to them, how they are compiled etc. This has
> downsides.
>
> As an example, our security-check tests rely on a number of -no-\*
> options being present in mingw-w64 ld. These options were patched into
> the ld in the Ubuntu Bionic binutils-mingw-w64 package, however, the
> binutils maintainer chose to stop doing that in Ubuntu Focal, which
> meant our security-check tests would not have been able to run (they
> have since decided to re-patch them in Hirsute). See #18629 for more
> details.
>
> In Guix, because we're in full control of our toolchains, we don't
> have to worry about issues like this, and can just directly apply the
> patches we want to use, and this is exactly what we've done in #22381,
> where we started patching the -no-\* linker options into our mingw-w64
> toolchain. Compiler / linker options or defaults that we may rely on,
> can no-longer be randomly changing out from underneath us.
>
> To that, some might say "well just don't change the Ubuntu base image,
> and things won't change". To that, I'd say, I don't want the project
> to be in a position where we are "stuck", and can't update our base
> image to use new tools (i.e compilers), because by doing so we'll just
> break something else, like our security tests.
>
> It would seem smarter to just remove the potential for those kinds of
> problem entirely, by constructing and using exactly the release build
> environment we want.
>
> "Well why not just patch / change things in gitian" - I wont say much
> on this, except for, the gitian environment is not at all setup to do
> the same kind of patching that we can fairly trivially achieve in
> Guix. Trying to patch and compile, for example, the mingw-w64
> toolchain in a gitian descriptor, would also just turn into a terrible
> mess.
>
> The same can be said about trying to gitian build anything using a
> glibc different than what is already available on that version of
> Ubuntu.
>
> This has backward-compatibility implications, as the version of glibc
> you're building against, essentially determines your runtime glibc
> compatiblity. However this can be extended by using the sorts of
> "workarounds" that we currently have in our glibc-back-compat code.
>
> Now, similar to the situation where you might want to use a newer
> Ubuntu release (for any number of reasons), doing so will mean that
> you have to build against a newer glibc (you just get whatever version
> comes with that release of Ubuntu).
>
> This means that as you use newer versions of glibc, the number of
> "workarounds" you need to maintain backwards compatibility pile up,
> get continually more complicated, and even start to leak out of
> Bitcoin Core code, and into our dependency system. See all the PRs
> linked in this comment:
> <https://github.com/bitcoin/bitcoin/pull/22418#issuecomment-876379846>.
>
> Changes like that leaking into depends are bad, because now normals
> builders, using depends, are now being subject to the side-effects of
> patches that are only really necessary in our release build
> environment. Maybe you'd argue that in that case we should only apply
> the depends patches when building releases, however then you've got a
> situation where release builds are even more divergent from
> "normal"/developer builds, and this means either maintaining an even
> more complex CI / testing routines, or they just end up less tested
> (guess which one is more likely to happen).
>
> If that all just sounds like a complicated mess, it's basically
> because it is. However the solution, using Guix, is actually pretty
> straight forward.
>
> When we are in complete control of our release environment, we can
> pick exactly the version of glibc we want to use (even at a per-HOST
> level), something that we definitely couldn't achieve, in any sort of
> straight forward fashion, if at all, using gitian.
>
> This is something we've done recently, in #22365. We are now building
> using glibc 2.27 for the RISC-V HOST, and using glibc 2.24 for all
> others, while maintaining runtime compatibility with glibc 2.17.
> You'll note this this is achieved without needing to use any of the
> work arounds from our glibc-back-compat code (#22405), or any of the
> other PRs / changes mentioned in the comment above.
>
> These are just two very practical benefits that using Guix provides
> (there are more), which ultimately all boil down to us being in much
> greater control of our release build environment. Something I am very
> happy about, and I think makes a lot of sense for a project like
> Bitcoin Core.
>
> The fact that there are also many people in the Guix space actively
> working on bootstrapability is just another big bonus, and even if
> that doesn't work exactly like some may want it too right now, it is
> rapidly being improved, month on month.
>
> Guix is also a much more likely pathway to fully bootstrapable Bitcoin
> Core builds that what gitian could ever provide.

dongcarl stated:

> I don't think we're going to run Guix inside Gitian for the releases,
> but personally I think people are free to do whatever they think make
> sense if they're committed to maintaining it

luke-jr added:

> You don't lose those Guix benefits by running it inside gitian; gitian just provides the isolation needed to avoid regressing on security


---

