| timeless | DocScrutinizer: yes. | 05:06 |
|---|---|---|
| timeless | Unfortunately, this is a basic flaw in the way Debian packages work. The binaries don't include enough information to capture how they were built | 05:10 |
| timeless | Romoxa (Oleg Romashin) most of the build optimizations and continued working on fennec for a while | 05:12 |
| enyc | timeless: hrrm... isn't that what '-dbg' and source packages are for? | 10:34 |
| timeless | no | 10:34 |
| enyc | timeless: reproducible-builds project being helpful, you can check you can rebuild he same checksum binary etc.? | 10:34 |
| timeless | they don't provide a way to guarantee a complete build env | 10:35 |
| timeless | enyc: the problem is that a binary package doesn't tell you which precise versions of dependent packages | 10:35 |
| timeless | it doesn't tell you which version of the compiler was used | 10:35 |
| timeless | or the linker | 10:35 |
| timeless | or... | 10:35 |
| enyc | timeless: aaat the build-log i see | 10:35 |
| timeless | yes, if you manage to get matching checksums, you know that you reproduced things | 10:35 |
| timeless | but, when you don't (which will happen 99.99% of the time, due to things that embed timecodes) | 10:36 |
| timeless | you don't know anything | 10:36 |
| enyc | timeless: surely its' expected to use tha same common pbuilder / debian release | 10:36 |
| enyc | timeless: thats part of why they use pbuilder, create new chroot with minimum deps for building each time | 10:36 |
| enyc | but then i understand it depends what 'updates' to a release were applied before/after a package was built | 10:37 |
| enyc | e.g. gcc with retpoline fixes -- has package been rebuilt-or-not, etc etc | 10:37 |
| timeless | enyc: anyway, the point is that debian packages do not encode enough info | 10:38 |
| timeless | if the package is from along time ago, (5-10 years), you're SOL if the build environment is long gone | 10:38 |
| enyc | have you made a propased patch to dpkg/tools etc.?, oloked into options for build logs/info to be stored eithre separately or embedded in the debs in a way that won't affect existing systems installing them...? | 10:39 |
| timeless | no | 10:39 |
| timeless | for one, i'm not sure it's entirely solvable | 10:39 |
| timeless | one could at least encode the version of installed packages | 10:39 |
| enyc | the pbuilder creates buildlogs | 10:40 |
| DocScrutinizer05 | embeeded timestamps, HAHA great. I guess a VM with patched time_t time(time_t *t) that simply returns a configurable constant might be useful | 10:40 |
| timeless | there's a faketime module, i use it to talk to openssl | 10:41 |
| enyc | well acutalyl reproducible-builds work to get rid of all build times etc. of course | 10:41 |
| timeless | enyc: anyway, build logs that are on a server that don't persist aren't helpful | 10:42 |
| Maxdamantus | I think I've noticed that particular packages in nix have workarounds for removing build times from things that store them. | 10:42 |
| timeless | remember, DocScrutinizer05 is asking about packages from 5-10 years ago | 10:42 |
| DocScrutinizer05 | actually jonwil is | 10:42 |
| timeless | the source servers in a number of cases are gone | 10:42 |
| timeless | heck, even the build env distros are gone | 10:43 |
| Maxdamantus | (and as a standard feature, file times are all set to `0` after building a derivation in the store) | 10:43 |
| timeless | (which scratchbox,...) | 10:43 |
| enyc | Maxdamantus: thats' to make the build reproducible ? | 10:43 |
| Maxdamantus | enyc: not exactly. | 10:44 |
| Maxdamantus | enyc: the build should be reproducible enough without that. | 10:44 |
| timeless | really, to some extent, anything that isn't part of the shipping .deb is probably a bad thing to rely on | 10:44 |
| timeless | it's pretty easy to not end up w/ the associated "dbg" packages in case no one used them and the repo got lost | 10:44 |
| enyc | well you say the problem isn't solvab/e, mut didn't rcatch what parts ooyou want solved and why, what the compatibilite problems would be, and how you do intend to solve those that should be solved. | 10:45 |
| timeless | anyway,... solvable? maybe, it'd probably require logging all installed package versions and checksums for all files | 10:45 |
| timeless | the question is ~ can one rebuild a build env in order to rebuild a package? | 10:46 |
| timeless | that's what's desired and in a number of cases desirable | 10:46 |
| Maxdamantus | That should be relatively doable in nix. | 10:46 |
| timeless | a fun version of this problem: | 10:46 |
| enyc | timeless: hrrm like i mentiont to leste people they shouldn't be expecting to reply on current githumb for contributors .. if you want to keep and reprouce/upstream fixes you need things like copyright file and clear status... hey-ho. | 10:46 |
| Maxdamantus | (though that's not relevant in this case) | 10:47 |
| timeless | a package X depends on a build-dep Y w/ some vague version requirement (not absolute) | 10:47 |
| timeless | in one version of Y, it defines a macro w/ one value, in another, it defines it with a different value | 10:47 |
| DocScrutinizer05 | could at least from now on make, cmake et al take care to include all info to the .deb per default? | 10:47 |
| Maxdamantus | What is "all info"? Every package on the build system? | 10:47 |
| Maxdamantus | or every package transitively reachable through the dependencies? | 10:48 |
| timeless | the resulting X package will have totally different shapes depending on the define, which is not something that you can tell by looking at the package | 10:48 |
| Maxdamantus | or just the dependencies? | 10:48 |
| DocScrutinizer05 | ouch | 10:48 |
| timeless | Maxdamantus: it would be safer if it did all packages, but for privacy reasons everyone would object to that | 10:49 |
| timeless | i think from a reproducible perspective, if your build env was able to record all packages that contributed to files that were opened during the build process | 10:49 |
| timeless | you'd get something vaguely good | 10:49 |
| DocScrutinizer05 | :nod: | 10:50 |
| DocScrutinizer05 | augias task | 10:50 |
| timeless | and again, what do you do if something is modifying how you see files | 10:51 |
| Maxdamantus | I'm not sure how it normally works on debian-based systems, though I think it hasn't worked this way with maemo, but: if everyone agrees that all official packages are built only by an automatic build server, it should be fairly reproducible. | 10:51 |
| timeless | Maxdamantus: i worked on maemo | 10:51 |
| timeless | the build servers more or less did things in parallel | 10:51 |
| Maxdamantus | since the build maintainers can just keep a git repository with all deb sources. | 10:51 |
| timeless | you could have races where different packages were built/used in parallel | 10:51 |
| timeless | such things were real problems | 10:51 |
| timeless | specifically, i tried to build a source code index of a given binary release | 10:52 |
| Maxdamantus | Heh, fun. | 10:52 |
| timeless | it wasn't really possible | 10:52 |
| timeless | because different binaries used different versions of dependent packages | 10:52 |
| * DocScrutinizer05 idly muses about keeping complete VMs in source 'repo' | 10:52 | |
| DocScrutinizer05 | one for each package - OUUUFF | 10:52 |
| timeless | DocScrutinizer05: real professional projects do that, more or less | 10:52 |
| timeless | not archiving the build env leads to problems like this | 10:53 |
| Maxdamantus | tbh, the complete VMs thing doesn't seem particularly unattainable. | 10:53 |
| Maxdamantus | If your storage mechanism involves deduplication, it should be fine. | 10:53 |
| Maxdamantus | eg, one obvious way of achieving that would be to fire off every build in an isolated snapshot of the filesystem. | 10:54 |
| Maxdamantus | then you just save the filesystem tree as a git tree in a bit repo. | 10:54 |
| timeless | Maxdamantus: i used dedup on the source system to avoid blowing up space w/ my weekly index | 10:54 |
| timeless | (i used zfs) | 10:54 |
| sparre | At one of my customers, we launch a VM on Amazon for every build-and-test run. | 10:55 |
| Maxdamantus | Yeah, I was thinking of the "snapshot" part in terms of btrfs, but zfs works too. | 10:55 |
| timeless | this was over a decade ago | 10:55 |
| * DocScrutinizer05 looked into dedup solutions a lot, for local workstation (btrfs, ext4) - it's a PITA and nothing _really_ worked | 10:55 | |
| timeless | in my case, since the files were generally predictably named, it was pretty easy | 10:55 |
| timeless | i built to a stage, got rid of the package versions from directory names | 10:56 |
| timeless | and then rsync'd over the previous week's files | 10:56 |
| timeless | and snapshotted | 10:56 |
| DocScrutinizer05 | yeah, that's somewhat sustainable and feasible | 10:56 |
| timeless | but, that didn't solve the stupid "x uses a@1; y uses a@2; z uses b@2 which uses a@?" | 10:57 |
| DocScrutinizer05 | :nod: | 10:57 |
| DocScrutinizer05 | the problem however feels like homegrown, the Godfathers of unix had a pretty clear notion about how to tackle this at very least on .SharedObject library level | 10:59 |
| timeless | sharedobjects are one things, but it doesn't cover macros | 10:59 |
| DocScrutinizer05 | "new" stuff should strictly follow same principles | 10:59 |
| DocScrutinizer05 | yeah, sure, that's why >>at least for...<< | 10:59 |
| timeless | anyway.... it sucked... and was very frustrating not being able to answer questions about what a binary was thinking | 11:01 |
| DocScrutinizer05 | coders would need to learn to *always* think fore/back-ward compatibility | 11:01 |
| timeless | grr. cat wants to go out | 11:02 |
| DocScrutinizer05 | but yeah, when you make a new (version of) package, you usually don't give a shit about backward and even less forward compatibility | 11:02 |
| * DocScrutinizer05 a few days ago looked into snap packages first time. Also sort of PITA, NFC where the apache running in snap env is keeping its vhost config | 11:18 | |
| DocScrutinizer05 | the damn nextcloud snap package even has a LetsEncrypt running, but I can't find how to enable apache HTTPS | 11:20 |
| timeless | is mod_md in hat thing? | 11:30 |
| jonwil | I think as far as build prerequisites go, the only option is to assume that if "configure" doesn't complain about stuff being out of date, its suitable to use. | 11:45 |
| Vajb | hehe interesting read of maemo builds back there | 22:24 |
| Vajb | last nite i wanted to prepare build enviroment for my upcoming android course... | 22:24 |
| Vajb | can admit that i don't like android studio | 22:25 |
Generated by irclog2html.py 2.17.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!