sadsnork | onefang, I was away from home all day but came back to find my initial rsync still running (currently at just under 21GB). I figured it was memory but "free" shows less than 20% memory used, and almost zero swap used. I tried the rsync without the "z" in case it was crappy CPU overhead but that changed nothing. | 02:46 |
---|---|---|
sadsnork | If I wget a large file from the VPS (just used a test file on cachefly.net) it does arrive at the full 100Mbit. The rsync does not appear broken, just slow... and I realize subsequent updates should obviously be much quicker overall. | 02:48 |
sadsnork | Just figured you should know, and that I should ask if you have seen that with anyone else before. | 02:48 |
gnarface | sadsnork: i can tell you that unexplained throughput issues to north america from the european mirrors have plagued the project from the beginning. i suspect active sabotage by someone with ISP level access, possibly in the guise of national security | 02:56 |
gnarface | escalated complaints to the colos have gone nowhere because the issues can't be reproduced locally and nobody higher up the chain is accountable to their customers directly | 02:57 |
gnarface | i dunno if that's directly related to what you're seeing right now or not but it's been common, if spurious | 02:59 |
sadsnork | WTF? I just tried to wget https://pkgmaster.devuan.org/devuan/dists/beowulf/main/installer-i386/current/images/netboot/mini.iso and as I sat watching the speed seemed to regularly fluctuate between about 120kB/s and 480kB/s. | 03:04 |
sadsnork | PS: Thanks for the reply gnarface... to be honest for a brief moment I was pissed and about to lash out at my VPS provider for screwing up my connection speed. | 03:05 |
gnarface | after spending a lot of time watching it bottom out to 0 then struggle to get back up to 8,000B/s (then bottom out to 0 again) i eventually set up a apt proxy. i understand that's not gonna help you though | 03:05 |
sadsnork | I just tried to get the same iso but changed to http instead of https and it SCREAMED for the first 80% of the transfer and then came grinding down to 383kB/s at the end. | 03:07 |
sadsnork | Then replaced "pkgmaster.devuan.org" with "sledjhamr.org/devuan" and the download completed in seconds with no slowdown at all. | 03:10 |
sadsnork | Is it seriously possible that someone is intentionally throttling traffic from pkgmaster.devuan.org to north america (or anywhere else for that matter)? | 03:11 |
sadsnork | Ridiculous, I tried wget'ing from a few different machines (on different north american networks) and they all have the same thing. :-( | 03:20 |
gnarface | sadsnork: as i recall, nobody that could actually be contacted about it at the colo was willing to acknowledge the possibility, but i don't see any other reasonable explanation for what i've seen | 03:34 |
gnarface | sadsnork: (though the possibility that it's someone above them, and they don't know is there... that's suggesting a government or at least a rogue government agent or employee is involved... but that'd be crazy... right?) | 03:35 |
sadsnork | Well I'd be willing to pull from a different server [though that sort of induces a double-copy delay in my updates], or provide my VPS as a resource to help figure out what is happening. | 03:37 |
gnarface | sadsnork: the best we could get was "must be on your end" but basically everyone on the continent was seeing the same problem | 03:37 |
gnarface | good hunting | 03:38 |
sadsnork | At first I [of course] thought it was rsync specific, but clearly it is also http and https traffic from that machine... and TO more than one destination. | 03:39 |
sadsnork | And if that machine provides full-speed downloads to the other various non-north-american servers then it does seem to be somewhere between that machine and here. That kind of stuff pisses me off. :-( | 03:40 |
gnarface | well that's a big if and from what i've seen it's more... temporal | 03:40 |
gnarface | sometimes it will and sometimes it won't | 03:40 |
sadsnork | Meaning "sometimes pkgmaster.devuan.org provides full speed downloads to non-north-american servers"? | 03:42 |
gnarface | yea, i think you can't even trust it to reliably suck | 03:43 |
sadsnork | Man, that is a bit of a mess. :-( | 03:47 |
sadsnork | Has there been any talk of potentially trying a different (possibly hidden) master? | 03:47 |
gnarface | good question. i'm not really on the inside loop so i don't know though. | 03:49 |
gnarface | you might be the first mirror on the continent too? | 03:50 |
sadsnork | There is uhhmmm beard.ly in the US I think | 03:51 |
gnarface | i agree it really has to be dealt with but it could actually get ugly if there's actually rogue government agents involved, screwing with it at the border routers | 03:52 |
gnarface | i can't imagine why they'd be doing that but it occurs to me that if it was happening to a lot of other traffic, it wouldn't be very likely that anyone else besides us would actually notice either | 03:52 |
sadsnork | Looks like there is also Berkley, but they are empty: http://mirrors.ocf.berkeley.edu/devuan/ | 03:52 |
adhoc | I just grabbed it to seew hat the speed is like; started off around 3 megabit/second, | 03:54 |
adhoc | 43.00M 85.7KB/s in 91s | 03:54 |
adhoc | dropped right down at the end to a few kilobyes/second | 03:55 |
gnarface | i lied, i can imagine why they'd be doing it, but speculating would pointlessly make me sound more paranoid | 03:55 |
sadsnork | Thanks adhoc, on the machines I tried it seemed to blast for 50% or 75% before dropping off but mine didn't go as low as 85k | 03:56 |
adhoc | but from home; 43.00M 2.60MB/s in 19s | 03:57 |
adhoc | I suspect the route has more congestion, to work | 03:57 |
adhoc | sadsnork: I had a similar experience on my work link, fast then tailed right off to almost nothing | 03:58 |
adhoc | home like started at 500KB/s got higher... | 03:58 |
sadsnork | Both home and work are north america? | 03:58 |
adhoc | I have also noticed a fair bit of RSO around the patching of OPENSSL at the moment | 03:58 |
adhoc | there is a remote expliot crash bug | 03:59 |
adhoc | so folks are probably burning the midnight oil to get things patched | 03:59 |
adhoc | and therefore less bandwidth avilable with outages | 03:59 |
adhoc | sadsnork: no, in Australia, but I am seeing similar issues | 03:59 |
adhoc | I just noticed, I am hitting differnt DNS targets | 04:00 |
adhoc | from home; 5.135.82.179 napier.devuan.org | 04:00 |
sadsnork | I had heard about the openssl thing earlier, though it would seem a bit odd if it would allow a download to start at top speed and then suddenly crash to almost nothing. | 04:01 |
adhoc | from work; 54.36.142.183 nash.devuan.org | 04:01 |
adhoc | oh, well no that is not what I would expect, | 04:01 |
sadsnork | I actually have a buddy in AU who says he has gigabit and can test for me when he gets home in a bit. | 04:01 |
adhoc | I expect network congestion while services are down for patching | 04:02 |
sadsnork | Hmmm, your DNS idea has me thinking... | 04:02 |
adhoc | sadsnork: so what are you pointing to? | 04:02 |
adhoc | I can't tell what the complete traceroute path is, many hops don't respond | 04:03 |
adhoc | napier seems to be in the EU | 04:04 |
adhoc | while nash appears to be in CONUS | 04:04 |
adhoc | 5.135.82.176 - 5.135.82.191 => address: 1013AK Amsterdam | 04:05 |
adhoc | 54.36.142.176 - 54.36.142.191 => address: 1013AK Amsterdam | 04:06 |
sadsnork | adhoc, I was on nash but made an /etc/hosts entry to force napier and restarted the rsync | 04:06 |
adhoc | sadsnork: you making a mirror? | 04:06 |
sadsnork | Yes... in Canada. | 04:06 |
sadsnork | OVH Montreal | 04:07 |
adhoc | using rsync ? | 04:07 |
sadsnork | Yes... I started the pull last night and today it was still running at 21GB. | 04:07 |
adhoc | it might pay to do a bit more research on the paths to the various mirrors around | 04:07 |
sadsnork | I have a dog to walk but am hopeful the numbers will be different when I get back. :-) | 04:08 |
adhoc | in the end, I had more success pulling from a mirror in Japan than anything in CONUS | 04:08 |
adhoc | if you are not already, get familiar with mtr, traceroute, tracepath and whois =) | 04:08 |
adhoc | I'd be happy to walk you through stuff if you get stuck =) | 04:09 |
adhoc | or need some comparison data | 04:09 |
rrq | afaik both nash and napier are in the Netherlands, but nash seems to have worse network | 04:18 |
rrq | more often DoS attacked as well | 04:19 |
Xenguy | adhoc, mtr largely replaces traceroute doesn't it? | 04:22 |
Xenguy | sadsnork, Is this an ISO mirror or package mirror? | 04:23 |
sadsnork | Xenguy, it is a package mirror. | 04:28 |
Xenguy | Glad to hear Canada is on the mirror map : -) | 04:29 |
sadsnork | When I nslook'ed up pkgmaster.devuan.org I got 54.36.142.183 and 5.135.82.179. Forcing my pull to come from the latter with a hosts file entry has bade a significant improvement in speed. | 04:30 |
Xenguy | I assume that once you have the first transfer done, subsequent rsync's will be much faster | 04:31 |
sadsnork | After a full day of rsyncing it had done about 20GB, and now in the 20 minutes I was walking the dog it has done almost another 7GB. | 04:31 |
rrq | sadsnork: pm | 04:32 |
Xenguy | But I don't really know much about the in's and out's of mirroring | 04:32 |
Xenguy | sadsnork, Nice | 04:32 |
sadsnork | So it does look like there is something up with 54.36.142.183 | 04:35 |
sadsnork | Good catch adhoc... I can see how DNS would make the results seem totally inconsistent. | 04:36 |
adhoc | sadsnork: grokking the infrastructure behind the service is key to understanding why it is misbehaving | 04:38 |
adhoc | Xenguy: learning a wide variety of the ttools available can lead to a better understanding | 04:39 |
adhoc | Xenguy: some times older tools seem more primative, however can yield different data | 04:40 |
sadsnork | I'm wondering if it could also be something on 54.36.142.183 itself. Possibly CPU capping out as it tries to compress all the rsync's for the various mirrors that are pulling at half hour intervals. | 04:40 |
adhoc | this is why I like both traceroute and tracepath | 04:40 |
adhoc | mtr is often the start of any investigation about what is goind on | 04:40 |
adhoc | but when you need to copy and paste the data, the older tools are better | 04:41 |
adhoc | sadsnork: without the monitoring data of that host, its network and infrastructure of the provider/host, we are only guessing | 04:41 |
adhoc | one end path seems more congested than the other, bursty traffic like that is hard to diagnose | 04:42 |
rrq | yes the networking for nash isn't too flash | 04:47 |
golinux | Changing the subject . . . have finished recoloring the chimaera icons: https://dev1galaxy.org/files/chimaera-icons.png | 04:48 |
Xenguy | adhoc, fair enough | 04:51 |
Xenguy | Congrats golinux | 04:52 |
golinux | Just now rearranging them sinto side by side so not so long | 04:53 |
golinux | Thanks. | 04:53 |
golinux | They look quite nice on the desktop | 04:53 |
golinux | Xenguy: Refresh the image. That's better | 04:58 |
sadsnork | The bad news: after about 10GB of great speeds, 5.135.82.179 is doing the same thing. | 05:15 |
* sadsnork goes to bed and will have another look in the morning. | 05:16 | |
adhoc | =) | 05:16 |
adhoc | need more data to work out what is going on =) | 05:16 |
snyper | hi guys/gals any way for wine to be added to repository ? | 06:53 |
snyper | dont know where else but i suggest you add command-not-found installed by default, i mean, why not ;) just convenient | 06:56 |
gnarface | snyper: it should be in the repo already, isn't it? | 07:05 |
gnarface | wine, that is | 07:06 |
snyper | well when i type wine in console with (command-not-found) installed, it doesnt find it | 07:06 |
snyper | maybe i did something wrong ? | 07:06 |
gnarface | maybe | 07:06 |
gnarface | could be your sources.list | 07:07 |
gnarface | i'm not familiar with "command-not-found" though | 07:07 |
snyper | is there a place i can go to suggest packages to add ? ex: rtl_433 etc | 07:07 |
snyper | ah ok, ill have a look, brb | 07:07 |
snyper | ok my bad, it shows up in synaptic, i usually cli it :P | 07:09 |
snyper | there must be something i missed about command-not-found, some sort of refresh or whatever | 07:09 |
gnarface | i dunno, i just use apt-get | 07:09 |
gnarface | the chances of getting a package added to the main distro that's not in debian are slim right now | 07:10 |
gnarface | but you might be able to get something put into proposed-updates | 07:10 |
snyper | thanks for the info gnarface , i guess i just got lazy with all the automation of the other distros xD | 07:10 |
snyper | ok what is this "proposed-updates" ? | 07:10 |
gnarface | basically packages they would have put in, if it didn't violate the whole not diverging from debian unnecessarily thing | 07:12 |
gnarface | i think adding it works just like adding beowulf-backports but it's called beowulf-proposed-updates | 07:12 |
snyper | yea, i might be controversal saying but imo they didnt need to deepthroat us all with systemd .... | 07:12 |
gnarface | it's not controversial around here but we've designated #devuan-offtopic for ranting | 07:13 |
gnarface | this is intended to be a support channel | 07:13 |
snyper | :D | 07:13 |
gnarface | i think you can get more direct attention about your package in #devuan-dev but you'll run into team members in every channel, they're just slow channels so you have to be patient | 07:13 |
gnarface | there's also a forum and mailing list | 07:14 |
snyper | ok cool, well thanks for the info, ill do some reading up on the proposed-updates, i never heard about it befor | 07:14 |
gnarface | it might just be called "beowulf-proposed" | 07:14 |
gnarface | i'm not sure | 07:14 |
snyper | ok, well keep up the good work guys, ive ben raging here in person for a while about the way linux has been taken over from the inside ://// gates on the linux board ... omg hehe | 07:15 |
gnarface | https://www.devuan.org/os/packages | 07:15 |
gnarface | here's some mor einfo on the sources | 07:15 |
gnarface | more info | 07:15 |
snyper | thanks again ;) | 07:16 |
gnarface | no problem | 07:16 |
onefang | Damn, I missed the pkgmaster conversation. | 08:35 |
onefang | sadsnork: I have no problem with you using sledjhamr.org for the initial rsync, though you are correct that later rsyncs should be with pkgmaster. BTW, sledjhamr.org is my server. | 09:02 |
onefang | #devuan-dev is probably a better place for these mirror admin discussions. | 09:02 |
aah5 | Can I use devuan with traditional BSD scripts (aka slackware like) instead of SysV or runit? | 12:36 |
crashoverride | That's a good question; fsmithred? | 12:37 |
crashoverride | I'd say: probably, but you might have to work for it. | 12:37 |
aah5 | Work for it? Like how? | 12:38 |
gnarface | minor path edits? | 12:38 |
aah5 | I guess I'll give it a try in a VM. | 12:38 |
gnarface | eh, i dunno how much work it would be honestly but i'm sure it's possible | 12:38 |
fsmithred | I don't know | 12:39 |
fsmithred | packages install files to pre-determined directories. If you move stuff, it's up to you to keep everything in order. | 12:39 |
fsmithred | why? | 12:40 |
aah5 | I don't want to move stuff around. Just use a different init system. | 12:40 |
fsmithred | sysvinit, openrc, runit are the main choices | 12:40 |
crashoverride | aah5: well, if you use a different init system, you have to move stuff around. | 12:40 |
crashoverride | fsmithred: have you ever tought of supporting s6? | 12:40 |
fsmithred | people are working on it, but it's moving slowly | 12:41 |
crashoverride | ok | 12:41 |
crashoverride | I might contribute to that. | 12:41 |
crashoverride | first I have to report my first bug, already. | 12:41 |
fsmithred | I've been playing with runit a bit | 12:41 |
crashoverride | v_v | 12:41 |
crashoverride | it's been a year since I spotted it. | 12:41 |
aah5 | If I can get bsd scripts to work with devuan, would you consider supporting it first class? | 12:41 |
crashoverride | and I'm postponing like there's too many tomorrows | 12:41 |
fsmithred | aah5, maybe if someone volunteers to do it and maintain it | 12:42 |
crashoverride | aah5: if it's done properly, probably. | 12:42 |
aah5 | SysV scripts get stored in rc.d right? | 12:45 |
lts | Usually /etc/init.d | 12:51 |
aah5 | Well the good news is that since slackware 7.0, their bsd style init has support for sysv init scripts. | 12:51 |
djph | ... I really have to get better at init scripts :( | 13:13 |
orcus-de | yup - same here .. rusty with it djph | 13:13 |
orcus-de | guess I got a reason now to autoconnect djph | 13:14 |
crashoverride | orcus-de: what's that? | 13:15 |
orcus-de | crashoverride: I noticed a friend is interested in devuan too ... | 13:16 |
orcus-de | and there are more coming in seemingly :-) | 13:16 |
orcus-de | \o SimonNL | 13:17 |
SimonNL | just checking | 13:17 |
crashoverride | orcus-de: ah :) | 13:17 |
orcus-de | crashoverride: looking for a replacement for centos ... and as systemd is my most "beloved" thing makes devuan a good candidate | 13:19 |
* orcus-de stopping off-topic chat now :-) | 13:19 | |
SimonNL | hasta leugo | 13:19 |
crashoverride | orcus-de: devuan is great; but if you're used to centos, I must warn you: devuan just works. | 13:20 |
crashoverride | orcus-de: so if you bill your customers for fixing broken things, now you'll have to find something else to bill for. | 13:20 |
orcus-de | oh centos did too - but I'm most of the time using debian based linux anyways .. and I often got stuck try using apt update or running update-grub at the wrong terminal anyways :-) ... so reducing platform would ease life a bit more | 13:22 |
crashoverride | orcus-de: whenever I was using centos, it did cause me a lot of pain | 13:26 |
orcus-de | it just depends on what you are using (or not) - I had little headaches there - but I'm writing most of my services I need on my own anyways .. it did run painless for me most of the time ... | 13:30 |
orcus-de | but centos will be history some time this year (at least for me) ... | 13:31 |
crashoverride | orcus-de: I used it for work, so, various things. | 13:47 |
crashoverride | orcus-de: but yeah, not fit for work in my experience. | 13:48 |
crashoverride | poor package quality, poor package toolchain, poor base software quality, and sub-standard experience in general. | 13:48 |
crashoverride | it's often that you upgrade something only for it to break | 13:49 |
orcus-de | crashoverride: :-) .. agree on that - if you have to adjust it more often, it can be "entertaining" .. my use case has been likely way more independent and stable | 13:49 |
crashoverride | nah but that's what I mean | 13:49 |
crashoverride | s/mean/&t/ | 13:49 |
crashoverride | with centos, you can definitely bill more for drinking with your buddies while your automated system fixes the problems because it's constantly the same blunders made upstream and so, something as simple as an automated script can deal with them. | 13:50 |
crashoverride | and if you don't have that script, you would work for a couple hours on that. | 13:50 |
crashoverride | so you bill 2 hours for running a script and having laughs while drinking sternie | 13:50 |
crashoverride | with devuan, tho, that's a different story: maintainers don't assume things never changed since installation on your system, and the package quality is thus higher. | 13:51 |
orcus-de | mine is more like an old vw beatle .. rarely to fix anything there after setup .. it just runs and consumes gas/energy // setting up centos without using scripts to set it up is stupid anyways .. I prefer to adjust the script if needed and then run it for deploy | 13:52 |
crashoverride | I feel like most distros just implement QA by installing a stock install of their software, and running their upgrade on that. | 13:52 |
crashoverride | which is silly at best (assuming the people are clueless and of average intelligence) | 13:52 |
djph | crashoverride: people are clueless | 13:53 |
orcus-de | ... heading out to see my wife - has been a nice talk ... will likely hang around more often in the future cheers crashoverride | 13:53 |
crashoverride | and downright despicable at worst (assuming the people have experience and of sufficient intelligence to obtain an IT degree) | 13:53 |
crashoverride | bis dann orcus-de | 13:53 |
crashoverride | viel spass | 13:53 |
crashoverride | djph: yeah, I know, but still. | 13:53 |
orcus-de | dir auch / diese deutschen sind einfach die pest :-) man wird sie einfach nicht los | 13:54 |
crashoverride | :D | 13:54 |
msiism | Is there a half-way convenient way to list all packages installed from a particular Apt repository in the terminal? | 19:29 |
msiism | I just recovered my system from a broken state after trying to install TDE, and I'd like to purge any left-overs. | 19:30 |
furrymcgee | /var/lib/apt/lists/? | 19:32 |
msiism | I've been through that successfully, as far as I can judge. | 19:33 |
msiism | I mean, I grpped all packages names from there, put them in a file, read that file and looked for every one of those things being installed. | 19:34 |
msiism | s/grpped/grep-ed | 19:34 |
amesser | dpkg -l | grep '^rc' | 19:42 |
amesser | arg sry | 19:43 |
msiism | Well, seems like everything's been cleaned out. | 19:47 |
xrogaan | msiism: have you solved your issue? | 20:05 |
msiism | xrogaan: I guess so. | 20:06 |
msiism | I mean, I'm going to try to get TDE installed right at this point. It's a bit too much of a headache for me. | 20:08 |
msiism | Seems there is some elogind-related screw-up that causes the trouble. | 20:08 |
xrogaan | To list the packets: something like this? grep -h -P -o "^Package: \K.*" /var/lib/apt/lists/deb.devuan.org_merged_dists_beowulf_*_Packages | sort -u | xargs dpkg -l | 20:08 |
msiism | Yeah, that's sort of what I did. | 20:09 |
furrymcgee | dctrl-tools or recutils read (almost) deb822 formatted files | 20:09 |
msiism | Except using a file to store the grep results and then going with that. | 20:10 |
msiism | There really should a way to do this with apt-cache somehow. | 20:10 |
xrogaan | no, I think this is the right way | 20:11 |
msiism | Like, apt-cache search --origin <repo> | 20:11 |
msiism | I mean "should" as in "should be implemented". | 20:11 |
msiism | Synaptic has search by "origin", I hear. | 20:11 |
msiism | Pretty neat. | 20:12 |
msiism | xrogaan: I mean, that command you gave me above seems like a moderately crazy way to get those results, if you ask me. | 20:14 |
msiism | I mean, it seems correct and all that. It's just horrid interface-wise. | 20:14 |
* msiism remebers the truth about Unix: "The user interface is horrid." ;) | 20:15 | |
furrymcgee | in my opinion its a good think debian packages dont store origin | 20:16 |
xrogaan | apt simply installs the packages based on your policies. | 20:18 |
xrogaan | It's a dependency manager, nothing more. | 20:18 |
msiism | furrymcgee: I think that information is immensely useful. | 20:18 |
furrymcgee | that would be immensely confusing all people like you | 20:20 |
xrogaan | aptitude search '?origin (reponame) ?installed' | 20:20 |
msiism | furrymcgee: I don't get what you mean, but I think I disagree. | 20:20 |
msiism | xrogaan: Okay, nice. | 20:20 |
xinomilo | dpkg --get-selections | 20:22 |
xrogaan | I don't know what is a valid repository name though | 20:24 |
msiism | I think sources.list(5) describes that. | 20:25 |
xrogaan | It's the origin though | 20:29 |
xrogaan | brb | 20:29 |
xrogaan | I think it depends on the Release file | 20:44 |
xrogaan | Origin: Devuan | 20:44 |
xrogaan | for deb.devuan.org_merged_dists_beowulf_InRelease | 20:45 |
msiism | Okay, I see. | 20:45 |
xrogaan | so, you need to get the `Origin: ` value from your target repository | 20:46 |
xrogaan | It accepts partial text, for example (devua) will work | 20:47 |
msiism | Hm… I'd be more interested in the actual repository, though. | 20:48 |
xrogaan | if the origin is unique, then that's it. | 20:50 |
msiism | Okay, true. | 20:52 |
xrogaan | repo are sometimes broken into multiple "pseudo" repo, like updates, proposed updates and such. | 20:55 |
xrogaan | Those serves as channel from the developer side, not designed to be proper standalone repositories. | 20:56 |
msiism | Okay, that too, yeah. | 21:09 |
Generated by irclog2html.py 2.17.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!