DocScrutinizer05 | folks. a coredump can be done with the process keeping running, right? | 07:33 |
---|---|---|
KotCzarny | yes, with gdb | 07:41 |
KotCzarny | (and probably other tools) | 07:42 |
DocScrutinizer05 | yeah, but obviously not with standard signal means | 07:50 |
DocScrutinizer05 | see man 2 signal and man 5 core | 07:50 |
DocScrutinizer05 | both say coredump implies process termination | 07:51 |
DocScrutinizer05 | well, to avoid falling for XY problem: infobot stops for a >2h each night at 00:20 UTC, and neither Tim (owner) nor I have any idea what's causing this. How would you try to tackle this shit? | 07:53 |
KotCzarny | it's not sigcore, but just a type of signals | 07:53 |
KotCzarny | ie. sigquit is core too | 07:53 |
DocScrutinizer05 | I know | 07:54 |
KotCzarny | as for the bot, check cron? | 07:54 |
KotCzarny | or scripts the bot is using, might be some log managing | 07:54 |
KotCzarny | or db rehash | 07:54 |
DocScrutinizer05 | I suggested this to Tim a dozen times already. Even suspected logrotate the culprit. No reply by Tim | 07:55 |
KotCzarny | do you have access to the box? | 07:55 |
DocScrutinizer05 | I need a commandline I demand him doing | 07:55 |
DocScrutinizer05 | no | 07:55 |
KotCzarny | some monitoring then? | 07:55 |
DocScrutinizer05 | now we're talking. What exactly= | 07:56 |
DocScrutinizer05 | ? | 07:56 |
KotCzarny | disk/network usage | 07:56 |
DocScrutinizer05 | meh | 07:56 |
KotCzarny | ie. bot uploading something big | 07:56 |
DocScrutinizer05 | nothing too unusual | 07:56 |
KotCzarny | if it's a bot stoppping responding, it might be some script running and blocking everything else | 07:57 |
KotCzarny | which would mean turning that script to run in parallel might help | 07:57 |
DocScrutinizer05 | to start with, I have no clear indication *why* the bot goes offline | 07:57 |
DocScrutinizer05 | I can remote-reboot the bot and already talked Tim into this reboot providing some diagnostic output. Alas I didn't find the one smart command to show what I need to know, about system status | 07:59 |
KotCzarny | it might be as simple as some script blocking the bot | 08:01 |
KotCzarny | which obviously wouldnt show in most diagnostics | 08:01 |
DocScrutinizer05 | ps doesn't show anything suspicious | 08:01 |
KotCzarny | easiest would be adding some debug log showing what funcs are starting and when | 08:02 |
KotCzarny | and popping that single print in most funcs | 08:02 |
DocScrutinizer05 | would seem like the bot goes OOM | 08:11 |
KotCzarny | which would sound as some script trying to run big data | 08:12 |
DocScrutinizer05 | well yeah. I could speculate all day long. I have no clue what to talk Tim into, to *fix* the issue | 08:14 |
DocScrutinizer05 | ~seen rimriker | 08:15 |
infobot | DocScrutinizer05: i haven't seen 'rimriker' | 08:15 |
DocScrutinizer05 | ~seen timriker | 08:15 |
infobot | timriker <~TimRiker@bzflag/projectlead/TimRiker> was last seen on IRC in channel #infobot, 25d 10h 35m 30s ago, saying: 'added a netstat -ntp|grep perl'. | 08:15 |
KotCzarny | clearly he has no time to fix it, and no one else has the access, which basically boils down to nothing is going to be done | 08:15 |
DocScrutinizer05 | that's a pretty useless revelation | 08:16 |
KotCzarny | but clearly directs to the deeper issue | 08:16 |
DocScrutinizer05 | so??? | 08:16 |
KotCzarny | you can speculate all day long, without results | 08:16 |
DocScrutinizer05 | so??? | 08:16 |
DocScrutinizer05 | do you want to tell me you're short of ideas how to tackle this? Noted | 08:17 |
DocScrutinizer05 | I'm too, right now | 08:18 |
KotCzarny | i've already shared few ideas, but without access they are unusable | 08:18 |
DocScrutinizer05 | I have 'access', just not the type of access you might think of when you hear the term | 08:19 |
DocScrutinizer05 | >>saying: 'added a netstat -ntp|grep perl'<< is what results from my type of "access", I demanded for a netstat to rule out uplink clogging | 08:20 |
KotCzarny | nah, it more sounds as some homecleaning script running in the bot | 08:21 |
DocScrutinizer05 | and I can trigger this and all the other diagnostic rools by doing a bot reboot | 08:21 |
KotCzarny | which takes a while, doesnt hit big cpu/network, but still blocks further execution | 08:21 |
DocScrutinizer05 | tools* | 08:21 |
DocScrutinizer05 | please check this https://pastebin.com/k2eACYn6 it has some errors that might well be worth a closer look and inverstigation. alas the bot is perl and thus I'm totally lost. /join #infobot for sourcecode etc | 08:27 |
DocScrutinizer05 | I gonna provide a reboot log based on operational state now | 08:38 |
DocScrutinizer05 | ~+status | 08:38 |
infobot | Since Sun Sep 30 02:25:36 2018, there have been 0 modifications, 0 questions, 0 dunnos, 0 morons and 0 commands. I have been awake for 4h 13m 4s this session, and currently reference 119769 factoids. I'm using about 80728 kB of memory. With 0 active forks. Process time user/system 12.89/0.57 child 0/0 | 08:38 |
DocScrutinizer05 | ~wb | 08:41 |
DocScrutinizer05 | oops | 08:41 |
DocScrutinizer05 | https://pastebin.com/cssbAu0W | 08:41 |
DocScrutinizer05 | gosh, does that PoS critter hang on reboot now? | 08:43 |
KotCzarny | chat.freenode.net should be irc.freenode.com ? | 08:46 |
KotCzarny | check shell ulimits | 08:48 |
KotCzarny | ulimit -a | 08:48 |
DocScrutinizer05 | well, when I want to tell Tim "please add a diagnostics line to check ulinit -a", I need a rationale WHY I think this is a relevant info, what we expect it to be, and what we hope to fix by messing with it | 08:50 |
KotCzarny | Out of memory! | 08:51 |
KotCzarny | this maybe | 08:51 |
KotCzarny | would be interesting to find that string exact in scripts | 08:51 |
DocScrutinizer05 | you seen the timestamps of those loglines in ibot's log? | 08:51 |
KotCzarny | could it could pinpoint the place where it does that big job | 08:52 |
KotCzarny | (although it might be generic perl err msg) | 08:52 |
KotCzarny | timestamps or not, it probably has more than one problem | 08:53 |
DocScrutinizer05 | it is | 08:53 |
DocScrutinizer05 | fsckng bot hangs on reboot!!! >:-( | 08:55 |
KotCzarny | perlmonks link to debug things: https://www.perlmonks.org/?node_id=598685 | 08:55 |
KotCzarny | also suggests checking ulimit | 08:55 |
DocScrutinizer05 | which actually migh be a GOOD thing, catching it while it does whatever it does when it takes 2h+ to re-login | 08:56 |
DocScrutinizer05 | ooh, no, that been me now, should stop messing with bot when tired | 08:56 |
KotCzarny | maybe it has some task to do which it tries to do for months but dies by killing | 08:57 |
DocScrutinizer05 | [2018-09-30 08:44:58] --> infobot (ibot@208.53.50.136) has joined this channel. | 08:57 |
DocScrutinizer05 | [2018-09-30 08:44:58] *** ChanServ sets mode: +v infobot | 08:57 |
DocScrutinizer05 | ~wb | 08:58 |
infobot | It's great to be back! | 08:58 |
DocScrutinizer05 | ~+status | 08:58 |
infobot | Since Sun Sep 30 06:56:10 2018, there have been 0 modifications, 1 question, 0 dunnos, 0 morons and 0 commands. I have been awake for 2m 11s this session, and currently reference 119769 factoids. I'm using about 80264 kB of memory. With 0 active forks. Process time user/system 1.29/0.12 child 0/0 | 08:58 |
DocScrutinizer05 | ~+uptime | 08:58 |
infobot | - Uptime for apt - | 08:58 |
infobot | Now: 2m 24s running infobot 1.5.4 (SVN) -- linux | 08:58 |
infobot | 1: 59d 8h 41m 19s running infobot 1.5.4 (SVN) -- linux, ended Sun Nov 14 18:39:57 2010 | 08:58 |
infobot | 2: 57d 3h 9m 23s running infobot 1.5.4 (SVN) -- linux, ended Fri Jun 26 20:39:27 2009 | 08:58 |
infobot | 3: 53d 8h 24m 4s running infobot 1.5.4 (SVN) -- linux, ended Tue Nov 7 05:34:20 2017 | 08:58 |
KotCzarny | random crap to debug memory in perl: http://perldoc.perl.org/perldebguts.html#Debugging-Perl-memory-usage | 09:00 |
KotCzarny | might actually be useful if you get a coredump and want to step through it | 09:01 |
KotCzarny | http://perldoc.perl.org/perldebguts.html#Debugging-Perl-Memory-Usage (proper link) | 09:02 |
DocScrutinizer05 | http://laforge.gnumonks.org/blog/20180929-fernvale-discount/ | 09:02 |
KotCzarny | 2g is being killed of around the world | 09:03 |
KotCzarny | might be the reason of lack of interest | 09:03 |
KotCzarny | *off | 09:04 |
DocScrutinizer05 | yep, sure | 09:23 |
DocScrutinizer05 | yet there are amazing things going on, like Michaela Spacefalconia trying to re-release a (Openmoko Freerunner) TI Calypso chipset based 2G modem module | 09:25 |
DocScrutinizer05 | I for one think ^^^ this stuff is more rewarding than Calypso | 09:27 |
sicelo | stupid question - but i must ask nonethelese - the telco in my country uses overhead lines, and our area is also prone to lightning strikes in summer. what is the best way (besides disconnecting, of course) to protect the adsl modem/router from this? | 21:31 |
KotCzarny | nothing protects you from direct (or even in neighbourhood) lightning hit | 21:33 |
KotCzarny | my friend had such incident few years ago | 21:33 |
KotCzarny | lightning hit the house 2-3 buildings away | 21:33 |
KotCzarny | his modem way flying across the room leaving sparks and burns trail | 21:34 |
KotCzarny | s/way/was | 21:35 |
sicelo | gosh | 21:37 |
sicelo | at least no one's reported such here | 21:38 |
KotCzarny | i have heard few stories about ethernet networks hung around the neighbourhoods too, those tend to burn routers/switches first | 21:38 |
KotCzarny | so just dont worry and keep your data safe | 21:39 |
KotCzarny | ie. store backups in different locations | 21:39 |
sicelo | well our modems dead now. no idea when isp will provide replacement. | 21:44 |
KotCzarny | there are some telephone line isolators, but as i've said, nothing protects from close hit | 21:45 |
KotCzarny | some power strips provide filtering for modem lines | 21:45 |
sicelo | isp doesn't like those. :) | 21:46 |
KotCzarny | they just have to pass high freqs, while still protecting from voltage spikes | 21:46 |
sicelo | they complained that it causes the signal to be 'sent back' to their more expensive equipment :D | 21:46 |
KotCzarny | rrrright | 21:46 |
sicelo | i never got time to study how they really work to see if their claim was correct (i doubt) | 21:47 |
KotCzarny | that might be true for some cheapo ones | 21:50 |
sicelo | guess we must be sure to disconnect then, assuming storm happens when we're around | 22:04 |
Generated by irclog2html.py 2.17.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!