Staff Meeting 21 August 2014

Log of staff meeting held on IRC in channel #Staff on 2014-08-21 from 01:06 to 02:52 UTC:

[01:06:13] * juggs motions 1st point of agenda for staff meeting 1. Staff meetings should not tag on board meetings. No-one likes to be summoned to a meeting 2 hours early.
[01:07:15] <NCommander> paulej72: no ;.;
[01:07:20] * NCommander needs food at some point
[01:07:24] <NCommander> I think we're ready to go
[01:07:36] <NCommander> juggs: the point been made
[01:07:40] <NCommander> And will be honored
[01:08:17] <Bytram> juggs: already agreed. see: <a href="http://logs.sylnt.us/%23staff/2014-08-21.html#00:56:51" target="_blank">http://logs.sylnt.us</a>
[01:08:28] -!- LaminatorX [LaminatorX!~470ab0b6@23-00-716-777.dhcp.stls.mo.charter.com] has joined #staff
[01:08:33] <juggs> who's chairing, who's minuting, what's on the agenda?
[01:08:58] * NCommander will chair
[01:09:02] <NCommander> volunteers for minuting?
[01:09:05] * LaminatorX will be back once he puts his kids to bed.
[01:09:09] <Bytram> I'd be happy to chair, but I don;tknow what's on the agenda
[01:09:15] <Bytram> :/
[01:09:26] <juggs> I don't think we have an agenda
[01:09:56] <Bytram> if I have an agenda to work from, I'll chair and minute.
[01:10:13] <NCommander> I don't think we have an agenda, I was planning to just go to everyone and ask what they wanted to discuss
[01:10:28] <TheMightyBuzzard> Bytram, just minute.
[01:10:40] <Bytram> TheMightyBuzzard: okay.
[01:10:47] <NCommander> So, lets go right down the list
[01:10:54] <NCommander> audioguy: floor is yours, anything to bring up?
[01:10:58] <audioguy> Agenda item: further discusssion of the pressing issue of stock, with emphasis on looking for specific solutions to contribute to the board
[01:11:13] * TheMightyBuzzard kicks audioguy
[01:11:25] <audioguy> Or has everyone had enough?
[01:11:29] <Bytram> $buritto audioguy
[01:11:59] * NCommander whacks audioguy
[01:12:04] * matt_ is ready for more =)
[01:12:05] <NCommander> Its a fair point though
[01:12:07] <TheMightyBuzzard> audioguy, is a meeting about it tomorrow. might as well save it until then.
[01:12:24] <audioguy> OK, I guess never mind that :-)
[01:12:24] * NCommander loves long discussions, was great in debate class <g>
[01:12:45] <paulej72> NCommander: you a master debater
[01:13:01] <NCommander> paulej72: also known as being stubborn
[01:13:05] <juggs> Agenda item: Team leads who are long time absent from position. It should be up to team leads to bring matters to staff meetings, that is not possible with absent team leads.
[01:13:22] <NCommander> juggs: will bring that up after going through everyone
[01:13:26] <audioguy> All I really have is some stuff related to possible virtual server changes, which are probably best left to a dev meeting, in the imnterest not not boring others to death
[01:13:27] <NCommander> audioguy: if nothing else, I'm moving on
[01:13:32] <NCommander> Ok
[01:13:35] <paulej72> I am the team leader for most of the groups
[01:13:39] <NCommander> Azrael: anything to bring up?
[01:13:46] -!- LaminatorX has quit [Quit: Web client closed]
[01:13:55] <juggs> paulej72, yep that makes sense.
[01:14:29] <audioguy> One possibbility - the situation with abuse of unicode.
[01:14:46] <audioguy> (cross teams )
[01:15:27] <Bytram> audioguy: good point; i've seen several (as in maybe a dozen or two) such posts... which were downmodded to oblivion =)
[01:15:29] <TheMightyBuzzard> audioguy, i think pj mostly handled that already. no more vertical overflow.
[01:15:54] * NCommander mulls
[01:15:56] <TheMightyBuzzard> it now takes up no more space than the same # of lines of ascii text would
[01:15:58] <NCommander> Maybe we shold go by team
[01:16:01] <NCommander> It will be more effective
[01:16:09] <NCommander> But lets bring up the UTF-8 stuff while we're here
[01:16:22] <Bytram> audioguy: fwiw, I tried to go 'outside' the comment box with &rtl; and &ltm; and variants and could not do so on dev.soylentnews.org
[01:16:32] <audioguy> So the plan is justy let them be modded down? I am a little woried about spammers posting ads in foeigjn languages and linking to them in emails, that sort of thing
[01:16:33] <NCommander> There's bene an uptick in downmods since the update landed
[01:16:36] * Bytram hasn't tried it on production
[01:17:08] <NCommander> audioguy: I'm open to ideas to preventing more abuse on this front. I don't want to loose the feature, especially since its long been a sticking point at the other site
[01:17:16] <audioguy> I assume javascript is already not permitted.
[01:17:40] <NCommander> audioguy: the javascript rule is that it has to fallback sanely
[01:17:43] <Bytram> audioguy: I tried it a few times, could not get it past the filters... TheMightyBuzzard -- what say you?
[01:17:44] <TheMightyBuzzard> audioguy, permitted... well you can put some in but it will just show as source.
[01:17:46] <paulej72> slash has a white list of tags
[01:18:08] <paulej72> <script> will get removed
[01:18:21] <Bytram> audioguy: are you talking about javascript in URLs?
[01:18:31] <audioguy> I don't want to lose it either, just trying to fimd out if we need to deal with this new world better. Also wondering: what if people enage in exteneded conversations in a foeign language, is this an 'English' site
[01:18:44] <audioguy> or in the text
[01:18:53] <Bytram> audioguy: e.g.: <a href="jaascript: blah blah blah">stuf<a/>
[01:19:14] <audioguy> What if jihadis start postimng stuff in rabic that is extremely imflammatory, etc.
[01:19:17] <TheMightyBuzzard> Bytram, links are stripped of script tags
[01:19:20] <Bytram> well, you get the idea; I could not get that kind of stuff past the filters.
[01:19:22] <NCommander> audioguy: well, at some point, I want to see user-created nexuses/ ala subreddits
[01:19:27] <NCommander> Which should allow non-English
[01:20:03] <Bytram> audioguy: "The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way. "
[01:20:17] <Bytram> appears following *ever* story...
[01:20:18] <audioguy> Some of these characters can also be used to create misleading links htt alook ok but actually go somewhere else - are links forced to 7 bit?
[01:20:32] <NCommander> audioguy: moot point, we now have unicode URLs
[01:20:33] <Bytram> as for the stories themselves, i think the Ediors can handle that. =)
[01:20:35] <paulej72> audioguy: that one we have to fix
[01:20:44] <TheMightyBuzzard> audioguy, idn links are currently broken. nothing but ascii gets through.
[01:20:51] <Bytram> audioguy: ^^^
[01:21:01] <TheMightyBuzzard> so once we fix it that can become an issue but not yet
[01:21:04] <Bytram> idn == International Domain Names
[01:21:09] <audioguy> I think that is GOOD, ascii only. (for links)
[01:21:26] <NCommander> audioguy: except that its perfectly valid now to have non-ascii URLs -_-;
[01:21:26] <TheMightyBuzzard> s'actually not. i'd like the code to support IDNs
[01:21:34] <Bytram> atm, yes, but will be a problem down the line. Whole rat's nets of issues to watch for.
[01:21:34] <paulej72> I think we will have an indicator that a link contains urf
[01:21:39] * NCommander would love to see the ability to load the UI in other languages
[01:21:52] <audioguy> Few in use yet though.
[01:21:57] <TheMightyBuzzard> paulej72, not yet but it was proposed.
[01:22:15] <paulej72> TheMightyBuzzard: “will have”
[01:22:26] <Bytram> at this point, I just want to take a moment to publicly thank TheMightyBuzzard for all his hard work blugeoning slashcode into handling UTF-8 characters.
[01:22:29] <TheMightyBuzzard> stupid eyeballs, betraying me again
[01:22:41] <NCommander> audioguy: point taken
[01:23:03] <TheMightyBuzzard> Bytram, all part of my master plan to hurry up meta-moderation
[01:23:08] <audioguy> HELL YES! Good going, of mighty bird!
[01:23:09] <Bytram> lOL!
[01:23:12] <NCommander> Right now, lets just keep an eye on it. The new CSS fixes shold prevent the worst of the abuse, and if it gets even worse, then we take more drastic steps
[01:23:13] <NCommander> k?
[01:23:24] <TheMightyBuzzard> sounds good
[01:23:26] <paulej72> NCommander: that would be really hard to do, but not impossible. You would need to pull out all of the text in the templates an put them into lookup templates
[01:23:41] <NCommander> paulej72: nah, just plug gettext into template tool
[01:23:49] <NCommander> Pretty sure thats already supported
[01:23:50] <TheMightyBuzzard> you admin types always have the banhammer if necessary
[01:23:51] * Bytram is reminded of the page-widening trolls of the other site... we'll deal with 'em as they come up; can't plan for *everyuthing*
[01:24:01] <NCommander> paulej72: its a future thing now
[01:24:12] <NCommander> My immediate priority is porting this hot mess to Apache 2
[01:24:20] <NCommander> Which is the last of the major shit to get me to sleep at nice
[01:24:21] <NCommander> *night
[01:24:35] <juggs> *nite
[01:24:50] <paulej72> *day
[01:24:55] <NCommander> .............
[01:24:59] <Bytram> NCommander: the benefit being... we can to a realse that is supported? (.e.g security fixes)
[01:24:59] <NCommander> you guys suck
[01:25:09] <NCommander> Bytram: bingo
[01:25:13] <NCommander> And more recent perl
[01:25:23] <NCommander> We're still stuck on 5.10, which I know caused pain in unicode world
[01:25:31] <TheMightyBuzzard> perl++
[01:25:31] <Bender> �karma - �perl: 1
[01:25:57] -!- LaminatorX [LaminatorX!~470ab0b6@23-00-716-777.dhcp.stls.mo.charter.com] has joined #staff
[01:25:57] <Bytram> elephant in the living room here: how are we going to regression test everything?
[01:26:24] <Bytram> or, do we let the community do the alpha testing when we roll it out?
[01:26:28] <NCommander> Bytram: usual matter, upgrade on a weekend and collect the bits as users step on landmines
[01:26:29] <audioguy> users, like evderyone else does. :-)
[01:26:31] <paulej72> on prod, let the users find the bugs
[01:27:03] <Bytram> ugh.
[01:27:08] <audioguy> I see great agreement on this. :-)
[01:27:15] <Bytram> LOL, but ugn. *UGH*!!!!!!!
[01:27:25] <NCommander> That's why we have a community, right?
[01:27:25] <TheMightyBuzzard> Bytram, you're our resident bug hunter. you can break anything.
[01:27:47] <NCommander> Ok, so I think we've covered UTF-8, Apache2
[01:27:49] <Bytram> You have *no* idea; them bugs *swarm* me, I tells ya!
[01:28:01] <NCommander> I had some editoral stuff to bring up, but without a LaminatorX ...
[01:28:11] <prospectacle> what about test.soylentnews.org for enthusiastic breakers-of-things to break things in the soon-to-be-released, and report what they broke via submitting an article
[01:28:18] * LaminatorX has returned.
[01:28:25] <paulej72> yeah ugh, but I have tried to have people find bugs on dev, but no love. Even when testing on staff, there was till bugs that we did not find until deploy on prod
[01:28:27] <NCommander> prospectacle: that causes bad juju
[01:28:36] * NCommander remembers improved threading
[01:28:38] <NCommander> >.<;
[01:28:44] <TheMightyBuzzard> prospectacle, dev is generally usable for that. your account on live will likely let you log in there.
[01:28:53] <prospectacle> ok thanks
[01:28:58] <NCommander> at some point, we need to refresh dev's database
[01:29:04] <paulej72> unless it is newer than April
[01:29:06] <NCommander> Ok, so UTF-8, Apache2
[01:29:10] <NCommander> Anything else on the dev side?
[01:29:17] <TheMightyBuzzard> yep, i got a few
[01:29:19] <Bytram> prospectacle: heads up, though... it's hard to tell what has been merged into it at any given time.
[01:29:53] <prospectacle> Does the general public know about dev? I mean some weird people might like trying to find bugs.
[01:29:56] <TheMightyBuzzard> 14.10: any objections to it being mostly a minor and quick bug/issue release?
[01:30:13] <TheMightyBuzzard> clear off a hell of a lot of small ones and leave the biggies for 14.12?
[01:30:23] <paulej72> none here, and I have lots of bug fixes
[01:30:33] <NCommander> TheMightyBuzzard: I'd like to land the Apache2 upgrade if finished in 14.10
[01:30:36] <LaminatorX> An occasional dev shutout on site news wouldn't be a bad thing.
[01:30:41] <NCommander> prospectacle: its been brought up a few times
[01:30:41] <paulej72> or 14.!3
[01:30:55] <TheMightyBuzzard> NCommander, i'd like to land m2 in there too but i see it running long like unicode did.
[01:31:26] <NCommander> TheMightyBuzzard: land it, but ship disabled
[01:31:28] <NCommander> Not a big issue
[01:31:31] <Bytram> NCommander: and I brought up some ideas on basic moderation with TheMightyBuzzard this AM, which might compound the situion
[01:31:44] <paulej72> NCommander: tackel Apache 2 if you want the rest or use will do the minor bug fixes
[01:31:48] <NCommander> Bytram: we need to compare notes, as I had thoughts for moderation that I've yet to get implemented
[01:31:54] <NCommander> but apache2 is a big sticker
[01:31:56] <TheMightyBuzzard> okay, item 2: that^^
[01:31:58] <Bytram> NCommander: works for me.
[01:32:14] * NCommander nods
[01:32:18] <NCommander> Anything else on dev?
[01:32:18] <TheMightyBuzzard> moderation changes in prep for m2. those would also fit well in with a light release.
[01:32:25] <TheMightyBuzzard> one more
[01:32:52] <TheMightyBuzzard> ddns for subscribers. pitched it to audioguy the other day and he was generally in favor.
[01:33:06] <Bytram> TheMightyBuzzard: ddns?
[01:33:19] <NCommander> dyanmic dns?
[01:33:25] <Bytram> .topic Staff meeting in progress.
[01:33:25] juggler changed topic of #staff to: Staff meeting in progress.
[01:33:31] <TheMightyBuzzard> dynamic dns. like get a hostname of bytramlikescheese.ddns.soylentnews.org that's always yours.
[01:33:36] <paulej72> is there an open source ddns cleint
[01:33:42] <TheMightyBuzzard> several
[01:33:53] <LaminatorX> As in, my host.soylentnews.org?
[01:34:01] <audioguy> This is easy to do just with bind config files.
[01:34:04] <Bytram> TheMightyBuzzard++ interesting. *always* mine? You mean so long as I continue my subscription? =)
[01:34:04] <Bender> �karma - �themightybuzzard: 1
[01:34:15] <paulej72> I would use another domin for that stuff.
[01:34:25] <TheMightyBuzzard> LaminatorX, probably need to make it whatever.ddns.soylentnews.org to avoid confusion
[01:34:26] <audioguy> agreed ^^
[01:34:34] <audioguy> another domain.
[01:34:36] <TheMightyBuzzard> or our shortening domain
[01:34:44] * NCommander isn't 100% sure this is useful, but ok :-)
[01:34:46] <paulej72> no not the .us one
[01:34:48] <LaminatorX> Sylnt.us then
[01:34:56] <LaminatorX> Or not
[01:35:04] <paulej72> too many restrictions on that domain
[01:35:07] <Bytram> TheMightyBuzzard: I like the idea of "whatever.ddns.soylentnews.org" as it will provide us free 'advertising' for the main site
[01:35:17] <TheMightyBuzzard> NCommander, it's not especially. think of it as another gold star.
[01:35:19] <audioguy> We have a lot of domains already voted as 'winners'
[01:35:33] <NCommander> indeed
[01:35:41] <paulej72> Bytram: that could lead us to being blocked by different spam lists
[01:35:52] <TheMightyBuzzard> fair point
[01:35:52] <NCommander> Let's try and get togheter a need/want/wish list for the next two releases
[01:35:59] <NCommander> Put it on the wiki/brainstorm/etc.
[01:35:59] <audioguy> Subscribers only.
[01:36:05] <Bytram> paulej72: yeah, you're right. darn!
[01:36:35] <TheMightyBuzzard> s'it for me.
[01:36:38] <NCommander> k
[01:36:40] <audioguy> Aother is to allow people to use us as slave, those are always needed and trivial to do.
[01:36:42] <NCommander> I think that covers dev
[01:36:57] <paulej72> I have sysop stuff
[01:37:39] <audioguy> go
[01:37:45] <NCommander> Lets go to sysop
[01:37:45] <NCommander> paulej72:
[01:37:51] <paulej72> We need to get our documentation up to date. I have spent too much time trying to fix things recently and I had no idea where the shit was
[01:38:12] <audioguy> Ok: here is what I have in the works:
[01:38:24] <paulej72> I know I am guilty of not documenting, so I am trying to fix this up.
[01:38:33] * NCommander is semi-guilty on this
[01:38:43] <NCommander> BIggest thing I need to document is the backup mechanism
[01:38:57] <audioguy> I am just finishing up some dns stuff and have a backlog of notes. I want to finish the twiki install next, and move suitable stuff over to it. I think that will help
[01:39:19] <paulej72> audioguy: whcih system did you install twiki on
[01:39:35] <audioguy> Thsi will give us a place to put slightly mnore sensitive stuff than the public wiki
[01:39:56] <audioguy> It isn't yet, really, I want ot pom staff ,achjine boron.
[01:40:01] <audioguy> want it on
[01:40:13] <audioguy> (this is limited to staff)
[01:40:15] <paulej72> ok
[01:40:51] <paulej72> I think we should go ahead with the plan to move stuff off of bery and move ti to boron and carbon as appropriate.
[01:41:07] <audioguy> I will send a note to either stafflist or admin list with more details when the time comes
[01:41:25] <paulej72> nitrogen could be moved to as well, but I was think of using that as a backup irc server.
[01:42:00] <NCommander> I'll be happy retiring bery
[01:42:07] <audioguy> re that move - I have been wanting to chamge main mail comfig for a while, this is a golden opportunity to just move that to the existing mail server on boron. This is all low usage staff mail
[01:42:09] <NCommander> That machine has long been the odd man out
[01:42:30] <NCommander> audioguy: do wait a bit before doing that
[01:42:40] * NCommander is going to upgrade boron to 14.04 this week
[01:42:44] <Bytram> paulej72: is there some kimd of report which gives system 'stats' on each of our servers? CPU Usage, RAM Usage, Swap, etc.? Would be nice to get a daily report.
[01:42:48] <audioguy> I don't do such things without a preannounce and clear plan.
[01:42:55] <NCommander> audioguy: cool :-)
[01:43:05] <NCommander> Bytram: there was icinga
[01:43:09] <NCommander> We should probably resurrect that
[01:43:10] <audioguy> unlike SOME around here. :-)
[01:43:22] <paulej72> NCommander: when I mentioned to xlefay about setting up a backup irc on nitrogen he mention that we should move that server to the London DC.
[01:44:03] <paulej72> icina is still there, I just cant figure out how to set it up.
[01:44:15] <Bytram> NCommander: not just *was*, still is: <a href="https://sentinel.soylentnews.org/" target="_blank">https://sentinel.soylentnews.org</a>
[01:44:50] <NCommander> oh, it got fixed
[01:44:50] <paulej72> lright now it give an error on carbon, because I switch the default webserver to nginx.
[01:44:53] <NCommander> sentinel poofed for awhile
[01:45:12] <Bytram> NCommander: yup, but it was un-poofed.
[01:45:24] <paulej72> plus the new severs are not bein monitored
[01:45:34] <NCommander> ugh
[01:45:48] <NCommander> FIxing that last bit would be relatively easy
[01:46:53] <audioguy> Do we NEED icinga? Has it EVER solved a problem or found one for us that was not already knoen?
[01:47:02] <audioguy> known
[01:47:10] <Bytram> From what I can tell, icinga is more about watching/checking for things running or not; does not give system 'loads'.
[01:47:18] <NCommander> adding loads won't be hard
[01:47:25] <NCommander> YOu can report anything via shell scripts and such
[01:47:26] <juggs> mrtg
[01:47:33] <NCommander> If someone wants to go through and setup nagios or mrtg though
[01:47:35] <NCommander> be my guest
[01:47:45] <audioguy> When something stops running we get pretty fast feedback from users.
[01:47:58] <paulej72> igcina was xlefay’s baby
[01:48:14] <juggs> NCommander, why are you not recommending Landscape?
[01:48:19] <juggs> ;)
[01:48:36] <audioguy> Landscape was running too
[01:48:41] <Bytram> okay, I forgot... what was it we were discussing?
[01:48:44] <audioguy> how many do we need?
[01:48:52] <juggs> quite
[01:49:21] <paulej72> consoldating servers and monitoring
[01:49:23] <NCommander> juggs: because I'm going to be leaving my employeer soon, and unliekly to get it for free :-P
[01:49:42] <juggs> rsyslog, snmp trap it - it doesn't matter. Make a fucking decision how you are going to monitor load and do it.
[01:49:57] <juggs> And set alerts
[01:50:07] <NCommander> Right, so moving onto consoldating stuff
[01:50:11] <audioguy> How about real sysadmins actually checking logs
[01:50:17] <NCommander> or not
[01:50:21] <Bytram> ummm, what was decided on documenting the systems?
[01:50:35] * NCommander coughs
[01:50:40] <NCommander> Sorry, I got distracted
[01:50:44] * Bytram offers another lozenge
[01:50:44] <NCommander> On the documentation front
[01:50:46] <paulej72> NCommander: needs to bust out his fast keyboard
[01:51:02] <NCommander> I have: document backup systems (NCommander), document other stuff (paul)
[01:51:09] <NCommander> probably need to be more specific on identifying doc gaps
[01:51:12] <Bytram> got it. thank
[01:51:41] <Bytram> paulej72: whenever you find an area lacking docs, can you just add it to some central place, like on the wiki?
[01:51:48] <audioguy> backup system a good start
[01:52:05] <paulej72> we should copy key config files for all of our services and put them in twiki
[01:52:25] <audioguy> Yes, I will start doig that once I have it set up.
[01:52:35] <Bytram> not the documention itself, but at least start a list of areas which are lacking.
[01:52:42] <Bytram> great!
[01:52:44] <paulej72> Bytram: that would almost be as much work as fixing the docs
[01:53:05] <audioguy> I believe some documentaion belogs in the connfig files, so it is handy when needed.
[01:53:14] <audioguy> config files
[01:53:59] <NCommander> What is needed is a list of where we got coverage gaps
[01:54:00] <Bytram> I don't much care *where*, but if there were one place where you could note when you stumble on an area that is deficient, it can (eventually) get tackled
[01:54:13] <paulej72> Plus we need to make sure all of our system have the same settings where possible and proper fallbacks. Lithium was not setup with both ldap and kerberos servers
[01:54:29] <Bytram> NCommander: exactly. Rather than a massive search, just build up a list over time as we stumble over the gaps.
[01:54:39] <audioguy> The main wiki for now, until I get twiki fully up. (few days, finishing some stuff on dns)
[01:55:46] <NCommander> anyone volunteering to take that list?
[01:55:48] * NCommander sees himself somewhat swamped for the rest of the week
[01:55:52] <NCommander> *take to compiling
[01:56:35] <paulej72> I have been working through the sysamin wiki pages so I can put down what is missing.
[01:56:38] <audioguy> I am doing this as I move through various things, but we all see different views of the system as we work on different things.
[01:56:53] <audioguy> MAke a page for 'missing docs' there
[01:56:54] <Bytram> NCommander: I'd say just grow it organically. Just note it when you find it... maybe toss in a sentinel string "HERE NEED BE DOCUMENTATION" and move on.
[01:56:55] <NCommander> Alright
[01:57:05] <NCommander> ONe last note
[01:57:14] <NCommander> Any object if I upgrade boron to 14.04?
[01:57:15] -!- LaminatorX_ [LaminatorX_!~470ab0b6@23-00-716-777.dhcp.stls.mo.charter.com] has joined #staff
[01:57:21] * NCommander is likely going to do it tongiht
[01:57:50] <Bytram> what's on it ? and what depends on what it is running? and what does boron depend on, too?
[01:57:51] <paulej72> yes, I want the backup docs wirten first.
[01:58:05] <audioguy> is the other dns machine fully finctional? We need at least one functional dns for the li domain to work.
[01:58:15] -!- LaminatorX has quit [Ping timeout: 244 seconds]
[01:58:34] <audioguy> we all log in on boron. Just don't mess that one up. :-)
[01:58:35] LaminatorX_ is now known as LamnatorX
[01:58:37] <NCommander> audioguy: helium is fully operational again
[01:58:43] <NCommander> so LDAP/DNS is fine
[01:58:48] <NCommander> paulej72: ask and you shall receive
[01:59:00] <audioguy> main concern is logioms then, I think
[01:59:01] <paulej72> SystemAdministration/Backups
[01:59:04] <audioguy> logins
[01:59:29] <NCommander> audioguy: you can get in via linode console worst case scenario, and I can setup a different box as a login one if needed
[01:59:48] <audioguy> I don't have access to linode console.
[02:00:10] <paulej72> javascript adversion
[02:00:19] -!- FunPika has quit [Ping timeout: 244 seconds]
[02:00:24] <audioguy> no problem is done quickly at any rate.
[02:00:24] <NCommander> paulej72: you can get into it via SSH
[02:00:33] <NCommander> ANyway
[02:00:39] <NCommander> anything else on sysops?
[02:00:40] <audioguy> I was wondering about that
[02:00:44] <paulej72> NCommander: that is thee thing, the password files may not be accessble to log in as root
[02:01:02] <NCommander> THat's why I keep a local backup of it
[02:01:18] <paulej72> we should have them emailed to the sysop staff.
[02:01:32] <NCommander> probably safer just to put it on t-wiki
[02:01:36] <Bytram> email == postcard
[02:01:42] <audioguy> yes, not emailed!
[02:02:00] <audioguy> put them in boron root where we can grab over secure ssh
[02:02:04] <paulej72> not if we loose twiki and ssh on boron at the same time
[02:02:23] <NCommander> audioguy: they're on helium in roots home folder which all sysops can get
[02:02:24] <audioguy> We shouldnot put passwords on twiki.
[02:02:25] <paulej72> if twiki is on boron that is a possibility
[02:02:50] <paulej72> NCommander: that is if boron is not being upgraded at the time
[02:03:09] <audioguy> Then we all need to just grab a copy from helium now.
[02:03:16] <paulej72> yes
[02:03:40] <NCommander> paulej72: I'm not upgrading it this instant
[02:03:45] <NCommander> If I do it tonight, it will be around midnight AKDT
[02:04:00] <audioguy> you know we are in staff, not a non public place right now. :-)
[02:04:13] <audioguy> just thought I wouod point that out. :-)
[02:04:21] <paulej72> yes I know, but in this case with boron down our only access to the other servers is via lish.
[02:04:29] * NCommander notes that nothing here is private; we've got most of the configuration on the wiki already
[02:04:48] <NCommander> paulej72: boron shoul donly be down during the reboot
[02:04:48] <paulej72> Should we set up as sceconday ssh entrance on carbon?
[02:04:57] <NCommander> I'll set that up
[02:04:57] <audioguy> yes, but eay to forget, in the heat of things - it has happened before.
[02:05:06] <TheMightyBuzzard> NCommander, you know you just doomed it, yes?
[02:05:09] <NCommander> I"ll tempt enable public login on carbon during the upgrade
[02:05:18] <NCommander> TheMightyBuzzard: yup :-)
[02:05:36] <paulej72> hydrogen was susposed to be down only a few minutes
[02:07:12] <NCommander> paulej72: technically speaking, its still up, just varnish is stopped :-P
[02:07:30] <paulej72> NCommander: the other issue that we need to look into is reducing gluster memeory usage
[02:07:44] <paulej72> I meant helium :)
[02:07:46] <NCommander> that too
[02:08:03] <NCommander> paulej72: helium was up, it just had some booting issues yours truly caused originally
[02:08:52] <paulej72> do we have db riplication working yet?
[02:09:10] <NCommander> yeah
[02:09:20] <NCommander> But I don't hae it setup a second reader on slash yet
[02:09:28] * NCommander is getting to the point he has to go
[02:09:31] <TheMightyBuzzard> paulej72, move all of the logs out of the glustered area
[02:09:35] <NCommander> Let's try and wrap this up quickly
[02:09:51] <paulej72> I have nothing else pressing
[02:10:15] * NCommander nods
[02:10:17] <NCommander> Editoral team?
[02:10:24] * TheMightyBuzzard is consulting over IM
[02:10:33] * NCommander had something to bring up, but its not urgent, and can take it to email
[02:10:38] -!- mechanicjay [mechanicjay!~jhowe@Soylent/Staff/Sysop/mechanicjay] has joined #staff
[02:10:38] -!- mode/#staff [+v mechanicjay] by juggler
[02:10:42] <audioguy> they are probably all asleep by now :-)
[02:10:45] <LamnatorX> One moment, let me switch to a proper keyboard.
[02:10:48] -!- LamnatorX has quit [Quit: Web client closed]
[02:11:30] -!- LaminatorX [LaminatorX!~470ab0b6@Soylent/Staff/Editor/LaminatorX] has joined #staff
[02:11:30] -!- mode/#staff [+v LaminatorX] by juggler
[02:11:42] <LaminatorX> Alright, ready.
[02:11:45] * NCommander didn't intend this to be a four hour long slosh
[02:11:47] <NCommander> Seriously sorry guys
[02:12:51] <NCommander> LaminatorX: your floor
[02:13:06] <LaminatorX> I'm happy with where we are staffwise. While each batch of new Editors suffers 50% attrition, we've got enough keepers now to handle two of us on leave simultaneously without straining the rest unduly.
[02:13:10] -!- Ethanol-fueled [Ethanol-fueled!~62b0c2de@cp88-216-234-862.sd.sd.cox.net] has joined #staff
[02:13:14] <paulej72> NCommander: just a quick fyi on the ipv4 backend links between boron and hydrogen/fluorine the spped is about half of the hydrogen to fluorine link
[02:13:58] <paulej72> LaminatorX: is there any that need their perms removed yet?
[02:14:43] <NCommander> paulej72: ugh
[02:14:56] <NCommander> LaminatorX: that's good to hear
[02:15:01] * NCommander fuck it, brins up his point
[02:15:04] <LaminatorX> We could probably do that for some of the early ones who've been idle for months. Let's wait a smidge longer though, I think 6 months idle is a nice round number, and we're not quite there yet.
[02:15:15] <NCommander> eh, no
[02:15:20] * NCommander is going to rathole us for 30 minutes
[02:15:32] <NCommander> LaminatorX: I removed girlwhoisplugged out awhile ago
[02:15:41] <NCommander> Not sure who else might need permission removal
[02:16:34] <LaminatorX> Dopefish and Cactus haven't posted in months as well. I don't have a problem with GWWPO being removed, though would've like to have been consulted.
[02:17:11] <paulej72> I thought he raged-quit or something
[02:17:19] <NCommander> LaminatorX: I did that quite a long time ago, I was just menthoning it now
[02:17:53] <paulej72> cant remember the specifics
[02:17:57] <LaminatorX> Gotcha. No ragequits from Editors, it seems. We seem to die with a whimper.
[02:18:30] <TheMightyBuzzard> wait till you see a sysadmin ragequit. can be a thing of horror and awe.
[02:19:03] <Ethanol-fueled> WHy was GirlWhoWasPluggedOut removed?
[02:19:09] <audioguy> unlikely with present crew.
[02:19:12] <LaminatorX> GungnirSniper rounds out the missing list.
[02:19:14] <paulej72> that is why you need to pet your sysadmins from time to time
[02:19:20] <Ethanol-fueled> Not that I care, I'm just kinda chuckling and curious.
[02:19:47] <NCommander> Ethanol-fueled: disappeared
[02:19:48] <paulej72> Ethanol-fueled: MIA for too long
[02:19:52] <LaminatorX> Zizban hadn't posted in a few weeks, but may just be in a back-to-school mode.
[02:19:56] <Ethanol-fueled> Flaked out, eh? Bummer.
[02:20:21] <audioguy> privileges are easily restored
[02:20:27] <Ethanol-fueled> School is a bitch, and the only reason why I'm not offering to help more (under a less-offensive alias, of course). Sorry guys.
[02:20:28] <paulej72> Zizban did say he would be irregular for a while
[02:20:51] <LaminatorX> I' not kidding about 50% attrition, this is less fun than it seems from the outside.
[02:20:52] * NCommander hates to do this, but I kinda need to bug out
[02:20:54] <Bytram> NCommander: I'm assuming that this is separate and distinct from the list of editors that appears at the bottom of a page and which shows what each editor was last working on?
[02:20:58] <NCommander> Can someone else take over for me?
[02:21:12] <NCommander> Bytram: that list is anyone whoe ver held admin privs, the stats table for them isn't cleared out
[02:21:14] <LaminatorX> It is that list, but that list is also historical.
[02:21:30] <LaminatorX> you'll note Barrabas is still on that list.
[02:21:37] <paulej72> I need to fix that. It bugs me
[02:21:47] * NCommander notes I think aside from IRC, this is the last thing to cover so I cna probably stick for 10 more mintues
[02:21:47] <Bytram> yup, and several others... hence my mentioning it.
[02:21:50] <juggs> Is this still the staff meeting? are we going alphabetically or by section? I dozed off I think
[02:21:53] <NCommander> paulej72: its a table somewhere in the DB
[02:21:56] <NCommander> juggs: staff meeting
[02:22:03] <NCommander> We need a better way to having this in the future
[02:22:05] <juggs> fuck me
[02:22:12] <paulej72> I nominate Bytram as our new meeting chair
[02:22:34] <TheMightyBuzzard> seconded
[02:22:43] <audioguy> unless there are other issues in editorial maybe we should just end?
[02:22:45] <Bytram> I'd be willing, but my schedule is a bit, varied.
[02:22:58] <paulej72> all those opposed
[02:23:00] <Bytram> umm, QA/test?
[02:23:04] <LaminatorX> We are quite grateful for working UTF-8.
[02:23:21] <LaminatorX> I have nothing pressing. Status report is: good.
[02:23:22] <paulej72> Bytram: just for the rest of this meeting
[02:23:47] <TheMightyBuzzard> awe, i was feelin a perfectly good coup
[02:23:53] <Bytram> ok, I *had* volunteered at the outset, IIRC =)
[02:24:23] <juggs> Did irc get a look in... or was that covered by paulej72 ?
[02:24:43] <paulej72> No we have not discussed that
[02:25:01] <Bytram> and at some point I'd like to bring up "QA and Test"
[02:25:21] <LaminatorX> Any other questions for Editorial, or shal I yield?
[02:25:39] <paulej72> none
[02:26:12] <Bytram> LaminatorX: do you have any issues/suggestions with the story submission/edin process?
[02:26:20] <Bytram> s/edin/edit/
[02:27:26] <LaminatorX> If I were to make a wishlist, a larger edit-box with larger type would be nice.
[02:27:44] <paulej72> LaminatorX: can do.
[02:28:08] <audioguy> We need an 'old guy' skin that has larger type. :-)
[02:28:14] <Bytram> LaminatorX: I noticed there is a 'drag' handle at the lower right-hand corner of the edit box.
[02:28:19] <paulej72> can do as well
[02:28:35] <Bytram> and a couple of "Ctrl-+" increases the font size (at least with FireFox)
[02:28:43] <paulej72> Bytram: that is only on some borwosers
[02:28:43] <TheMightyBuzzard> we need a oldschool skin too where blockquotes are prepended by a > and unstyled
[02:28:52] <audioguy> yeah, I do that
[02:29:04] <paulej72> TheMightyBuzzard: that is the vt100 theme
[02:29:05] <audioguy> works on opera too
[02:29:17] <TheMightyBuzzard> paulej72, yep yep
[02:29:27] <LaminatorX> The type on the page overall is fine, the edit box is just smaller. It never occurred to me to enlarge the whole page.
[02:29:44] <Bytram> hth
[02:30:27] <LaminatorX> As far as flaggin spam submissions for admins to block, it would be nice to move them into a different queue like the way the Hold function works.
[02:30:27] <paulej72> LaminatorX: I had that on my peronal todo list to fix at some point. Going to make a lot of boxes in the admin section larger and easir to edit in.
[02:30:48] <audioguy> bump the font size in css on the form, I often do that.
[02:31:16] <juggs> NCommander, I see no meeting, it is out of control. /me motions to exclude NCommander from the chair in future.
[02:31:38] * NCommander is barely here at the moment
[02:31:47] <NCommander> juggs: I passed chair off awhile ago
[02:31:56] <juggs> uch
[02:32:04] <LaminatorX> I'm happy to have an interface discussion with devs outside of the global staff meeting.
[02:32:04] <audioguy> Blame Bytram :-)
[02:32:27] * Bytram accepts blame
[02:32:55] <Bytram> Is that it for Editorial?
[02:32:55] <audioguy> We should just end.
[02:33:08] <Bytram> QA/Test
[02:33:12] <audioguy> 3 hours 32 minutes
[02:33:21] <LaminatorX> Editorial has nothing else to bring-up. (Feel free to email me with your kettle-of-fish, NCommander.)
[02:33:48] <Bytram> Testing of UTF-8 was a challenge...
[02:34:19] <Bytram> TheMightyBuzzard: kept finding areas in the code which were not amenable to allowing UTF-8 chars through correctly, so it was a moving target.
[02:34:39] <Bytram> He did a *great* job of keeping me in the loop, so to speak.
[02:34:59] <TheMightyBuzzard> easy with one dev n one qa
[02:35:07] <TheMightyBuzzard> gets complicated from there
[02:35:08] <Bytram> I've hit at in a number of nefarious ways, but don't know how much of the code actually got covered.
[02:35:26] <Bytram> so, far, though. it *seems* to be holding up well.
[02:35:58] * Bytram should really write up a doc covering unicode, UTF-8, IDN, RFCs, etc.
[02:36:02] <paulej72> not enough as we found bugs after deploy, but utf8 is a very large test sruface
[02:36:28] <TheMightyBuzzard> fat woman vs midget big, yep
[02:36:48] <paulej72> fat woman vs a filed mouse
[02:37:08] <Bytram> if I had a specification which listed exactly what was to be changed from a user's perspective, as well as what was going to/from the DB, then I could have provided better coverage... still given from what we started with, I'd say we got a bunch of nasties out of the way before deploy.
[02:37:48] <Bytram> Am mentioning this primarily to put a stae in the ground for futre reference.
[02:37:51] <TheMightyBuzzard> prolly a good idea.
[02:38:00] <paulej72> Bytram: part of the issue here is we are still learing where all of the code is located in slash
[02:38:11] <TheMightyBuzzard> true dat
[02:38:13] <Bytram> *exactly*
[02:38:23] <TheMightyBuzzard> and then promptly forgetting half of what we learned.
[02:38:38] <LaminatorX> Is there documentation of the overall archetechture going on as part of the effort?
[02:38:40] <Bytram> now that we have some 'institutional' knowledge, it would be good to record it (maybe as some kind of post-mortem).
[02:38:55] <paulej72> we don’t know what the whole scope od a chenge is until we aply it and see where it breaks
[02:39:19] <Bytram> paulej72++ bingo
[02:39:19] <Bender> �karma - �paulej72: 10
[02:39:40] * TheMightyBuzzard makes notes
[02:39:59] <Bytram> but, as we find new edges and corners while developing, it woud be helpful if someone could make a note so the rest of us don't have to skin our knuckles, too.
[02:40:15] <paulej72> I am trying to put that knowledge inot the git commits
[02:40:16] <Bytram> one other thing
[02:40:16] <TheMightyBuzzard> damned fine idea.
[02:40:44] <Bytram> the site experienced some, instability, for a few days...
[02:41:03] <Bytram> paulej72++ you did a phenomenal job getting things back up and running.
[02:41:03] <Bender> �karma - �paulej72: 11
[02:41:13] <paulej72> many out of band deploys right now
[02:41:24] * Bytram has no idea how but suspects a secret stash of dilithium crystals
[02:41:51] <Bytram> that's my report.
[02:42:04] <paulej72> nah I just stopped th server that was running shitty.
[02:42:10] -!- NCommander has quit [Ping timeout: 244 seconds]
[02:42:14] <Bytram> and, if nobody else has anything, I make a motion to adjourn
[02:42:38] <LaminatorX> 2nded
[02:42:42] <paulej72> Bad thing is I have not been able to figure out why it is acting shitty.
[02:42:45] <paulej72> Does juggs have IRC stuff
[02:43:08] <paulej72> or did sleep get to him first
[02:43:17] <Bytram> actually, that just reminded me.
[02:43:22] <matt_> [22:36] * juggs fades out of existence for now
[02:43:27] <matt_> ^from #soylent
[02:43:38] <Bytram> the LOGIN msg of the day lsits a bunch of folks running IRC. are they still all active?
[02:43:59] <Bytram> here it comes...
[02:44:00] <Bytram> * - ==============================================================
[02:44:01] <Bytram> * - # The IRC staff: #
[02:44:01] <Bytram> * - # #
[02:44:01] <Bytram> * - # * xlefay - Team leader #
[02:44:01] <Bytram> * - # * Landon - Member #
[02:44:04] <Bytram> * - # * MrBluze - Member #
[02:44:06] <Bytram> * - # * Kobach - Member #
[02:44:08] <Bytram> * - # * FunPika - Member #
[02:44:08] <paulej72> no, but I had other thing on my todo lit to fix that
[02:44:10] <Bytram> * - # #
[02:44:12] <Bytram> * - ==============================================================
[02:44:14] <Bytram> * - # If you require any assistance, please /join #help #
[02:44:16] <Bytram> * - ==============================================================
[02:44:24] <Bytram> paulej72: great! thanks a bunch!
[02:44:43] <LaminatorX> I believe MrBluze departed after the name vote.
[02:44:54] <Bytram> nod nod
[02:45:19] * Bytram needs to be going
[02:45:35] <Ethanol-fueled> MrBluze left as well, eh?
[02:45:45] <Bytram> Does IRC have a report?
[02:45:57] <TheMightyBuzzard> kobach split too, no?
[02:46:06] <paulej72> yes
[02:46:24] <TheMightyBuzzard> basically down to landon and... well really just landon on that list.
[02:46:45] <LaminatorX> FunPika was here earlier.
[02:46:51] <TheMightyBuzzard> ya? sweet
[02:46:58] <Ethanol-fueled> xlefay is lurking.
[02:47:12] <Bytram> I may be mistaken, but I think xlefay is in more of an 'inactive' state than a 'quit' state.
[02:47:36] <paulej72> but he could show up at any time and be more active.
[02:47:39] <TheMightyBuzzard> bout my understanding too
[02:48:38] <mechanicjay> I miss my buddy xlefay
[02:48:39] <TheMightyBuzzard> should probably reorganize that list to reflect the current situation though. even if nobody ever reads the motd.
[02:49:00] <Bytram> a quick check suggests his last msg on IRC was about 9 days ago... " Just can't be as active on IRC as I used to be, got actual stuff to do nowadays :("
[02:49:29] <Bytram> not certain, but that's the best I can tell on short notice using crutchy's ~last command on #Soylent
[02:49:34] <paulej72> I think the only thing that Juggs wanted to bring up is staff abuing their IRC privs by kicking people and oping bots that should not be
[02:49:39] <LaminatorX> I imagine that will be most of us at one point or another.
[02:49:47] <Ethanol-fueled> Bytram, what set kobach off?
[02:50:02] <Bytram> no idea at all.
[02:50:24] <Bytram> I may have confused him with MrBluze
[02:50:35] <TheMightyBuzzard> i only ever .kick the real bastards. only that TheMightyBuzzard guy so far.
[02:50:45] <Bytram> Ok, if there's nothing else in re: IRC... do I hear a motion to adjourn?
[02:51:01] <LaminatorX> Said motion is cuttently on the table.
[02:51:10] <paulej72> kobach got upset when we called him out on abusing his irc privs
[02:51:10] <Bytram> vote
[02:51:11] <LaminatorX> s/tt/rr
[02:51:16] <TheMightyBuzzard> move it out the way then to make room for the pizza
[02:51:18] <audioguy> yeah
[02:51:23] <TheMightyBuzzard> aye
[02:51:33] <Bytram> aye
[02:51:38] <LaminatorX> eye
[02:51:45] <matt_> i
[02:51:55] <paulej72> I
[02:52:22] <Bytram> from the chair's perspective, the motion is carried and I hereby call this staff meeting adjourned.
[02:52:27] * Bytram raps imaginary gavel