Timestamps are in GMT/BST.
[0:21] * es7 uses unstable
[0:31] <KenMan> how ? does it actually work ?
[0:42] <es7> Seems to work pretty good actually
[0:46] <mikeDOTd> no doubt, scaling the major hurdle that freenet-stable faces
[0:46] <mikeDOTd> s/the/is the
[1:08] * es7 (~jonmerrim@c-67-168-176-97.client.comcast.net) Quit ("BitchX: the official IRC client of the 2004 Olympic Games")
[1:24] * Ash-Fox (Hal-9000@aap229.neoplus.adsl.tpnet.pl) has joined #FreeNET
[2:56] * null_pointer_ (~null_poin@S0106005018225cc7.wp.shawcable.net) has joined #freenet
[2:57] <null_pointer_> Is the latest stable build broken? All I'm getting a route not found
[3:06] <Ash-Fox> Not sure... Working on a borg cube here
[3:08] <iip_i2p> <mule> What's going on in the network. i have only around 90 connections in each of two nodes. and i get repeated RNFs in nodes that are up for many weeks.
[3:08] <null_pointer_> borg cube eh?
[3:13] <iip_i2p> <mule> are borg cubes disturbing communication?
[3:33] * null_pointer_ (~null_poin@S0106005018225cc7.wp.shawcable.net) has left #freenet
[4:20] * |UK-Monster| (~bbtt@cpc2-warr1-5-0-cust27.bagu.cable.ntl.com) has joined #freenet
[4:21] <iip_i2p> <Newsbyte> let the toad be heard
[4:22] <mazzanet> ps aux | grep Newsbyte | grep -v grep | awk '{print $2}' | xargs -l1 kill -9
[4:23] <mazzanet> .
[4:24] <iip_i2p> <Newsbyte> Newsbyte is immune for grepping
[4:24] <mazzanet> killall iip
[4:24] <mazzanet> :P
[4:26] <iip_i2p> <Newsbyte> hmm...that's also a self-destruct option... ;-)
[4:27] <iip_i2p> <Newsbyte> so, I hear all those wonderful things about 5088?
[4:27] <iip_i2p> <Newsbyte> that it's full of RNF and DNF....so, ermm...
[4:27] <iip_i2p> <Newsbyte> what's new under the sun?
[4:31] <iip_i2p> <Newsbyte> that said, I didn't try out 5088 myself, yet. What are your experiences, mazza?
[4:32] <mazzanet> i'm an unstable boy
[4:32] <iip_i2p> <Newsbyte> well, weren't the changes implemented there first? :-)
[4:33] <mazzanet> *shrug*
[4:35] <iip_i2p> <Newsbyte> blasfemy! How dare you to shrug! It's forbidden by Saint Sanity and his prophet Toad!
[4:41] * leex (~bbtt@cpc2-warr1-5-0-cust27.bagu.cable.ntl.com) Quit (Read error: 110 (Connection timed out))
[4:41] * mazzanet chops newsbyte's length of cat5
[4:46] <iip_i2p> <Newsbyte> cat5, cat5...do you think I use such obsolete material?! Pooh!
[4:48] * spaetz (~spaetz@217-162-197-136.dclient.hispeed.ch) has joined #freenet
[4:48] <iip_i2p> <Newsbyte> /me spits at cat5. Now, something different may be 5 pussies, but cat5 surely is for wussies!
[4:49] <iip_i2p> <Newsbyte> /me is a great poet :-)
[4:50] * thelema|away (~thelema@adsl-65-66-61-223.dsl.stlsmo.swbell.net) Quit (Read error: 60 (Operation timed out))
[4:51] <ShaunMacPherson> hi :)
[4:51] <iip_i2p> <Newsbyte> hi
[4:52] <ShaunMacPherson> why are you hiding on iip newsbyte ;p
[4:52] <iip_i2p> <Newsbyte> with thelema arriving just now, we'll be out of Freenet-trouble in no time...
[4:52] <iip_i2p> <Newsbyte> all my worshippers are here, shaun
[4:52] <iip_i2p> <Newsbyte> ;-)
[4:52] <ShaunMacPherson> :)
[4:56] * mazzanet kicks iip
[4:56] <mazzanet> the installer wants random data
[4:57] <iip_i2p> <Newsbyte> you mean, like what is in your brain?
[4:57] <iip_i2p> <Newsbyte> ;-p
[4:57] <mazzanet> but i've been moving my mouse randomly and press random keys for the last 5 mins
[4:57] <mazzanet> but the ok button isn't enabled
[4:58] <iip_i2p> <Newsbyte> such is the life of men and mice
[4:58] <iip_i2p> <Newsbyte> so, shaun, what do you think of build 5088?
[4:59] <ShaunMacPherson> i havent tried it yet, is it working?
[4:59] <ShaunMacPherson> i heard they are going to reset all the datastores are something, wahts gong on :)
[5:01] <iip_i2p> <Newsbyte> well, whether it's working I would like to know myself. Heard there were lots of RNFs and such, but that's nothing new under the sun
[5:01] <iip_i2p> <Newsbyte> and the datastore reset was for the small-chunk size thingy; but I'm not even sure when toad is going to do that
[5:03] <ShaunMacPherson> me either :)
[5:03] <ShaunMacPherson> im just waiting for it to work hehe
[5:03] <ShaunMacPherson> someone forked mute into java, i was going to try that out but my router seems 'busy' so i cant port forward
[5:04] <iip_i2p> <Newsbyte> wow! you are THAT young?!
[5:04] <ShaunMacPherson> ;p
[5:04] <iip_i2p> <Newsbyte> that you can wait and still live when freenet will be working
[5:04] <ShaunMacPherson> i know :)
[5:05] <ShaunMacPherson> i got your joke, waiting isn't that bad hehe
[5:05] <iip_i2p> <Newsbyte> only for the patient
[5:06] <ShaunMacPherson> i've been organizing my files for the day when i will release all my content for upload, so even if it was working i wouldnt be ready
[5:07] * Ash-Fox (Hal-9000@aap229.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[5:07] <iip_i2p> <Newsbyte> be sure to make all your files smaller then 32 kb ;-)
[5:07] <ShaunMacPherson> :P
[5:07] <mazzanet> god damn
[5:07] <mazzanet> how much random data does this iip want
[5:08] <iip_i2p> <Newsbyte> I think a bit less then there are quantummechanical particles in the universe, maz
[5:10] <ShaunMacPherson> lots :)
[5:12] <iip_i2p> <Newsbyte> pfff...I dunno what to think. I'm in principle for making the small-chunks thingy before 0.6, if it's possible financially, but man...0.6 has been WAY overdue
[5:13] <ShaunMacPherson> what is the benefit of a whole bunch of small chunks
[5:16] <iip_i2p> <Newsbyte> I think toad made a post explaning the benefits
[5:18] <iip_i2p> <Newsbyte> a) Help beat traffic analysis. This would greatly reduce our
[5:18] <iip_i2p> <Newsbyte> vulnerability to passive traffic analysis.
[5:18] <iip_i2p> <Newsbyte> b) Make routing simpler. We could cut out ~ 3 of the KeyspaceEstimators.
[5:19] <iip_i2p> <Newsbyte> and there is a c and d also :-)
[5:19] <ShaunMacPherson> hmmm
[5:19] <ShaunMacPherson> sounds good :)
[5:20] <iip_i2p> <Newsbyte> indeed, but some questions remain: when will it be implemnted; before or after 0.6
[5:20] <iip_i2p> <Newsbyte> indeed, but some questions remain: when will it be implemnted; before or after 0.6
[5:20] <ShaunMacPherson> it will be
[5:20] <iip_i2p> <Newsbyte> and what size. I would opt for 128 or 64 at the smallest, toad wants to try 32
[5:20] * spaetz (~spaetz@217-162-197-136.dclient.hispeed.ch) Quit (Read error: 110 (Connection timed out))
[5:22] <iip_i2p> <Newsbyte> but with 32kb, the overhead will be between 12 and 18%, which make you question whether the benefits small chunks give is not going to be reduced if they become THAT small
[5:23] <ShaunMacPherson> that is too high of overhead
[5:23] <ShaunMacPherson> chunks should be 128 :)
[5:24] <ShaunMacPherson> 15% overhead ontop of freenet's overhead is too muh
[5:24] <iip_i2p> <Newsbyte> well, on itself, in absolute terms, it's not much (around 5 kb, toad guesses). But obviously, the relative percentage augments the smaller the chunks become.
[5:27] <ShaunMacPherson> yes
[5:28] <mazzanet> newsbyte
[5:28] <mazzanet> help
[5:29] <mazzanet> i installed iip on my nix box
[5:29] <mazzanet> started it
[5:29] <mazzanet> and xchat is stuck on "Connected. Now logging in."
[5:30] <mazzanet> oh noes
[5:30] <mazzanet> [2004/08/03-09:48:43]:sockOpen: open failed: host(iip.invisiblenet.net), port(20046)Failed to open an outbound connection.
[5:33] <ShaunMacPherson> what version of linux?
[5:34] <mazzanet> slackware
[5:37] <ShaunMacPherson> i have knoppix
[5:37] <ShaunMacPherson> not sure how to fix your problem though im still learning it, hopefully newsbyte is around :)
[5:37] <ShaunMacPherson> still
[5:40] * mazzanet slaps a large trout around a bit with a newsbyte
[5:40] * mazzanet slaps a large trout around a bit with a iip_i2p
[5:43] * tyesinclair (~tyesincla@170.252.64.1) has joined #freenet
[5:43] * tyesinclair (~tyesincla@170.252.64.1) has left #freenet
[5:49] * eMBee (~mbaehr@sirius.iaeste.tuwien.ac.at) Quit (Remote closed the connection)
[5:55] * eMBee (~mbaehr@sirius.iaeste.tuwien.ac.at) has joined #freenet
[6:00] <iip_i2p> <Xmazza> hi!
[6:00] <mazzanet> woooo
[6:02] <iip_i2p> <Xmazza> *slaps ShaunMacPherson around a bit with a large trout*
[6:03] <mazzanet> hehe
[6:04] * Hory (~Miranda@81.196.25.110) has joined #FreeNet
[6:12] * eMBee (~mbaehr@sirius.iaeste.tuwien.ac.at) Quit (Remote closed the connection)
[6:17] <ShaunMacPherson> hi :)
[6:18] <mazzanet> got it working :P
[6:18] <ShaunMacPherson> how?
[6:19] <mazzanet> using the nix client
[6:19] <ShaunMacPherson> kool
[6:20] <iip_i2p> <Xmazza> yay for remote x over ssh and cygwin :D
[6:21] <ShaunMacPherson> :p
[6:24] * sanity (~ian@81-178-89-164.dsl.pipex.com) has joined #freenet
[6:24] * ChanServ sets mode +o sanity
[6:33] * Overand (common@80.103.252.64.snet.net) Quit ("OMG I'M QUITTING!")
[6:35] * thelema|away (~thelema@adsl-65-65-202-46.dsl.stlsmo.swbell.net) has joined #freenet
[6:38] * ShaunMacPherson is now known as Shaun-Away
[6:52] * Ash-Fox (Hal-9000@aai53.neoplus.adsl.tpnet.pl) has joined #FreeNET
[6:55] * eMBee (~mbaehr@sirius.iaeste.tuwien.ac.at) has joined #freenet
[6:58] * lostlogic (~lostlogic@node-402418b2.mdw.onnet.us.uu.net) has joined #freenet
[7:05] * Ash-Fox (Hal-9000@aai53.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[7:18] * Pascal (Pascal@c-24-13-17-191.client.comcast.net) Quit ()
[7:18] * guido^pe (~unknown@dsl-213-023-243-202.arcor-ip.net) has joined #freenet
[7:46] * robilad[afk] is now known as robilad
[7:58] * guido^pe (~unknown@dsl-213-023-243-202.arcor-ip.net) Quit (Remote closed the connection)
[7:58] * Hory (~Miranda@81.196.25.110) Quit (Connection timed out)
[7:59] * lostlogic (~lostlogic@node-402418b2.mdw.onnet.us.uu.net) Quit (Remote closed the connection)
[8:04] * greycat (~wooledg@192.35.79.70) has joined #freenet
[8:06] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[8:22] * robilad is now known as robilad[afk]
[8:30] * g4lt (galt@g4lt.base.supporter.pdpc) has joined #freenet
[8:33] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Nick collision from services.)
[8:33] * Ash-Foxeus (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[8:45] * lolo-laptop (~lostlogic@68.251.84.226) has joined #freenet
[8:48] * ninja_ (im10ninjas@modem-741.javan.dialup.pol.co.uk) has joined #freenet
[9:15] * toad_ (toad@82-32-16-91.cable.ubr03.azte.blueyonder.co.uk) has joined #freenet
[9:16] <greycat> Ribbit.
[9:29] <lolo-laptop> Croak.
[9:31] * tyesinclair (~tyesincla@170.252.64.1) has joined #freenet
[9:32] * tyesinclair (~tyesincla@170.252.64.1) has left #freenet
[9:33] * hirvox (~hirvox@cs181027153.pp.htv.fi) has joined #freenet
[9:40] <Ash-Foxeus> .
[9:44] <toad_> ribbit
[9:45] <lolo-laptop> toad_: fix rate limiting, please?
[9:45] <lolo-laptop> :)
[9:45] <lolo-laptop> :-D
[9:48] <g4lt> lolo-laptop he's been trying to do that since The Day that Freenet Melted ;P
[10:00] * robilad[afk] is now known as robilad
[10:06] * Jormngndr (null@ip68-1-168-77.ri.ri.cox.net) has left #freenet
[10:11] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[10:12] * Ash-Foxeus (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[10:12] * nextime (~nextime@danex.i-m-c.it) Quit ("VACANZEEEEEEEEEE")
[10:12] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Nick collision from services.)
[10:12] * Ash-Foxeus (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[10:23] * g4lt beats firefox-0.8, since it doesn't like localhost:8888 :(
[10:25] * Ash-Foxeus (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[10:34] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[10:37] <KenMan> 'The Day that Freenet Melted' - this is such a fallacy !
[10:38] <g4lt> KenMan this is sarcasm, from the defcon talk
[10:39] <g4lt> you can't put a day on freenet's meltdown, but it was doing bad things to my host as early as a year and a half ago
[10:40] <KenMan> and here i thought things didn't start going downhill until "Day 2'
[10:40] <KenMan> in, what, 1999 ??
[10:41] <g4lt> there were actually two meltdowns, one was a content meltdown, one was a protocol meltdown
[10:42] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[10:46] * galt (~galt@g4lt.base.supporter.pdpc) has joined #freenet
[10:46] * g4lt (galt@g4lt.base.supporter.pdpc) has left #freenet
[10:46] * ninja_ (im10ninjas@modem-741.javan.dialup.pol.co.uk) Quit (Nick collision from services.)
[10:46] * ninja_ (im10ninjas@modem-2924.lion.dialup.pol.co.uk) has joined #freenet
[10:49] <KenMan> FWIW this is all I have learned about 5088 so far - I set LGB=5088, and am calculating 1000qph/no. peers for MRI (if utilization were at 100%)
[10:49] <KenMan> i'm at 55 peers, MRI=205s. The actual number of queries is about 600/hr, thus 60% utilization.
[10:50] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[10:50] * thelema|away is now known as thelema
[10:50] <KenMan> queries per minute twitches quite a bit , roughly from 5 to 15.
[10:51] <KenMan> so there are 'clumps' of queries scrambling around the network, i guess.
[10:51] <thelema> 10 queries per minute. Wow, that's a lot less than the kqph we used to have.
[10:51] <KenMan> no, i'm forcing it to that by sending out a fixed MRI which cannot produce any more than 1000qph.
[10:51] <KenMan> as an experiment
[10:51] <thelema> ah. hmm...
[10:51] <thelema> you want to find out if load will even out?
[10:52] <KenMan> rate limiting produces an almost perfect 10qpm average, but the twitchiness is from an external cause
[10:52] <KenMan> i don't fully understand what the network is doing, but clearly the total requests are surging on other nodes
[10:52] <KenMan> according to other peoples reports
[10:53] <KenMan> when I ran stock 5088, queries rose and fell from 3K/hr to 20K the next hour, and that's as 'stable' as it got
[10:54] * lolo-laptop sets his LGB to 5088 and fires back up his node
[10:54] <KenMan> that's a large volume of queries to come and go, one that seems hard to explain by weird rate limiting effects alone
[10:57] <KenMan> the upshot here is that my backoff is fairly level at about 70%, rather than the 80-90% recently become 'common'
[10:57] <KenMan> it twitches a bit too, in direct response to the query twitch
[10:58] <KenMan> but it has a perfectly stable average level, 66%
[10:59] * KenMan just realized this is the perfect environment in which to study the effects of MRI-from-routes distributions
[10:59] <KenMan> and how they affect backoff, thus routing effectiveness
[10:59] <thelema> KenMan: having a mri that's lower than the global average means nodes will be less backed off than normal, since you're sending less queries out.
[10:59] <thelema> That is a good idea, studying mri this way.
[11:00] <KenMan> the average of incoming mris from other peers has become much more erratic with 5085+ ... :(
[11:01] <thelema> :(
[11:01] * toad_ will be back soon
[11:01] * toad_ is just answering email
[11:05] * gqjga (~bla@195.25.40.253) has joined #freenet
[11:05] <gqjga> hi
[11:05] * gqjga is now known as ganja
[11:05] * ganja is now known as ganjaa
[11:06] <ganjaa> anybody has read autonomy a freedom of thought here?
[11:06] <KenMan> here, somebody else try to analze these, given my fixed-mri explanation above ... http://mywebpages.comcast.net/jkcorson/FreeNet/5088_1kqph_c.png http://mywebpages.comcast.net/jkcorson/FreeNet/5088_1kqph_bw.png
[11:06] <thelema> "autonomy a freedom of thought"?
[11:06] <toad_> brb, going to get drinks
[11:06] <toad_> ganjaa: yes
[11:08] * toad_ back. How Do You Want To Mess Up Freenet Today?
[11:08] <toad_> <lolo-laptop> toad_: fix rate limiting, please? - well, I've been trying to but all I've got so far is yet more oscillations
[11:08] <KenMan> lets do it with flower-power ;)
[11:08] <lolo-laptop> toad_: I was just playin' I know you're working on it, bro
[11:09] <toad_> <KenMan> and here i thought things didn't start going downhill until "Day 2'
[11:09] <toad_> <KenMan> in, what, 1999 ??
[11:09] <toad_> <g4lt> there were actually two meltdowns, one was a content meltdown, one was a protocol meltdown
[11:09] * toad_ shares KenMan's opinions
[11:10] <toad_> lolo-laptop: your confidence (nay, faith) is appreciated!
[11:11] <toad_> KenMan: what's the current situation? What do you think causes the oscillations?
[11:11] <KenMan> my biggest distress is caused by the wiggliness of incoming MRIs in the 5088_1kqph_c graph
[11:12] <toad_> why do incoming MRIs matter?
[11:12] <toad_> oh, your saying... ah
[11:12] <toad_> ugh
[11:12] <toad_> that's hideous
[11:12] <KenMan> they greatly influence the degree of backoff
[11:12] <toad_> you're saying that the entire network is resonating
[11:12] <KenMan> badly
[11:13] <toad_> that this is contagious because the number of backed off nodes somehow influences the MRI
[11:13] <KenMan> on a very fine scale
[11:13] <toad_> how does it do thatr?
[11:13] <iip_i2p> <mule> what is the resonance frequency you observe?
[11:13] <KenMan> no, in reverse, incoming MRIs determine the degree of backoff, AND the degree of wiggliness of backoff
[11:14] <toad_> KenMan: how does it cascade?
[11:14] <toad_> mule: it's not very consistent
[11:14] <toad_> have a look at http://mywebpages.comcast.net/jkcorson/FreeNet/5088_1kqph_c.png
[11:14] <KenMan> i don't understand your question, please restate
[11:14] <toad_> KenMan: how does it propagate across the network?
[11:15] <KenMan> oh goodness, it just stuffs crinkly plastic paper in every nodes ears...
[11:16] <toad_> we discovered the hard way that if backed off nodes directly influence outgoing MRIs, we're stuffed
[11:16] <toad_> how does it indirectly influence it@?
[11:16] <lolo-laptop> I don't think my node resonates, it just plain eats all available bandwidth and all the nodes it talks to are always backed off
[11:16] <toad_> lolo-laptop: well, KenMan's does
[11:16] <toad_> and it looks like it's some sort of network wide phenomenon
[11:17] <lolo-laptop> icky.
[11:17] <toad_> how does one node's resonation cause another node to resonate?
[11:17] <KenMan> lets go at this from the other end. IF we could have perfectly fixed MRI for each link, AND we had a perfectly constant supply of queries, backoff would never really change, maybe ?
[11:17] <toad_> uh#
[11:17] <toad_> on average
[11:18] <toad_> although the set of available peers would change constantly
[11:18] <KenMan> in response to NGR choices, you mean ?
[11:18] <toad_> no, just as nodes become available
[11:19] <toad_> and cease to be available
[11:19] <KenMan> although the set would change, the number of available routes would remain constant, right ?
[11:19] <toad_> ugh, i have a fair number of nasty MRIs...
[11:19] <toad_> KenMan: approximately yes
[11:19] <KenMan> that sounds good. And very far from reality.
[11:19] <toad_> a LOT of them are in 6 figures...
[11:20] <toad_> most of those are in middle six figureds
[11:20] <toad_> KenMan: well, it would still resonate
[11:20] <KenMan> the average from all routes , a month ago, was 200s
[11:20] <KenMan> and it was fairly steady, i think. Lemme look for an older graph...
[11:21] <toad_> what were you thinking of doing to getGlobalQuota?
[11:21] <toad_> forcing it to 1000?
[11:22] <KenMan> not exactly, but you may be on a similar path of thought
[11:23] <KenMan> actually, that may be exactly one of my 'corrections'
[11:24] * toad_ hates this weather
[11:24] <toad_> hot and humid and horrible
[11:24] * ganjaa (~bla@195.25.40.253) Quit (Read error: 104 (Connection reset by peer))
[11:24] <KenMan> i had 3 goals : smooth the rate of change of MRIs, force all nodes to strive for the same level of queries, and double the MRIs-out for nodes that have a user doing any browsing in the last 60 seconds.
[11:25] <toad_> KenMan: well, even the recent changes have a slow rate of change of globalQuota
[11:25] <toad_> however it still oscillates heavily
[11:25] <KenMan> that last one attempts to make freenet route around 'sources of queries'
[11:25] <toad_> really heavily
[11:25] <toad_> KenMan: uh, well, maybe
[11:25] <toad_> it also makes it clear which nodes are producing queries
[11:25] <toad_> which is generally a bad thing
[11:26] <toad_> but i suppose premix routing will fix that
[11:26] <KenMan> no, i mean, "don't allow sequential MRIs to change by more than 10s / minute" : not a percentage
[11:26] <toad_> KenMan: what the fsck happened at ~ 3AM on the t3c graph?
[11:26] <KenMan> yeah, doubling MRIs has a bad effect on anonymity, but we just aren't there yet in that regard :(
[11:27] <KenMan> toad - it was just normal evolution, i guess. I let it go for an hour more, and then had to do a reboot
[11:27] <toad_> KenMan: it's not normal evolution
[11:27] <toad_> look at it
[11:28] <KenMan> i wanted to see if it would do something like that, and it did :(
[11:28] <toad_> something happened
[11:28] <KenMan> i know, i meant it was the normal evolution of whatever code that was ...
[11:28] <toad_> the rateLimitingLoad dropped by 50%
[11:28] <KenMan> t3 = toad's 3rd jar
[11:28] <toad_> instantly
[11:28] <toad_> and settled there for a while despite a brief spike
[11:29] <toad_> t2 has a hell of an oscillation
[11:30] <toad_> it's a problem that these things take so long
[11:30] <toad_> fixed key sizes would help in that respect
[11:30] <KenMan> t3 - I was excited at 11:00, but that excitement diminished until around 19:00, where I hoped things would smooth out for good
[11:30] <toad_> but it would be disruptive, and there'd be a lot of work to do on the client side...
[11:31] <KenMan> okay, the first and most important thing you can to do help smooth these oscillations (make a note for later)- currently, MRIs for a new connection start at zero, and take a while to approach the average outgoing value for the other peers.
[11:31] <toad_> KenMan: are we sure that the oscillations are a big deal?
[11:31] <KenMan> Start them off at the average and let then adjust , instead
[11:32] <toad_> MRIs for a new connection? huh?
[11:32] <toad_> they start at the minimum iirc
[11:32] <toad_> which is globalQuota/#conns
[11:32] <KenMan> MRIs are extremely important, they determine backoff. I think we just sort-of agreed above that NGR doesn't determine the backoff as much as MRIs do.
[11:32] <toad_> or something like that
[11:32] <toad_> no
[11:33] <KenMan> well, in logging them, they seem to start off at zero, and then go up.
[11:33] <toad_> I just suggested a theory and you completely ignored it
[11:33] <toad_> I don't get it
[11:33] <KenMan> Unfortunately, I am not logging them at this time... must hack PeerHandler again.
[11:33] <toad_> you mean after the node starts up?
[11:33] <toad_> or do you mean an established node getting a conn from a node it wasn't previously connected to?
[11:33] <KenMan> sorry, i didn't see your theory - what were you saying ?
[11:34] <KenMan> established node gets a new peer, initializes the MRI for that peer at zero, when it has a much higher average for all the other peers
[11:34] <toad_> no way
[11:34] <toad_> that is simply not what happens
[11:35] <KenMan> i think so, much do more logging.
[11:35] <toad_> not intentionally anyway
[11:35] <toad_> show me the logs
[11:35] <KenMan> well, mistakes usually are not intentional ;)
[11:35] <KenMan> i will set up logging again
[11:35] <toad_> the intent is to start it at the minimum
[11:35] <toad_> except that's the minimum quota
[11:35] <toad_> not the minimum MRI
[11:35] * robilad is now known as robilad[afk]
[11:35] <toad_> KenMan: logLevelDetail=freenet.PeerHandler:debuyg
[11:35] <toad_> debug even
[11:35] <toad_> no f*cking problem
[11:35] <toad_> you don't need to restart
[11:36] <KenMan> Too noisy ! I just want a 50 char log line !!
[11:36] <toad_> if you restart all the data will be meaningless anyway
[11:36] <KenMan> okay, if I can just flip it on, thats good too.
[11:36] <toad_> KenMan: so use the above and grep it!
[11:36] <toad_> and then sed it
[11:36] <toad_> and we have no chance of fixing it if we only have the raw values anyway
[11:36] <KenMan> how do i zip the fly again ?
[11:36] <toad_> eh?
[11:36] <KenMan> make the change effective ;)
[11:37] <toad_> ah
[11:37] <toad_> change it in the log file
[11:37] <toad_> it should pick it up
[11:37] <toad_> however, outgoing MRIs are shown on the Open Conns page
[11:37] <KenMan> done
[11:38] <KenMan> i need a log of every MRI sent, and who it gets sent to.
[11:38] <toad_> why?
[11:38] <KenMan> analysis
[11:38] <KenMan> I've done it before and it was very revealing.
[11:38] <KenMan> When looking at the ones coming in.
[11:38] <toad_> well, logLevelDetail as above should give you that
[11:39] <KenMan> And the ones going out.
[11:39] * toad_ checks out his Open Conns page
[11:39] <KenMan> that is a nasty page. How do you segregate the active nodes from the inactive ones ??
[11:39] <toad_> hmmm, they do seem to start low
[11:39] <toad_> ah i see
[11:40] <toad_> sort by
[11:40] * toad_ tries sort by requests received
[11:40] <KenMan> okay, logging is flogging my friggin' disk now :o I might be good for ten minutes worth of collection, at this rate :p
[11:41] <toad_> KenMan: the theory you discarded was the idea that it might actually be a routing problem
[11:42] <KenMan> i doubt it, rate limiting does even give routing a change :(
[11:42] <KenMan> don't give it a chance
[11:42] <toad_> KenMan: either the nodes are individually resonating, or the whole network is
[11:42] <toad_> if it's the latter there must be some means for it to propagate from one node to another
[11:42] <toad_> what is it?
[11:42] <KenMan> i think the whole net is... look at transfer rates and look at the graph of # connections, they are weird
[11:43] <toad_> well then IT MUST INVOLVE ROUTING!
[11:43] <KenMan> unfortunately, MRI is a form of routing :p
[11:44] <KenMan> i don't know, maybe you are right. I don't understand NGR nearly as well as you ...
[11:44] <toad_> my open connections all have exactly the same MRI apparently: 464030.44372190937ms
[11:44] <KenMan> But I do see how MRIs are influencing NGR in a negative manner
[11:44] <toad_> KenMan: my point exactly. If it is propagating, it must go through routing of requests somehow
[11:45] <toad_> not NGR itself
[11:45] <KenMan> toad - how many nodes are you looking at ? does it include ones that aren't connected, by any chance ?
[11:45] <toad_> but it would mean that the incoming MRIs were influencing the outgoing ones
[11:45] <toad_> KenMan: the ones that aren't connected have varying MRIs
[11:45] <toad_> the ones that are all appear to have exactly the same MRI
[11:45] <KenMan> outgoing MRIs tend to cluster around a very tight value
[11:45] <toad_> ah, i found one that didn't
[11:46] <toad_> 358145.49445710523ms
[11:46] <toad_> no that doesn't count
[11:46] <toad_> no connections
[11:46] <KenMan> it is a 'bug' or side effect of the code you didn't expect, is all
[11:46] <toad_> what is?
[11:47] <KenMan> that all the values are the same
[11:47] <toad_> perhaps
[11:47] <toad_> in any case it doesn't look like your theory has any weight to it
[11:47] <KenMan> 183M of logs in , what , 10 minutes ?? BLARGH. I'll craft my own log messages.
[11:48] <toad_> hmm
[11:48] <toad_> there's more variation in the stuff in my older logs...
[11:48] <KenMan> well, I will go focus on MRI behavior , unless you suggest a better place
[11:48] <toad_> KenMan: fair enough but it'll take you another 7 hours for it to start
[11:48] <toad_> well it has to be something to do with MRIs
[11:49] <KenMan> no, that's the beauty of my code, it is stable from MINUTE 1 ...
[11:49] <toad_> huh?
[11:49] <KenMan> look at 5088_1kqph_c
[11:49] <KenMan> that is from start up...
[11:49] <toad_> ummm no
[11:49] <toad_> i mean.. nevermind
[11:50] * toad_ has the best job i could possibly ever get, and I've failed miserably. I suppose I should just go kill myself, but that's no fun.
[11:50] <KenMan> I guess one thing is, if we don't find a solution in another day, will you consider reinstating 5084, so you can go do development ??
[11:51] <toad_> KenMan: what the fuck do you think i'm doing?
[11:51] <toad_> if not development?
[11:51] <KenMan> You haven't failed at all. You just haven't accomplished anything ! (I am ribbing you !)
[11:51] <toad_> and what chance do we have of fixing it on unstable, if THIS didn't show up on unstable?
[11:51] <toad_> we have to fuck up the users' experience if we're ever to have any chance of getting anywhere
[11:52] * KenMan agrees and likes that statement in an odd way
[11:52] <toad_> it might be an artifact of a mixed network
[11:52] <KenMan> you have to break things before you can really understand and fix them :)
[11:52] <toad_> the problem is it'd take at least a couple weeks for us to know whether it is if we reset stable
[11:52] <toad_> it might be an artifact of very large files
[11:53] <toad_> the problem is it would probably take several months work and at least another month's migration time to find THAT out
[11:53] <KenMan> it is not from large files, we've always had them...
[11:54] <KenMan> well, we have a stable network at 5088. That is working for some people some of the time. But not well, for most people, most of the time.
[11:54] <KenMan> So, if you stepped back to known behaviour with a 5089/4 , you could then do development on the mods that made things go bad, in stable
[11:54] <toad_> eh?
[11:55] <toad_> KenMan: large files are an issue because they make it extremely hard to see what's going on with rate limiting
[11:55] <KenMan> put everyone else back at 5084 , called 5089. Then YOU run 5088 and figure it out...
[11:55] <toad_> KenMan: IT WORKED ON UNSTABLE. IT WORKED ON UNSTABLE. IT WORKED ON UNSTABLE. IT WORKED ON UNSTABLE....
[11:55] <KenMan> ...figure out what's wrong with 5085-5088 code
[11:55] <toad_> okay it didn't work perfectly
[11:55] <toad_> but I never heard about any of this bullshit
[11:55] <KenMan> yeah, yeah. That's why you might do better to develop a single node in a sea of relative calm.
[11:56] <toad_> KenMan: if it is a network level thing then it will not show up in that situation
[11:56] <KenMan> at least, for addressing the unknown stuff that is 5085-5088
[11:56] <toad_> if it is a node level thing it should show up regardless, even on the unstable branch
[11:56] <toad_> which it didn't
[11:56] <thelema> toad_: many changes got merged to stable without proper testing.
[11:56] <toad_> unless you were asleep through the months we spent on this stuff on unstable
[11:56] <toad_> thelema: most of them didn't directly affect any of this stuff
[11:57] <thelema> still no excuse.
[11:57] <KenMan> no, there was NEW code that got slammed in somewhere on thursday or friday, i thought. Maybe I was confused by the jars you sent me
[11:57] <toad_> the BIG rate limiting and routing changes were HEAVILY tested for MONTHS
[11:57] <KenMan> yeah, i think they are okay.
[11:57] <KenMan> i think something else snuck in there along the way
[11:58] <KenMan> in the past 5 days
[11:58] <toad_> like what?
[11:58] <toad_> the bugfix for the 5087-5084 interaction?
[11:58] <KenMan> Pascal put stuff in, but that looked okay . granted i didn't inspect it closely
[11:58] <toad_> to stable?!
[11:59] <KenMan> i dunno if it got into stable
[11:59] <KenMan> do you have a branch in CVS for stable ? i thought you said you just patched up Version.java to produce stable jars
[11:59] <toad_> no, we have a branch
[11:59] * KenMan feels a little better now !!
[11:59] <toad_> heh
[12:00] <toad_> well, as i said... most of the changes were in unstable first
[12:00] <toad_> do you want to try resetting stable based on current 5088?
[12:00] <toad_> i'm totally convinced that that will still oscillate...
[12:00] <KenMan> i am too. But we could be wrong.
[12:00] <toad_> but if my recent changes still oscillate too, then there's not much else i can do short of fixed file sizes
[12:01] <KenMan> no, lets figure out one problem at a time, although one reset total would be good
[12:01] <toad_> well the problem is it's not going to be one reset
[12:01] <toad_> it's going to be one a month for the foreseeable future
[12:01] <toad_> KenMan: how can it possibly NOT oscillate without fixed file sizes?
[12:01] <toad_> or at least without much smaller file sizes?
[12:02] <toad_> it will always oscillate with large files being moved
[12:02] <toad_> at least it will unless we manage to figure out something akin to my recent changes
[12:02] <KenMan> coincidental differences in bandwidth capacities, aligned with the stars...
[12:02] <toad_> which not only do not prevent oscillations, they seem to make it WORSE
[12:03] <toad_> it may be that a more accurate bandwidth predicting load estimator would help
[12:03] <toad_> but i'd expect the current thing to work
[12:03] <toad_> oh also:
[12:03] <toad_> what's your limit set to?
[12:03] * interrupt (~chatzilla@64.122.23.213) has joined #freenet
[12:04] <KenMan> 18000
[12:04] <toad_> do you have any wierd settings?
[12:04] <KenMan> predicting thing went into cvs ?
[12:04] <toad_> well if you look at the bandwidth graphs...
[12:04] <KenMan> i am ALL WEIRDed out right now, dude.
[12:04] <toad_> I wouldn't expect that straight like
[12:04] <toad_> line
[12:04] <toad_> that straight line would be expected if it hit the hard limit
[12:04] <KenMan> i hacked PeerHandler
[12:05] <KenMan> which graph do you refer to ?
[12:05] <KenMan> oh, you mean, in general, my bandwidth use outgoing
[12:05] <toad_> well in particular do you have unusual settings for lowLevelBWLimitFudgeFactor or lowLevelBWLimitMultiplier ?
[12:05] <KenMan> no
[12:05] <toad_> they're %'ed out?
[12:05] <KenMan> no odd setting in .conf
[12:05] <KenMan> yes
[12:06] <toad_> hmmm
[12:06] * KenMan just verified that
[12:07] <toad_> maybe we don't need the kludge factor most of the time... maybe it's just for the peaks... maybe the result of this is that we normally flatline on our limit, not 40% above it as we should, if high level limiting is kaput... maybe pigs can fly...
[12:07] * KenMan likes free-range free-sky pig, it has more flavor
[12:07] * toad_ thinks freenet is too ambitious for a bunch of nerds and a fulltime uneducated underpaid hacker
[12:08] <KenMan> haha
[12:08] * toad_ thinks freenet is too ambitious for a major Fortune 100 corporation's research and development labs
[12:08] * toad_ thinks freenet is too ambitious for the United States Department of Energy
[12:08] <KenMan> well, in all honesty, and i don't mean to be mean, sometimes I think you overreach in your attempts.
[12:08] <lolo-laptop> toad_: what it needs is something like the Linux kernel development team... all of them ;-) :-P
[12:08] <KenMan> just remember, simple is good !! and THEN look at the code :p
[12:08] <toad_> lolo-laptop: they'd have given up and moved on to something more useful years ago though
[12:09] <lolo-laptop> hehe
[12:09] <toad_> lolo-laptop: like artificial intelligence!
[12:09] * KenMan believes MRI management is in sorry shape, and causing lots of problems
[12:10] <toad_> KenMan: would you please explain your theory of how more frequent MRI updates cause problems?
[12:10] <KenMan> sure, i try to.
[12:10] <toad_> I can only see more frequent MRI updates causing problems if we were teetering on the edge anyway and the whole MRI system was completely broken as designed
[12:10] <KenMan> well...
[12:11] <toad_> KenMan: you're about to say "yup", right?
[12:11] <KenMan> The change was, 'communicate MRI more often' . Do you agree that this could raise query levels ?
[12:11] <KenMan> ;)
[12:11] <toad_> yes, it could raise query levels
[12:11] <KenMan> toad knows me pretty well too...
[12:11] <toad_> if it raises it above healthy levels then the whole MRI system is broken as designed
[12:12] * thelema is now known as thelema|away
[12:12] <KenMan> it is not just a higher level of queries that hurts. It is the degree of fluctuation in the MRI values, across a single link
[12:12] <toad_> well, please try to get some stats on that
[12:12] <KenMan> if you are attempting to code 'predictive' and running averaged measurements on something so flaky, it won't work out.
[12:12] <toad_> huh?
[12:13] <KenMan> That's my evaluation right now. I will spend more time looking at MRIs again.
[12:13] <KenMan> they are all over the map. The idea of "rate control" is to produce a smooth rate. Not just to say "back off for 14.3ms, then go wild again" one time.
[12:14] * toad_ greps some old logs to see how it changes on a single link over time...
[12:14] <KenMan> if we put constraints on how much MRI can change in a single step , i expect the system would smooth out a good amount
[12:14] <toad_> KenMan: but we do!
[12:14] <toad_> globalQuota is heavily averaged out
[12:14] <KenMan> well, then they are not tight enough
[12:14] <toad_> the problem is that globalQuota varies dramatically in the LONG TERM
[12:14] <toad_> you've seen the cycles
[12:15] <KenMan> my concern is that MRIs can vary by quite a large amount in the very very short term
[12:15] <toad_> it has a period of HOURS, but it still oscillates, with a HUGE magnitude
[12:15] <toad_> KenMan: why does that matter?
[12:15] <KenMan> because it will NOT produce any meaningful "rate control"
[12:16] <KenMan> you get it right on the first query, and then not so good on the next 4000, well, what would you expect the results to be ??
[12:16] <toad_> my preliminary research suggests it doesn't vary THAT much
[12:16] <KenMan> mild changes can whoop the remaining code pretty bad, apparently.
[12:17] <KenMan> i don't know, I'll go look for 'evidence' and come back :)
[12:17] <toad_> hmm
[12:17] <toad_> KenMan: huh?
[12:17] <toad_> I don't see why any of that would cause the long term high magnitude oscillations anyway
[12:17] <KenMan> no, they are brand new.
[12:18] <KenMan> what are you going to do with stable ? you probably need to do /something/ soon.
[12:18] <KenMan> if a reset is the best choice, lets do it .
[12:19] <lolo-laptop> yes, soon... I get the feeling the network is degrading (in population and in condition) at a pretty painful rate
[12:19] <KenMan> if backing up to 5084, so you can study the 'normal' things there, helps, then do that !
[12:19] * toad_ writing some scripts to track change on a single link from logs...
[12:20] <KenMan> don't rule it out as an option. It is more likely to produce a 'steady' system than the reset, according to both our intuitions...
[12:20] <toad_> don't rule out what?
[12:20] <toad_> oh
[12:20] <toad_> yeah
[12:20] <toad_> well
[12:20] <toad_> I don't want to do that because I don't care that much about having a stable system _NOW_. I care about IMPROVING the system.
[12:21] * KenMan is laughing at how hard toad tries to stay clean - "I've never had to revert anything in my life!" ;)
[12:22] <KenMan> a business would shoot you right now, if you were maintaining a production system ! Which is why this is (usually) such a friendly environment.
[12:23] <KenMan> okay, i can't help the cynicism. "I care about IMPROVING the system" - well now we have more opportunity for that !!
[12:23] <KenMan> toad knows I don't mean anything by it :)
[12:24] <toad_> KenMan: it's not a production system
[12:25] * toad_ generating log of changes
[12:25] <toad_> if you would show me how to make a nice gnuplot of it that would be nice
[12:25] <toad_> i'll have a file of data in the form:
[12:25] <toad_> DATE 3:04:05 AM VALUE 153785.46796329718
[12:26] <toad_> DATE 3:05:01 AM VALUE 100208.18530783503
[12:26] <toad_> DATE 3:05:02 AM VALUE 103279.82116109946
[12:26] <toad_> DATE 3:05:07 AM VALUE 106191.34432415123
[12:26] <toad_> DATE 3:05:13 AM VALUE 109037.06741987322
[12:26] <toad_> DATE 3:05:17 AM VALUE 112258.43499108932
[12:26] <toad_> DATE 3:05:23 AM VALUE 116149.92844112865
[12:26] <toad_> DATE 3:05:27 AM VALUE 119737.04220711038
[12:26] <toad_> DATE 3:05:32 AM VALUE 123309.73529153665
[12:29] <toad_> hmmm
[12:29] <toad_> KenMan: what would you consider to be too fast changes?
[12:29] <toad_> I don't see anything I'd really consider as dramatic changes
[12:29] <toad_> it is moving quite a bit
[12:30] <toad_> lets see, to get from 61160.544370296004 to 124740.84604491128 takes from 3:02:45 to 3:03:08
[12:30] <toad_> so it more or less doubles in 1 minute
[12:31] <KenMan> that is terrible !!
[12:31] <toad_> yeah, i wonder why it does that...
[12:31] <KenMan> toad , GNUPlot can read that file as is. The single command would be plot 'filename' using $(5) with lines
[12:32] <KenMan> but, you want to use the time value for an x axis. This would plot them evenly, whereas time might not flow evenly
[12:32] <toad_> okay, detail:
[12:32] <toad_> PRI Aug 3, 2004 3:02:45 AM (freenet.PeerHandler, YThread-355, DEBUG): globalQuot
[12:32] <toad_> a: 1019.1896724443246, totalRequests: 1002.5319181584707, totalRequestsThisNode:
[12:32] <toad_> 42.20372311048563, thisNodeMinQuota: 14.797820381859694, thisNodeQuota: 58.8614
[12:32] <toad_> 77396339545, minRequestInterval: 61160.544370296004 (freenet.PeerHandler@53beaf
[12:32] <toad_> (DSA(97dc 7be0 f0b5 f704 6851 c24f 828c 5520 a7e2 04d9),tcp/213.184.241.46:2004
[12:32] <toad_> 6, sessions=1, presentations=3, ID=DSA(97dc 7be0 f0b5 f704 6851 c24f 828c 5520
[12:32] <toad_> a7e2 04d9), version=Fred,0.5,STABLE-1.50,5088): outbound attempts=0:1/1)
[12:33] <KenMan> I'll send you a script you can use for a file that looks like that.
[12:34] <toad_> now, the second one, has globalQuota of 1013, so more or less the same, totalRequests the same, totalRequestsThisNode 42, so almost identical, thisNodeMinQuota is identical, but mRI is doubled
[12:34] <toad_> why?
[12:34] <KenMan> i make log lines that look like what you showed above. Just DATE DSA(1234 5678 MRIout=value (drop the fractional part)
[12:34] <toad_> globalQuota: 1019.1896724443246, totalRequests: 1002.5319181584707, (globalQuota < totalRequests)
[12:34] <KenMan> bad math is my best guess :p
[12:34] <toad_> versus globalQuota: 1013.0517767664014, totalRequests: 1026.8182904088674
[12:34] <toad_> globalQuota > totalRequests
[12:34] <toad_> that is why...
[12:39] <toad_> hmmm
[12:39] <toad_> how can we make globalQuota consistent with totalRequests?
[12:40] <toad_> what if we are in fact getting slightly too many requests?
[12:40] <KenMan> gnuplot> set xdata time
[12:40] <KenMan> gnuplot> set timefmt "%H:%M:%S"
[12:40] <KenMan> gnuplot> plot 'filename' using 2:($5) with points pt 2
[12:40] <KenMan> what version of gnuplot do you have ? I strongly recommend 4.0 (latest from www.gnuplot.info)
[12:41] <toad_> MRI is doubled because thisNodeQuota is halved
[12:41] <toad_> gnuplot 4.0 patchlevel 0
[12:41] <toad_> KenMan: I see a problem, if I accept your assertion that short term fluctuations are a major problem
[12:41] <toad_> are they?
[12:41] <KenMan> excellent. Try the above with even a small amount of data to verify it works
[12:42] <KenMan> they are when you start applying averages to them...
[12:42] <toad_> hmm?
[12:42] <toad_> what do we do with inbound MRIs?
[12:43] <KenMan> i don't know, you know that better than me...
[12:43] <toad_> and the whole point of averages is to even out short term fluctuations
[12:43] <toad_> inbound MRIs have absolutely no direct effect on outbound MRIs
[12:43] <toad_> if there is an indirect effect, we've got problems
[12:43] <KenMan> that is a problem in my book.
[12:43] <toad_> KenMan: what is?
[12:43] <KenMan> if MRIs are level steady, then outgoing MRIs should approx average incoming MRIs
[12:44] <toad_> only if all nodes have identical resources
[12:44] <toad_> which they don't
[12:44] <toad_> and identical request rates
[12:44] <toad_> which they also don't
[12:44] <KenMan> we don't take the implied (and wildly changing) capacity provided by inbound MRIs into our calculations of outgoing query capacity
[12:44] <toad_> KenMan: so?
[12:44] <KenMan> no, for a single node is what i meant re: in = out.
[12:44] <toad_> we can't
[12:45] <toad_> we cannot take into account inbound MRIs in outbound MRI calculation
[12:45] <toad_> if we do, things break. badly.
[12:45] <KenMan> suppose, incoming MRIs allow us to send 1723 querys in the next hour. Do we prevent our outboind MRIs from inviting more than that ?
[12:45] <toad_> no
[12:45] <toad_> because if we do then our downstream nodes accordingly adjust their MRIs
[12:45] <toad_> and then that propagates back to us
[12:45] <toad_> and very soon the whole network has maximum MRI on all routes
[12:45] <KenMan> well, computing 1723 is ridiculous. In fact, we can't compute the capacity off the fluctuating values.
[12:46] <toad_> huh?
[12:46] <KenMan> but if MRIs were more steady, we could do a LOT with that info
[12:46] <toad_> no
[12:46] <KenMan> we could produce a ... "RATE" with them. :(
[12:46] <toad_> if incoming MRI influences outgoing MRI, then there is no reason to expect it not to lead to a network wide freezedown (mathematically speaking)
[12:47] <toad_> no
[12:47] <KenMan> okay, lets avoid that specific discussion then.
[12:47] <toad_> incoming MRIs do not influence outgoing MRIs directly
[12:47] <KenMan> I know that.
[12:47] <toad_> they cannot, or we get into DEEP shit
[12:47] <toad_> now where were we?
[12:47] <KenMan> we can get in deep shit anyways :)
[12:47] <toad_> dubious arithmetic
[12:47] <KenMan> gnuplotting your MRIs , maybe
[12:47] <toad_> well
[12:47] <toad_> two instances
[12:47] <toad_> about a minute apart
[12:48] <toad_> one time we have MRI of 1 minute
[12:48] <toad_> the second time we have MRI of 2 minutes
[12:48] <toad_> the first instance and the second instance have virtually all variables very close together
[12:48] <KenMan> that is a change of 60000ms / 1 minute, right ?
[12:48] <toad_> except that in the second instance globalQuota is a little less than totalRequests, and in the first it's a little more
[12:48] <toad_> KenMan: correct
[12:49] <toad_> now, this is a natural response to globalQuota falling relative to totalRequests - we solicit fewer requests
[12:49] * ninja_ (im10ninjas@modem-2924.lion.dialup.pol.co.uk) Quit (Read error: 60 (Operation timed out))
[12:49] <toad_> this node incidentally has low traffic - totalRequestsThisNode is 42qph
[12:50] <toad_> out of a total global quota of 1kqph
[12:50] <KenMan> that is not low. that is high
[12:50] <KenMan> well, for how many peers
[12:50] <toad_> KenMan: why is this a problem, and how does it contribute to the wider difficulties?
[12:50] <KenMan> i don't think tRTN is quite right in its calcs
[12:50] <toad_> KenMan: 80 or so
[12:50] <toad_> okay, it's middling
[12:50] * ninja_ (~im10ninja@modem-3943.lion.dialup.pol.co.uk) has joined #freenet
[12:51] <toad_> I don't see any evidence here for tRTN being wron
[12:51] <toad_> g
[12:51] <KenMan> 42qph * 80 = 3360 qph, if that was the average.
[12:51] <KenMan> if 42qph were the average
[12:51] <toad_> it looks like a straightforward artifact of the current rate limiting formula
[12:51] <KenMan> clearly this node must be a heavy node.
[12:51] <toad_> either we find a way to live with it or we change the formula
[12:52] <KenMan> change the formule.
[12:52] <toad_> now, please explain why it is calamitous?
[12:52] * ninja_ (~im10ninja@modem-3943.lion.dialup.pol.co.uk) Quit (Client Quit)
[12:53] <toad_> KenMan: I need something concrete in order to change the formula
[12:53] <toad_> why is the rapid change in MRI a problem?
[12:53] <KenMan> because , if you want to write code that says the "rate" from peer A is Xqph, but it's really 2X 3X 0.5X 3X 5X 0.5X then you can't expect any kind of reliable behavior
[12:53] <toad_> no, it doesn't get that high
[12:53] * ninja_ (im10ninjas@modem-3943.lion.dialup.pol.co.uk) has joined #freenet
[12:53] <toad_> it oscillates around roughly where it should be
[12:53] <toad_> it might go from 1X to 2X
[12:54] <toad_> maybe 0.5X to 2X
[12:54] <KenMan> but the amplitude of oscillation is too high
[12:54] <KenMan> a slight change in 42qph results in big differences in some of your maths
[12:55] * KenMan is just arguing out his backside, based on 'educated intuition' ...
[12:55] <toad_> 44170.14622331551 to 166416.24118165512
[12:55] <toad_> so a factor of 4 - 0.5X to 2X
[12:55] <KenMan> with how much time between ?
[12:56] <toad_> that's over the whole hour
[12:56] <KenMan> n/m
[12:56] <toad_> though there is one really fast change:
[12:56] <toad_> DATE 3:04:17 AM VALUE 166416.24118165512
[12:56] <toad_> DATE 3:04:19 AM VALUE 57973.455695235985
[12:56] <toad_> and another one:
[12:56] <toad_> DATE 3:05:37 AM VALUE 127088.51320605753
[12:56] <toad_> DATE 3:05:38 AM VALUE 68531.99160260695
[12:56] <toad_> and another one:
[12:56] <toad_> DATE 3:07:57 AM VALUE 144728.6970710296
[12:56] <toad_> DATE 3:08:03 AM VALUE 44170.14622331551
[12:56] <KenMan> that's not for the same peer, is it ?
[12:56] <KenMan> you need to focus on single links.
[12:57] <toad_> and probably some more
[12:57] <toad_> this is a single link
[12:57] <KenMan> good
[12:57] <KenMan> a wiggly-factor of 2 can be destructive in some of the other variables used
[12:57] <KenMan> 4 is not helpful.
[12:58] <toad_> KenMan: do you have ANY evidence that short term fluctuations are a problem?
[12:58] <toad_> whether actual evidence or solid theory?
[12:58] <KenMan> I am really going to try to give you some...
[12:58] <toad_> or are you just wasting my time by focussing on the wrong oscillations?
[12:58] <KenMan> yes, that's my plan ;)
[12:58] <toad_> my concern is the large-scale globalQuota oscillations, mainly
[12:58] <KenMan> yes, it should be right now...
[12:58] <KenMan> let me go play with MRIs...
[12:59] <KenMan> but don't expect me to let this issue go anytime soon ...
[12:59] * KenMan is like a dog with a bone
[12:59] <toad_> well I want to fix the problem that there is evidence for
[12:59] <KenMan> agreed
[13:00] <toad_> it's quite clear to me that globalQuota oscillating all over the place is a problem
[13:00] <toad_> or should be a problem
[13:00] <KenMan> and we need to identify a cause
[13:00] <toad_> I'm not so convinced about link-local short term moderate magnitude oscillations are a problem
[13:00] <toad_> okay
[13:00] <KenMan> i think it is all caused by somebody's spider run-amok
[13:01] <toad_> KenMan: I don't think so
[13:01] <toad_> anyway
[13:01] <toad_> you're going to go do something
[13:01] <KenMan> thanks for reminding me :)
[13:01] <toad_> should I stay here and try to do some more work, or skive off and go to aikido despite not having accomplished anything today?
[13:01] <KenMan> it could clear your mind...
[13:02] <toad_> (partly because of getting up late, partly because of the fucking weather, partly because of the fan being in the wrong orientation)
[13:02] <toad_> well maybe so, but by the time I get back I don't know how much work I'd get done in the evening
[13:02] <toad_> sometimes I get good stuff done
[13:02] <KenMan> don't sweat it for another day. But don't leave stable alone for too many days.
[13:02] <toad_> it partly depends on the status of FOOD :)
[13:03] <KenMan> :)
[13:03] <toad_> okay
[13:03] <toad_> bbl
[13:03] <KenMan> besides, your workday schedule has only begun, no ??
[13:03] <toad_> ~ 22:00 GMT, at the earliest
[13:03] <KenMan> come back in 6 hours...
[13:03] <toad_> errr BST
[13:05] <KenMan> if even one single 5087 node were still out there, it /could/ have a majorly bad effect. But _this_ bad ??
[13:22] * jayo (jcl@ool-18bf6dac.dyn.optonline.net) has joined #freenet
[13:23] * jayo is now known as jay
[13:26] * ninja_ (im10ninjas@modem-3943.lion.dialup.pol.co.uk) Quit (Nick collision from services.)
[13:26] * salahx (salahx@sc1-24.217.174.147.charter-stl.com) has joined #freenet
[13:26] * ninja_ (im10ninjas@modem-2401.fruitbat.dialup.pol.co.uk) has joined #freenet
[13:31] <iip_i2p> <mule2p> KenMan, I count 6 5087 nodes in my connections. And even a 5076 is still listed.
[13:33] <iip_i2p> <mule2p> KenMan, and of course quite a lot of 5084 nodes.
[13:45] * salahx (salahx@sc1-24.217.174.147.charter-stl.com) Quit ()
[14:28] * hobx (~hobx@h196n1fls21o1077.bredband.comhem.se) has joined #freenet
[14:28] * ChanServ sets mode +o hobx
[14:37] <KenMan> well, not much we can do about that part, except try to create fewer poison-node releases in the future :)
[14:38] * KenMan has stumbled upon a clever solution to MRI rate control.
[14:39] * KenMan will probably have to implement it himself though, with the toad's permission...
[14:40] <KenMan> my biggest contention with query rate limiting is that is produces WAY too high a degree of backoff
[14:42] <KenMan> my clever idea *almost* guarantees 50% backoff , which is not so great, but it would be an improvement.
[14:43] <KenMan> 50% backoff might not be so bad, if it is consistently at (or near) 50%. This means on average we would get our second choice node.
[14:43] <KenMan> recently, backoffCount has been around 2, which means third choice on average.
[14:46] <KenMan> gack - average incomingRequestInterval is ALL OVER THE PLACE. This means that backoffCount runs at first choice for a little while, then at last choice for a while. Or something like that anyway. It is NOT consistent.
[15:20] <iip_i2p> <mule2p> KenMan, I don't know of the details. However on a recent fuqid download i get 11 blocks succeeded, 6600 DNF and 17300 RNF. Not sane, I guess.
[15:20] <KenMan> no, definitely not sane :(
[15:23] <iip_i2p> <mule2p> What I further observe is that on one of my nodes the no of open connections is down to around 60, used to be 200 for 5084.
[15:26] <iip_i2p> <mule2p> Is the number of nodes it tries before giving up a function of the open connections? Fproxy indicates RNF after 18 nodes tried today, was around 35 yesterday.
[15:27] <iip_i2p> <mule2p> And it almost always indicates RNF :(
[15:28] <KenMan> sort of. Someone seemed to believe, if enough routes are not available, that means we just need more peers... :p
[15:31] * ninja_ (im10ninjas@modem-2401.fruitbat.dialup.pol.co.uk) Quit (Nick collision from services.)
[15:31] * ninja_ (im10ninjas@modem-3059.leopard.dialup.pol.co.uk) has joined #freenet
[15:39] * guido^pe (~unknown@dsl-213-023-243-202.arcor-ip.net) has joined #freenet
[15:42] <KenMan> I apologize for my presentation skills, anyway - http://mywebpages.comcast.net/jkcorson/FreeNet/queryRateControl.png
[15:42] <KenMan> sanity: thoughts ??
[15:46] <iip_i2p> <west234> can someone help
[15:46] <KenMan> west234 - probably not right now. What is wrong ?
[15:51] <iip_i2p> <west234> does'n work 5088 on win
[15:52] <iip_i2p> <west234> it cannot start node
[15:57] <KenMan> west234: do you have java ? do you have any way to determine what *does* happen ? can you locate a freenet.log file anywhere on your system ?
[15:57] <KenMan> if so, do the dates/times reflect the time when you try to start it up ?
[16:03] <KenMan> re: queryRateControl - we could simply redefine the use of mRI as "moderated Rate Interval" , thus no need to reset the network. Of course, code changes are needed. There are several assumptions rolled into the proposal. But I think the idea is sound, anyway.
[16:03] <iip_i2p> <west234> ok i try to expl. 5083 works fine even now. i download new freenet.jar and freenet-ext.jar + seed. But then it starts, after 15 min HDD always works and nothing happens
[16:04] <KenMan> wow. How large is your datastore ?? You cannot access the fproxy web page ? Please paste the last 2 lines from your freenet.log file here.
[16:06] <KenMan> re: queryRateControl - if we only allow small or no change in X , we would have perfect rate control.
[16:07] <KenMan> the biggest problem that interferes with this scheme is - transport time. But, clearly, the receiver will get the mRI after the sender has started his measurement of the interval period. It might need sequence numbers or something. But since we already send an acknowledgement on (almost) every query already...
[16:08] <KenMan> we wouldn't need to update X, mRI sender would simply drop/reject every "2nd onwards" query within the window.
[16:10] <KenMan> with 100 peers, and query levels like 2000qph, the normal interval should be near 2000/100 = 20qph = 3 minutes :o
[16:11] <KenMan> one of the current problems with rate limiting is that it doesn't work with small numbers of peers. I believe my scheme would work equally well with any number of peers.
[16:11] <KenMan> of course, the choice of the interval is very important , and should take number of peers into account, if a target qph level is being attempted.
[16:38] * ninja_ (im10ninjas@modem-3059.leopard.dialup.pol.co.uk) Quit (Remote closed the connection)
[16:38] <iip_i2p> <west234> can someone help with 5088 and WIN
[16:38] * greycat (~wooledg@192.35.79.70) Quit ("Lick Bush in '04")
[16:39] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[16:39] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[16:39] * ninja_ (im10ninjas@modem-3059.leopard.dialup.pol.co.uk) has joined #freenet
[17:30] * ninja_ (im10ninjas@modem-3059.leopard.dialup.pol.co.uk) Quit (Nick collision from services.)
[17:30] * ninja_ (im10ninjas@modem-2995.lion.dialup.pol.co.uk) has joined #freenet
[17:32] <mazzanet> KenMan: that pic is somewhat interesting and at the same time confusing
[17:33] <KenMan> yes, it is a little short on details... i just think it is an easier model to work with.
[17:34] <KenMan> i am leveraging the random distribution of query arrivals against the ever-increasing degree of backoff we all experience.
[17:34] <KenMan> or attempting to, anyway :)
[17:37] <KenMan> it is the *node's* responsibility to go out and get as many peers as is computationally possible. Therefore, 2nd choice routing will not hurt much. If each node could have, say, 8000 peers, then 1st and 2nd choice would be virtually indistinguishable :p
[17:38] <KenMan> now, i will state something (serious) that toad will have trouble with - if the supplied period for each route were very close to the same value, each backoffCount event would be very close to 1.
[17:39] <KenMan> The more you let them wander off and do their own thing, the more the backoffCount events will vary
[17:40] <KenMan> i can't draw a picture or supply a 10 line mathematical proof, but it is true. "You can trust me"
[17:41] <KenMan> the average will remain at 1, but we also should be trying to reduce the standard deviation for backoffCount
[17:41] * smaster_ (~smaster@ppp-59-161.25-151.libero.it) has joined #freenet
[17:44] <KenMan> aha! toad will nail me on a technicality... "but backoffCount is a discrete value - how close can you get ??"
[17:45] <KenMan> ;)
[17:52] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Nick collision from services.)
[17:52] * Ash-Foxeus (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[17:54] <KenMan> to concisely summarize my observations, we need to CONTROL totalRequestsPerHour in PeerHandler , not just make a rough measurement of it (and attempt to adapt)
[17:58] * Ash-Foxeus (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[17:58] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[17:59] <KenMan> RouteNotFound causes most of the imbalance between incomingRequests and outgoingRequests
[17:59] <KenMan> getting a hit on the datastore helps, but not nearly enough to compensate for this
[18:12] * interrupt (~chatzilla@64.122.23.213) Quit ("Chatzilla 0.9.64b [Mozilla rv:1.7/20040707]")
[18:19] * guido^pe (~unknown@dsl-213-023-243-202.arcor-ip.net) Quit (Remote closed the connection)
[18:36] <KenMan> it would be nice to know how many RNFs are due to looping, and how many are caused by route suffocation (100% backoff) !!
[18:37] * greycat (rfc1413@wooledge.org) has joined #freenet
[18:37] <KenMan> me-ouwww !
[18:37] <greycat> I guess that's better than croaking.
[18:38] <KenMan> and it hurts less than ribbitting, too...
[18:42] * lolo-laptop (~lostlogic@68.251.84.226) Quit ("Client exiting")
[18:45] <toad_> looping?
[18:46] <toad_> RIBBIT!
[18:56] <toad_> <KenMan> my clever idea almost guarantees 50% backoff , which is not so great, but it would be an improvement. - uh oh... you're talking about feedback from incoming RIs to outgoing RIs... I've explained why that won't work...
[18:58] * toad_ (toad@82-32-16-91.cable.ubr03.azte.blueyonder.co.uk) Quit ("using sirc version 2.211+KSIRC/1.3.10")
[18:58] * toad_ (toad@82-32-16-91.cable.ubr03.azte.blueyonder.co.uk) has joined #freenet
[18:58] * toad_ (toad@82-32-16-91.cable.ubr03.azte.blueyonder.co.uk) Quit (Read error: 104 (Connection reset by peer))
[18:58] * toad_ (toad@82-32-16-91.cable.ubr03.azte.blueyonder.co.uk) has joined #freenet
[18:58] * salahx (salahx@sc1-24.217.174.147.charter-stl.com) has joined #freenet
[18:58] <toad_> aaaaargh!
[18:58] * toad_ lost his buffer...
[18:59] <toad_> KenMan: I don't understand what you are doing on http://mywebpages.comcast.net/jkcorson/FreeNet/queryRateControl.png
[18:59] <iip_i2p> <oOo> The one with the answer to all freenet's problems ? ;)
[18:59] <toad_> the last part doesn't make sense
[19:00] <toad_> oOo: if you say so
[19:00] <iip_i2p> <oOo> J/K...
[19:01] <toad_> anyway
[19:01] <toad_> WHERE IS KENMAN?!
[19:01] <toad_> :<
[19:01] * smaster_ (~smaster@ppp-59-161.25-151.libero.it) Quit ("https://bfi.s0ftpj.org")
[19:01] <iip_i2p> <oOo> Don't you log this channel ?
[19:01] <toad_> ransfers active (Transmitting/Receiving) 56 (14/42)
[19:01] <toad_> Transfers active (Transmitting/Receiving) 56 (14/42)
[19:02] <toad_> good point...
[19:02] <toad_> except i think it's down at the moment :<
[19:03] <toad_> now, the problem with my theory that KenMan's excessive transmit counts are fictitious, is that mine are pretty low
[19:03] <toad_> this suggests they are genuine
[19:03] <salahx> i got the same thing
[19:03] <salahx> 80+ tramnfers at once!
[19:04] <toad_> <KenMan> sanity: thoughts ?? - you can't expect sanity to comment on mundane freenet issues relating to saving the universe from censorship, he's a busy man :)
[19:05] <toad_> <KenMan> re: queryRateControl - if we only allow small or no change in X , we would have perfect rate control. - bollocks :)
[19:05] <toad_> if we allow no change in MRI, MRIs might work better
[19:05] <toad_> however they'd either chronically overload or chronically underload the node using them
[19:06] <toad_> [16:08] <KenMan> we wouldn't need to update X, mRI sender would simply drop/reject every "2nd onwards" query within the window.
[19:06] <toad_> [16:10] <KenMan> with 100 peers, and query levels like 2000qph, the normal interval should be near 2000/100 = 20qph = 3 minutes :o
[19:06] <toad_> [16:11] <KenMan> one of the current problems with rate limiting is that it doesn't work with small numbers of peers. I believe my scheme would work equally well with any number of peers.
[19:06] <toad_> [16:11] <KenMan> of course, the choice of the interval is very important , and should take number of peers into account, if a target qph level is being attempted.
[19:06] <toad_> just like KenMan to come up with some not well documented idea to solve all freenet's problems and then sod off
[19:08] <toad_> <KenMan> to concisely summarize my observations, we need to CONTROL totalRequestsPerHour in PeerHandler , not just make a rough measurement of it (and attempt to adapt) - there may be some truth in that, but it is probably impossible
[19:09] * toad_ changes topic to 'Upgrade to 5088 | Unstable: Upgrade to 60178 | Logs: http://newton.matcmp.ncc.edu/~lockej/freenet/chanlog/ | KenMan: please read the logs'
[19:13] <toad_> I'm sure if I understood what KenMan is trying to say I'd be able to instantly ridicule and demolish it
[19:13] <toad_> but it's infuriating not having him here to explain it
[19:13] <KenMan> hello
[19:13] <salahx> i'm loooking at it
[19:13] <salahx> speak of the devil
[19:13] <toad_> KenMan: hey!
[19:13] <toad_> could you please explain what is going on on the png you posted?
[19:14] <toad_> afaics we can change it from "0 reqs in next N millis" to "1 reqs in next N millis" simply by abolishing the non-averaged test in StandardNodeEstimator.isAvailable
[19:14] <toad_> I don't see why this would help things any though
[19:15] <toad_> our current test is: a) time since last req sent > MRI, AND b) average time since last req sent for last (not sure number) > MRI
[19:15] * toad_ waits for acknowledgement from KenMan, who is obviously situated in geostationary orbit
[19:16] <KenMan> i'm here, i am not good at explaining things. IF we made interval Xms a stable value, and we adjusted it properly, then we would get 50% backoff.
[19:17] <toad_> why?
[19:17] <toad_> it sounds like fuzzy thinking to me
[19:17] <KenMan> i believe that some of the rate limiting algorithms attempt to minimize Y. Thus maximizing backoff.
[19:17] <KenMan> i don't *know* that to be true, I'm just guessing.
[19:17] <toad_> if you send the requestor "you can send me requests every X millis", then if he wants to send more than that, he will backoff
[19:18] <KenMan> yes, but the thing I'm not good at explaining about relates to the "make small changes to rates issue."
[19:18] <KenMan> queries arrive on their own time, and thus they can only choose from those routes which are not backed off.
[19:18] <toad_> KenMan: there are fundamental issues with that
[19:18] <toad_> point 1. We do not know what the ideal rate is!
[19:18] <toad_> point 2. See point 1.
[19:18] <toad_> point 3. Not all nodes are up forever.
[19:19] <toad_> there is no point 4.
[19:19] <KenMan> you surely do a herculean effort of approximating and attempting to guess what the ideal rate is at any instant in time...
[19:19] <toad_> KenMan: okay, so you are saying that if we average it, we are more flexible as regards individual timings
[19:19] <toad_> that is perfectly true
[19:19] <toad_> however if we have a constant stream of requests we will still get backed off
[19:20] <KenMan> thus we need to BACK THEM OFF
[19:20] * lostlogic (~lostlogic@node-402418b2.mdw.onnet.us.uu.net) has joined #freenet
[19:20] <toad_> currently it's a combination of a hard limit and an average
[19:20] <KenMan> we DO have very good control (over a longish time period) of how many queries will come in. The ONLY tool we have for that is MRI.
[19:20] <toad_> if we eliminate the hard limit, we'd behave slightly better in some situations by your estimation
[19:20] <toad_> but only at the beginning
[19:20] <toad_> KenMan: huh?
[19:21] <toad_> yup
[19:21] <toad_> see the graphs
[19:21] <toad_> the total incoming queries follows globalQuota very closely
[19:21] <toad_> that part of rate limiting is working
[19:21] <toad_> what we need to do is decide how to calculate globalQuota accurately and stably
[19:21] * toad_ is trying to be constructive here...
[19:22] * KenMan is trying very hard also... this is one complex gob of code
[19:22] <toad_> one way to do this is to have an average without an event horizon
[19:22] <toad_> but then if things DO change, we want to react to them
[19:22] <toad_> and if you take out the event horizon, reacting to change is REALLY slow
[19:23] <KenMan> i think we react too fast, too much. Too much for others to react to us.
[19:23] <toad_> as regards backoff, the only way I can see for backoff to be dealt with by rate limiting is for us to factor it in in our outgoing MRIs, and I've yet to be convinced that that won't be catastrophic
[19:23] <toad_> KenMan: hmm?
[19:23] <toad_> KenMan: just how long do you want to make our reaction times? Currently they are pretty long
[19:24] <toad_> if we make them much longer we completely disenfranchise anyone whose node hasn't been up for WEEKS
[19:24] <toad_> and make things very difficult to test
[19:24] <toad_> i'm talking globalQuota here of course
[19:24] <KenMan> okay, i accept "factor it in our outgoing MRIs" as a good enough reaction. I'll stop wasting your time now.
[19:24] <toad_> KenMan: eh?
[19:25] <toad_> you'll find that that works well on a single node
[19:25] <toad_> but once widely deployed, it causes the entire network to screech to a halt
[19:25] <KenMan> under what circumstances can a node get sooo constipated that it has no choice but to RNF ??
[19:25] <toad_> KenMan: that depends not only on backoff but also on the downstream nodes' responses
[19:25] <toad_> if they all RNF (or otherwise reject), then we RNF even though backoff << rtsize
[19:26] <toad_> especially given that we spend an HTL hop on each one we route to
[19:26] <KenMan> no, i mean, under what circumstances can a node be left holding the bag, as in "no one will let me talk to them, sorry"
[19:26] <toad_> okay, either we have no connections, or all our nodes are backed off at this instant
[19:26] <KenMan> where backoff = rtsize ??
[19:26] <toad_> (sooner or later some will become contactable)
[19:27] <KenMan> yup, that one.
[19:27] <toad_> right
[19:27] <KenMan> and this situation is probably very active only on certain nodes...
[19:27] * toad_ changes topic to 'Upgrade to 5088 | Unstable: Upgrade to 60178 | Logs: http://newton.matcmp.ncc.edu/~lockej/freenet/chanlog/'
[19:27] <toad_> hmm?
[19:27] <KenMan> weak, slow nodes are much more likely to get into that condition
[19:27] <toad_> well, I don't think we can really attack backoff from this anglre
[19:28] <toad_> from past experience
[19:28] <toad_> if you have a mechanism for incoming MRIs to influence outgoing MRIs that you can PROVE won't cause a network freezedown, we should implement it
[19:28] <toad_> KenMan: are they?
[19:28] <toad_> I would have thought they'd get low traffic
[19:28] <toad_> although they'd have few connections
[19:29] <KenMan> It should be possible for us to massage and manage backoff such that it sits at a fairly steady percentage
[19:29] <toad_> indeed it is
[19:29] <KenMan> i don't know how to make it happen in the fastest manner possible, that is all.
[19:29] <toad_> we just need to take it into account in load
[19:29] <toad_> the problem is all attempts at doing this have been catastrophic
[19:30] <KenMan> all your emails say "Nothing is Impossible" and attribute that to someone else. :p
[19:30] <toad_> at this point anything less than a mathematical proof (or conceivably a realistic simulation) will not persuade me
[19:30] <KenMan> that's fair. I can't produce mathematic proofs :(
[19:30] <toad_> I'm not saying it's impossible. I'm saying I have no idea how to do it.
[19:30] <toad_> well can you produce a convincing logical argument?
[19:31] <toad_> also...
[19:31] <KenMan> I thought I had one in that queryRateControl idea, but it confused other people as well. So, I didn't make the case (or details) sufficiently.
[19:31] <toad_> I don't think 50% backoff is necessarily good enough
[19:31] <KenMan> it may be all you can get, without freezedown
[19:32] <toad_> KenMan: afaics your suggestion is only marginally different to what we have now and won't cause any major change in behaviour
[19:32] <salahx> So basically what touy want to do is have the mRI such that there are 50% rotuer aviabnle when it hits the next node ?
[19:32] <KenMan> or everyone collaborating on enforcing a global 10 qph :(
[19:32] <toad_> well it ought to be possible to move the load back to the content sources
[19:32] <KenMan> yes, i said as much in my email to you. The idea was to present the problem in a different form, hopefully one that could be worked with a little easier.
[19:32] <toad_> KenMan: the problem is if you enforce it, you have backoff
[19:33] <toad_> because that's your means of enforcing it!
[19:33] <KenMan> IF we could magically pluck the proper value for X from the sky, and stick to it, THEN do you believe it would produce 50% backoff for that one route ?
[19:33] <toad_> there might be something in the idea of calculating what our capacity is and using that as a fixed value
[19:33] <toad_> KenMan: nope
[19:34] <toad_> KenMan: the amount of backoff it produces is simply due to the amount of traffic that wants to go down that route
[19:34] <KenMan> 50% backoff over time, for that one route. Why do you disagree. How can you disagree ?
[19:34] <toad_> and the amount of traffic that route can accept
[19:34] <toad_> KenMan: how can you believe such a fallacy? :)
[19:34] <KenMan> wink
[19:34] <toad_> your explanation on the png is not very explanatory sadly :)
[19:35] <KenMan> i know... i know. The critical variable is the correct choice of X.
[19:35] <toad_> with regards to what?
[19:35] <toad_> look, if a link has a capacity of 10 requests
[19:35] <KenMan> causing the amount of time spent in the backedOff condition to settle at 50%.
[19:35] <toad_> then no amount of magic and thinking around things will squeeze 20 requests in
[19:36] <toad_> KenMan: that is not a practical change, that is woolly thinking and redefining the problem
[19:36] * Shaun-Away is now known as ShaunMacPherson
[19:36] <toad_> what is t1?
[19:36] <salahx> 1.544 Mbps/s
[19:36] <salahx> errr 1.544 Mpbs
[19:36] <toad_> hehe
[19:36] * ninja_ (im10ninjas@modem-2995.lion.dialup.pol.co.uk) Quit (Read error: 60 (Operation timed out))
[19:36] <toad_> KenMan: what is t1?
[19:36] <salahx> in the US anyway
[19:36] <toad_> salahx: aka one primary ISDN line?
[19:36] <KenMan> t0 (time of communication of X) to t1 is the amount of time spent in the backed off condition
[19:36] <salahx> yep
[19:37] <toad_> KenMan: what does point t1 in time represent?
[19:37] <KenMan> t1 is the time at which that route becomes unconstipated
[19:37] <toad_> what event happens at t1?
[19:37] <KenMan> route becomes available (in today's current model).
[19:37] <toad_> okay so where is X?
[19:37] <KenMan> X = t1 - t0 in todays model
[19:38] <toad_> I am referring to the bottom half
[19:38] <toad_> what is going on there?
[19:38] <toad_> we send X = 30 seconds
[19:38] <KenMan> that means that the receiver is alloted one request at any point in time for 30 seconds (after reception)
[19:39] <toad_> then we can have a request any time during that 30 seconds? is that the idea? with the start of the 30 seconds being the previous request?
[19:39] <KenMan> assume reception time=send time for simplicity
[19:39] <toad_> or rather the expiration of the last backoff time
[19:39] <KenMan> no, t1 being the start of the next OK to send interval
[19:39] <KenMan> so the route would be available during t0 -> t2 , and "backed off" for t2 -> t1 (using the picture)
[19:40] <toad_> how is that different to today?
[19:40] <toad_> if we haven't sent any requests, it's similar...
[19:40] <KenMan> in many ways it is not. But, do you see where 'Y' is in the top half ?
[19:40] * ninja_ (im10ninjas@modem-1174.lion.dialup.pol.co.uk) has joined #freenet
[19:41] <toad_> hi ninja_
[19:41] <toad_> having problems?
[19:41] <ShaunMacPherson> hi
[19:41] <toad_> KenMan: I don't see why the two are significantly different
[19:41] <toad_> it's just the average versus minimum thing
[19:41] <KenMan> it is just a different representation. We are arguing the same thing here... avg vs min - exactly !!
[19:42] <toad_> it's not exactly the same as the average thing... or is it?
[19:42] <KenMan> now, how could this benefit us ?
[19:42] <toad_> it won't benefit us significantly. it will result in increased traffic.
[19:42] <toad_> for a given MRI
[19:42] <toad_> and any imagined benefits are just that - imagined
[19:42] <KenMan> well, spend a smallish amount of time comparing the two, and if ANY revelations result, then I have helped. Otherwise, I am sorry for wasting your time.
[19:42] <toad_> if the line is busy, it will be backed off
[19:43] <KenMan> i am busy dreaming of a world where rates are fairly constant... :(
[19:43] <toad_> KenMan: so help me build that world
[19:44] <toad_> I doubt that averaging it instead of using a combination of fixed intervals and averages will make much difference
[19:44] <toad_> but finding a stable way to calculate globalQuota will make a big difference
[19:44] <KenMan> well, the simplest thing you can try is to set a minimum value for MRI, one that they cannot go below. somewhere around 5 to 15 seconds.
[19:44] <toad_> unfortunately we cannot simply decide "we have enough bandwidth for exactly N requests per hour", because (apart from the obvious variables), we have other load factors than bandwidth
[19:45] <KenMan> this is only a violation of logic when it is used between two nodes that can support 10K qph between them anyway
[19:45] <toad_> KenMan: yuck
[19:45] <toad_> and IMHO that won't actually prevent major oscillations
[19:45] <toad_> oh, have you found evidence of new nodes being assigned very low MRIs yet?
[19:45] <toad_> if so that's a bug we can fix
[19:46] <KenMan> what exactly results when a route uses an MRI below , say , 1 second ??
[19:46] * toad_ would expect greedy/busy nodes to get good MRIs
[19:46] <toad_> KenMan: it gets lots of requests
[19:46] * KenMan on phone...
[19:46] <toad_> KenMan: if you have proof that we are assigning very low MRIs to new nodes, please share it
[19:47] <toad_> there is some good news... requestCompletionTime is usually below 1,000,000ms on my node, in hourly averages
[19:48] <toad_> of course my node is underloaded for its bandwidth...
[19:49] * Gunnar (~gunnar@34.80-203-209.nextgentel.com) Quit (Remote closed the connection)
[19:51] <toad_> KenMan: please ping me/tell me when you're available
[19:51] * KenMan will do
[19:54] <KenMan> toad_: i'm back. I accidentally got a 7G freenet.log file, apparently it did NOT pick it up when I removed that special logDetail setting you gave me
[19:54] <toad_> :<
[19:55] <toad_> KenMan: okay
[19:55] <toad_> ideas for globalQuota?
[19:55] <KenMan> i changed it to 'logLevelDetail=' in the file , but it overflowed my disk when I wasn't looking. So I do have lots of data to process
[19:55] * KenMan sheepishly admits he doesn't understand what globalQuota is *supposed* to represent
[19:56] * KenMan hands toad a stick to hit him with
[19:57] <KenMan> okay, i'll take a stab in the dark. globalQuota is supposed to represent how many queries we have the capacity to send out. No, that's not right.
[19:57] <toad_> almost
[19:57] <KenMan> globalQuota represents the total number of queries we want to take in ??
[19:57] <toad_> globalQuota is our capacity for RECEIVING and processing queries
[19:57] <toad_> right
[19:58] <toad_> incoming queries closely approximates it
[19:58] <toad_> which means that that part of rate limiting is working and should not be touched
[19:58] <KenMan> other way round u mean
[19:58] <toad_> what we need to hack at is globalQuota
[19:58] <toad_> KenMan: partly that too
[19:58] <toad_> they closely approximate one another
[19:58] <KenMan> sorry, i will only accept one or the other. I can't handle it both ways
[19:58] <toad_> well...
[19:59] <toad_> right now globalQuota is incoming queries / load
[19:59] <toad_> more or less
[19:59] <toad_> so that's the relationship
[19:59] <KenMan> but incoming queries IS load (in the abstract) , no ?
[19:59] <toad_> however, the intent is that we adjust MRIs so that we expect to get globalQuota requests total
[19:59] <toad_> KenMan: no :)
[20:00] <toad_> load can be caused by all sorts of things - external CPU or bandwidth stresses, or equally often, requests taking bloody ages to finish
[20:00] <toad_> 5088_1kqph_c.png suggests maybe there IS a problem in the globalQuota -> MRI subsystem...
[20:01] <toad_> incoming queries seems highly variable
[20:01] <toad_> but it is variable over the short term... and it does seem to center on the target 1kqph line
[20:01] <KenMan> remember, I set MRIsOut ~= 216s there (1000qph divided over no. peers)
[20:02] <KenMan> no, it centers on 10 qpm. 600 qph. 60 percent utilized/backedOff
[20:02] <toad_> KenMan: you forced it? I thought you set globalQuota = 1000qph?
[20:02] <toad_> hmmm
[20:02] <KenMan> no, i compute it right there on the fly. It's a pretty damned basic experiment.
[20:02] <toad_> well it'd be useful to try stock code with Node.globalQuota() fixed to return 1000
[20:02] <toad_> that would establish whether that part of the system is working
[20:02] <KenMan> for you perhaps ;) as you understand the guts a little better...
[20:02] <toad_> (given that the forced value is actually feasible)
[20:03] <toad_> KenMan: yes, for me :)
[20:03] <toad_> for both of us really, because if that part doesn't work, we've got problems that are probably fixable
[20:03] <toad_> even if not easily so
[20:03] <toad_> we'd just need to fix the algos etc that go from globalQuota to individual MRIs
[20:04] <KenMan> okay, i can draw you nearer my line of thought - it would be interesting to see utilization rate, if I used 2000qph. Or 500 qph.
[20:04] <toad_> that's a daunting prospect but if there's a problem it will be fixable (I hope)
[20:04] <toad_> well, if you use 2000qph, you'll probably have way too many requests
[20:04] <toad_> you'll end up rejecting some, and having crazy numbers of trailers :)
[20:05] <KenMan> so there is some optimum level "out there" that is fairly constant...
[20:05] <toad_> I hope so
[20:05] <KenMan> at least, I hope it is fairly constant.
[20:05] <toad_> but we may have to react to short term changes too which is the real PITA
[20:05] <KenMan> you say reactO, I say controlO ;)
[20:06] <toad_> control HOW?
[20:06] <toad_> we cannot control external stimuli
[20:06] <toad_> we cannot control the user loading up Quake
[20:06] <KenMan> we need to add an ESP module into the MRI calculation :)
[20:06] <toad_> we cannot control requests that have already started
[20:06] <toad_> we cannot even reliably predict whether they will succeed
[20:06] <KenMan> sorry, i ONLY consider queries and transfers as "load" in my dream world.
[20:06] <salahx> you man ffrmo that java.phsycic package sanity keep talking about?
[20:07] <toad_> salahx: lose the h
[20:07] <KenMan> that the one :)
[20:07] <toad_> java.psychic
[20:07] * Pascal (Pascal@c-24-13-17-191.client.comcast.net) has joined #Freenet
[20:07] <toad_> interface PsychicProvider is the main interface class :)
[20:07] <Pascal> howdy
[20:07] <toad_> hi
[20:08] <KenMan> part of my conversion for mRI calculation attempted to confront the random external stimuli behavior
[20:08] <toad_> KenMan: okay, so your argument is that the main load factor is bandwidth, so we figure out how many requests we can do per hour
[20:08] <toad_> then we modulate it by transient load
[20:08] <KenMan> basically, at least, that sounds spot on
[20:09] <toad_> unfortunately that makes several hideous assumptions...
[20:09] <KenMan> oh wait, do they say 'spot on' in england ? they sure don't do it here in the USA ...
[20:09] <toad_> e.g. long term CPU usage is negligible for queries
[20:09] <toad_> KenMan: in dubious SF sit-coms, they do (at least one idiot does)
[20:09] <toad_> occasionally
[20:10] <KenMan> okay... thanks. I wanted to make sure i didn't try to use it with an african, or a german, then.
[20:10] <toad_> okay well
[20:10] * KenMan shuffles that away for future reference
[20:10] <toad_> the problem here is that you don't accept that factors other than bandwidth are dependant on request load
[20:11] <toad_> whereas our current approach is to assume that they are linearly proportionate to it
[20:11] <KenMan> no, i really don't. I try to live in a simple world...
[20:11] <toad_> IMHO a good part of CPU usage is dependant on request load
[20:12] <toad_> if we have a routingTime of 2000ms and a messageSendTimeRequest of 5000ms, then we should cut down our request load
[20:12] <KenMan> bandwidth is the most precious resource, by far (until you 'tune up NGR' some more) - jab
[20:12] <toad_> well even bandwidth...
[20:12] <toad_> the limits may be wrong, or not set at all
[20:12] <toad_> messageSendTimeRequest gives us a reasonable idea re both bandwidth and CPU
[20:12] <toad_> other things may be running on the link
[20:13] <toad_> now, you could argue that that's not our problem...
[20:13] <KenMan> we will figure out bw limits in any case, if the current algorithms were 100% bug-free...
[20:13] <toad_> KenMan: huh?
[20:13] <KenMan> all this constant adjusting in reaction to this or that <- not a jab
[20:14] <KenMan> let's do the static globalQuota dealie ...
[20:14] <toad_> additionally we are probably wrong about our estimation of how much bandwidth is going to be used by one request... we can perhaps fix that but there are things we can't account for e.g. announcements, inserts, etc
[20:15] <toad_> KenMan: I need something watertight because I'll need to justify it to sanity
[20:15] <KenMan> and my disk is going to be getting very warm, extracting 10MBs of MRI data from 7GB of logs... :)
[20:15] <toad_> he invented the current assumptions/globalQuota estimation algo
[20:16] <lostlogic> I told a friend about your psychic class, and we've started defining various religions as OO constructs :)
[20:16] <toad_> lostlogic: heh
[20:16] <KenMan> heehee, sounds interesting
[20:16] <toad_> class OperatingThetan extends Scientologist implements Blackmailable, HighContributionIndividual :)
[20:18] <KenMan> I thought inserts obeyed MRIs, you explicitly told me that. Oh, "accounting for them" . Gotcha.
[20:18] <toad_> KenMan: well basically that's it - we can try to adjust it on the fly according to what happens; if we do, we have to decide how fast to do it. Or we can try to force something based on various crazy assumptions and approximations as a fixed value.
[20:18] <toad_> KenMan: inserts do obey MRIs. but they use a lot more bandwidth than regular requests do.
[20:18] <toad_> at least they use a lot more transfers
[20:19] <KenMan> so a big monster is out there eating inserts ??
[20:20] <salahx> that probalby explains why my node seesm to be getting so much gas then
[20:20] * toad_ observes t3c...
[20:20] <KenMan> or is it just that the inserts are getting themselves lost in the fray ?
[20:20] <toad_> we have a spike in requests, and THEN we have a spike in load...
[20:20] <toad_> KenMan: hrrm?
[20:21] <KenMan> where are failed inserts going to, again ??
[20:21] <toad_> uh, they fail :)
[20:21] <toad_> the great bitbucket in the sky...
[20:22] <toad_> i dunno maybe there's more to it than that
[20:22] <KenMan> heh
[20:22] <KenMan> i have two other ideas that might help out (longer term)... one is to /slightly/ reciprocate MRI values over a single link.
[20:23] <toad_> reciprocate?
[20:23] <iip_i2p> <Newsbyte> indeed
[20:23] <KenMan> The other is to backoff requestors (our peers) when we start browsing , to alleviate RNFs due to FProxy use
[20:23] <toad_> Newsbyte: indeed!
[20:23] <toad_> KenMan: yuck
[20:23] <iip_i2p> <Newsbyte> so, toad, how's the new build doing?
[20:24] * toad_ wonders if there are regex ignore's in this client...
[20:24] <toad_> KenMan: what shall we do about globalQuota? that is the pressing concern
[20:24] <KenMan> investigate ! graph ! murmur !!!
[20:24] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[20:24] <toad_> we've done all 3
[20:25] <KenMan> oh, right.
[20:25] <iip_i2p> <Newsbyte> so, toad, how's the new build doing?
[20:25] <KenMan> test a fixed globalQuota to evaluate its effectiveness
[20:26] <toad_> KenMan: why did I un-average load?
[20:26] <toad_> oh, because it would keep falling if once started...
[20:26] <toad_> or rising
[20:26] <KenMan> you mean, why did you allow some links to be heavier than others...
[20:26] <toad_> eh?
[20:26] <KenMan> ?
[20:26] <iip_i2p> <Newsbyte> do I hear the word 'crappy'?
[20:26] <toad_> what do you mean?
[20:26] <toad_> Newsbyte: we are working on it
[20:27] <toad_> please stand by. the revolution will be televised.
[20:27] <iip_i2p> <Newsbyte> no doubt
[20:27] <KenMan> why did you "un average load" ? I don't recall when you did that
[20:27] <toad_> KenMan: well, it was globalQuota = queries incoming / average load
[20:27] <iip_i2p> <Newsbyte> in all those years of non-working freenet, I never claimed you or anyone wasn't working on it. :-)
[20:27] <toad_> now it's queries incoming / raw load
[20:28] <KenMan> Newsbyte : piss off ! (now I know that is british)
[20:28] <toad_> iirc i did this because if it was averaged, it would keep falling (or rising) after the stimulus had gone away
[20:28] <iip_i2p> <Newsbyte> I think that working on it was never the problem. Only the not-working was.
[20:28] <toad_> it may be that that's not a problem, and the stimulus should be averaged out by longer averages
[20:29] <iip_i2p> <Newsbyte> I can't piss off untill I've (again) had my catharsis in whining :-)
[20:29] <toad_> Newsbyte: when we think that we need more data, and we need a testnet to generate it, we'll let you know!
[20:29] <toad_> right now we have lots of data and we still have no clue what's happening
[20:29] <iip_i2p> <Newsbyte> lol
[20:30] <toad_> KenMan: yes, trying out fixed globalQuota may be enlightening
[20:30] <iip_i2p> <Newsbyte> if only....but at this pace, the testnetwork will be there around 2036
[20:30] <iip_i2p> <Newsbyte> I had something a bit sooner in mind :-)
[20:30] <KenMan> why do you resist my minimum limit for MRIs ??? do you realize how much damage an MRI of 1000ms can do ? it is almost as bad as 5084 did to your sister^H^H^H i mean, 5087
[20:30] <toad_> Newsbyte: if we get freenet done before I die that will be enough ;)
[20:30] <toad_> KenMan: I don't see what the justification is for imposing such arbitrary limits
[20:31] <toad_> if a node that can't handle such things generates them, then THAT is a bug
[20:31] <KenMan> freenet can never be done, just as computers can never be secure.
[20:31] <iip_i2p> <Newsbyte> why? You planning to have a hartattack that soon? ;-p
[20:31] <toad_> but you haven't provided any evidence that that happens
[20:31] <toad_> Newsbyte: test network 2037, freenet works 2087
[20:32] <toad_> freenet banned 2088 :)
[20:32] <iip_i2p> <Newsbyte> look, I know it's been said earlier, but...couldn't we just forget about NGR and rate-limiting, and just make a new 0.6 version with the native-OS encryptionthingy, and maybe the small chunk sizes?
[20:33] <KenMan> 2007 - toad gets "real" job and the world goes for wont of an anonymous rabbit...
[20:33] <iip_i2p> <Newsbyte> make a 5084 with those adde improvements and call it 0.6, and then go back on the eternal quest for the right algorithm and what not.
[20:33] * lostlogic votes with Newsbyte!!! CPAlgoRT, all the CPU and threading and etc. improvements, no NGR, no Rate Limiting!
[20:33] <toad_> KenMan: on the basis of my experience on freenet? I doubt it
[20:34] <toad_> lostlogic: sod that
[20:34] <toad_> apart from anything else it's completely untested
[20:34] <KenMan> let's live dangerously, just use FTP :) and call it an ony-mouse
[20:34] <lostlogic> toad_: harumph
[20:35] <toad_> if you want to throw years of YOUR work down the drain that's your own problem
[20:35] <KenMan> we have a bad network. How bad is it ? does it need immediate repair ??
[20:35] <iip_i2p> <Newsbyte> I think many objective freenetters would vote for that. It's only those too closely involved that want to 'bring it to a good end' cost what cost, because they are already to involved with it, and made huge efforts and wasted time in it
[20:35] <iip_i2p> <Newsbyte> but really, maybe just LET IT BE, and go on with the more straightforward stuff for now, at least untill 0.6
[20:36] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[20:36] <toad_> Newsbyte: the bottom line is the current network is unsatisfactory. ALL previous freenets have been equally unsatisfactory.
[20:36] <lostlogic> toad_: every coder has to do it from time to time, that is life.
[20:36] <KenMan> our options are revert to 5084, and reset to 5089 only at this point in time. "feel the pressure"
[20:36] <toad_> KenMan: well, 5088-only
[20:36] <toad_> lostlogic: 5084 is not the second coming of christ
[20:36] <toad_> it is no less crappy than numerous released builds in the past
[20:37] <iip_i2p> <Newsbyte> I know it's hard, but sometimes it's necessary. Besides, it may be not permanent (if it's not basically flawed, that is), but one could at least pospone it untill after 0.6
[20:37] <KenMan> i'm just concerned that if you don't take swift action, this channel will be flooded beyond use <wink>
[20:37] <toad_> well I can always /kickban iip_i2p :)
[20:37] <lostlogic> toad_: hrm? I wasn't saying 5084, I was saying CPAlgoRT ;-)
[20:37] <iip_i2p> <Newsbyte> really, toad, maybe you can't see it this way, but it's been TOO LONG. Really!!! It's TIME to focus on something else for the 0.6.
[20:37] <KenMan> 5084 was voted "best stable freenet release of 2004" in P2P magazine. wasn't it ?
[20:38] * ChanServ sets mode +o toad_
[20:38] <KenMan> anyway... let me dig at these MRIs gobbled off of stable...
[20:38] <toad_> anyone want a piece of me? this isn't freenet, it's IRC, and I'm the op! ;)
[20:39] * KenMan brandishes a shield ... but no weapons :o
[20:39] * salahx is on unstable anyway :)
[20:39] <KenMan> salahx: any reports from the field ?
[20:39] * pupok (~janie@81-178-89-164.dsl.pipex.com) has joined #freenet
[20:39] <lostlogic> toad_: I don't want to fight, I have no authority, I just have an opinion as someone who has used freenet a lot and has coded on it a _LITTLE_
[20:40] <salahx> kits if RNF if I start using my node for local rrequests
[20:40] <toad_> Newsbyte: I honestly don't see any problem with having a MAJOR slashdotting-CNNing-etc release every few years which is a major improvement on the previous version
[20:40] <toad_> but it has to be a MAJOR IMPROVEMENT ON THE PREVIOUS VERSION !
[20:40] <KenMan> if you did revert to 5084, nothing would change except for all these complaints... shouldn't take 20 minutes todo.
[20:40] <iip_i2p> <Newsbyte> I do wanna fight! ;-p But I'm right nevertheless
[20:40] <toad_> I doubt it
[20:40] <toad_> and we have to fix the bugs
[20:40] <toad_> lol Newsbyte
[20:40] <salahx> well already what we've got know is FAR better than 0.5
[20:40] <toad_> KenMan: do you have graphs for 5084?
[20:40] <toad_> salahx: is it?
[20:41] <toad_> push/pull tests are probably total crap
[20:41] <KenMan> 5084graphs: unfortunately , not that i know of
[20:41] <toad_> RNFs are common
[20:41] <toad_> connections are very few
[20:41] <KenMan> wait, i used 5084 when it was new, i think.
[20:41] <iip_i2p> <Newsbyte> look toad, be sensible...we've been at this since ages, just move on for now, pls. How many zillion times will you try to make it work? We need a new version to get extra support/finances/etc.
[20:41] <KenMan> and it was GOOOODDDD ! Oh Yeah Baby!
[20:41] <toad_> KenMan: also, unstable from a similar period would be interesting
[20:41] <KenMan> i have that, just can't match the two up...
[20:41] <toad_> Newsbyte: UNTIL IT WORKS, or until we run out of money
[20:41] <KenMan> 60106 ? earlier ??
[20:42] <toad_> I'm not putting out a release that is worse than the last formal release
[20:42] <toad_> KenMan: there or a bit later probably
[20:42] <KenMan> it was good. seriously.
[20:42] <toad_> it doesn't have to work perfectly
[20:43] <toad_> it does have to work BETTER
[20:43] <salahx> th'ats godo otherwise there'd never be another release :)
[20:43] <salahx> how do you define "better" though ?
[20:43] <toad_> KenMan: ~ 60128
[20:43] <toad_> anyway what I want to see is was there EVER smooth limiting?
[20:44] <toad_> salahx: oit
[20:44] <toad_> it's not that hard
[20:44] <toad_> not constantly RNFing would help
[20:44] <iip_i2p> <Newsbyte> that's not sensible. If the 0.6 version has some new features (and the nativethingy + smaller chunks would do that) and it doesn't perform less then 0.5, just make a 0.6 of it. It's changed enough, goddamnit. And if you are rational, you will agree that not having any finances (and thus you without coding, and freenet in the drain) is NOT better then having a less then perfect beta-0.6
[20:44] <KenMan> http://mywebpages.comcast.net/jkcorson/FreeNet/index.html
[20:44] <salahx> smmoth rate limiting? As in fairly constat mRI as opposed to teh totally random ones now ?
[20:44] <toad_> push/pull tests working at reasonable HTL and reasonable tries most of the time on stable would be more impressive
[20:45] <KenMan> 60125 is interesting...
[20:45] <toad_> hmmm
[20:45] <toad_> you're only including the minor stuff
[20:45] <toad_> although it's useful to see that trailers are under control
[20:45] <KenMan> i know, i was only sharpening my skills... :(
[20:46] <toad_> still from what we have, it's likely that there was a cycle
[20:46] <toad_> http://mywebpages.comcast.net/jkcorson/FreeNet/60116mri.png
[20:46] <toad_> http://mywebpages.comcast.net/jkcorson/FreeNet/60116mriplot.png
[20:46] <toad_> definite cycle
[20:46] <toad_> in fact 2 of them
[20:47] <toad_> woah http://mywebpages.comcast.net/jkcorson/FreeNet/60115-qpm.png
[20:47] <KenMan> you see how some MRIs jump off the bottom ... that is actually a long time to be that low...
[20:47] <toad_> granted that might be due to a specific bug
[20:47] <salahx> like a wave
[20:48] <iip_i2p> <Newsbyte> it maybe due to anything
[20:48] <KenMan> yeah, i thought it was worth putting a snapshot in the album of that one !!
[20:48] <salahx> oh THIS can't be good
[20:48] <iip_i2p> <Newsbyte> would it help if I begged you to leave it alone for the moment, and concentrate on straightforward stuff, that could improve Freenet?
[20:49] <salahx> Aug 3, 2004 8:08:57 PM (freenet.transport.WriteSelectorLoop, Network writing thread, NORMAL): fixKeys added freenet.transport.WriteSelectorLoop$SendJob: java.nio.HeapByteBuffer[pos=8192 lim=8192 cap=8192],java.nio.channels.SocketChannel[connected local=/192.168.0.1:8888 remote=/192.168.0.254:1436],freenet.support.io.NIOOutputStream@5a7e74,0,12963109,prio=-10: sun.nio.ch.SelectionKeyImpl@1e90164 - AND IT WROTE SUCCESSFULLY! This can happen
[20:49] <toad_> Newsbyte: what the hell do you think I am trying to do here?!
[20:49] <KenMan> the quality is not really there in my early work. Then I read some books about photography, and framing the subject... oh. never mind. I didn't realize you could hear me :)
[20:49] <toad_> and no, I suppose it isn't straightforward stuff
[20:49] * KenMan sometimes thinks with his keyboard open
[20:49] <toad_> but it is essential stuff
[20:49] <iip_i2p> <Newsbyte> well, NOT working on the small-chunks idea
[20:49] <toad_> the best thing we can do for stable is probably to reset on 5088
[20:50] <toad_> Newsbyte: I'm not, at the moment
[20:50] <toad_> I'm working on rate limiting
[20:50] <toad_> with KenMan
[20:50] <iip_i2p> <Newsbyte> there you go, that's my point
[20:50] * toad_ hmm re 5084*.png
[20:50] <KenMan> if toad gets tired , and wants to go to bed, he can re-release 5084 as 5089 and be done with it. Or reset to a pure 5088 net. sure.
[20:51] <toad_> definite oscillation, maybe not such magnitude
[20:51] <toad_> KenMan: well, it's possible there's some dumb bug in 5088 that causes connection problems
[20:51] <toad_> in which case we will have to do another reset :)
[20:51] <KenMan> I'll be damned. I do have a 5084 snapshot, june 6th...
[20:51] <toad_> I mean even if we did one
[20:52] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[20:52] <toad_> w.r.t. tiredness, this is the best time of day in summer :<
[20:52] <toad_> because it's not so urghably hot
[20:52] <iip_i2p> <Newsbyte> how long have you been busy with NGR and rate-limiting...and what has it brought us? Isn't it just possible, that, maybe, we ought to concentrate on something else, so we can make a 0.6 with reasonable performance (as the 50874 build) and then maybe waste another year on further NGR and rate-limiting? ;-)
[20:52] <KenMan> and here i just thought you were a manic schizzo ... ;)
[20:53] * toad_ considers whether to /ignore iip_i2p
[20:53] <toad_> it might be the easy option, it's a question of collateral damage
[20:53] <toad_> KenMan: manic?
[20:54] * toad_ disputes both manic and schizo
[20:54] <salahx> well if you do back out to 5084 don't back out the autmoic IP detection changes!
[20:54] <toad_> unless you were talking to Newsbyte ;)
[20:54] <KenMan> ah hah. I'm on to something ! manic schizzo , manic schizzo
[20:54] * KenMan points at toad
[20:54] <iip_i2p> <Newsbyte> let me ask differently: is there ever, somehow, somewhere, going to be a point where you say: we've spend enough time on that, let's move on.(?) Or is it more a question of: we've spend so much time in it, we just HAVE to spend more on it, otherwise we spend that other time for nothing?
[20:55] * toad_ has delusions of godhood, and delusions of being a worm, on alternate days! I must be a manic schizo!
[20:55] <KenMan> see ?!
[20:55] * toad_ just humoring KenMan
[20:55] * toad_ never understood delusions of godhood, it just seems _so_ surreal
[20:55] <KenMan> anyway, make a call. Do 5084 again, 5088 pure net reset, or do nothing. Just to irritate that newsbyte guy.
[20:55] <KenMan> ;)
[20:56] <toad_> that's probably a good idea
[20:56] <toad_> i'll need some volunteers though
[20:56] <iip_i2p> <Newsbyte> me is difficult to irritate ;-)
[20:56] <toad_> anyone want to help out with the stable reset?
[20:56] <KenMan> :) just don't cost us all the users we have...
[20:56] <toad_> KenMan: i'll try...
[20:56] <salahx> I'm on unstalbe now but I;d be glad ot switch voer for a whiel for the reset
[20:56] <KenMan> I'll do it... and not even tinker with the code !! :o
[20:57] <toad_> it will cost a lot of users, stable resets generally cause islands of dont-give-a-fuck-but-can't-be-bothered-to-uninstall users
[20:57] <toad_> so inactive users - nodes not users
[20:57] <iip_i2p> <Newsbyte> 5084 was fine
[20:57] <toad_> also it'll take weeks to recover
[20:57] <KenMan> advertise is as a revert , perhaps ?
[20:57] <toad_> numbers wise
[20:57] <toad_> KenMan: uh, no
[20:57] <KenMan> why would 5089 , LGB 5088 (built from 5084 source) be so evil ???
[20:57] <iip_i2p> <Newsbyte> 5084 was fine!say :just incorporate the nativethingy and smallchunks, and it will be good enough to be a 0.6
[20:57] <toad_> there is absolutely no point in dishonesty in open source, and everything to lose
[20:58] <iip_i2p> <Newsbyte> true
[20:58] * KenMan is intensely intrigued by toad's resistance to do a single reversion ...
[20:58] <iip_i2p> <Newsbyte> I'm being so honest, I'm actually becoming Open Source myself ;-)
[20:59] <KenMan> uh oh. I think he ignored himself. Now he is muted.
[20:59] <toad_> hmm
[20:59] <iip_i2p> <Newsbyte> lol
[21:00] <iip_i2p> <Newsbyte> well, I pitty him, sometimes, because he gets the flac of evrybody
[21:00] * KenMan a hush falls over the crowd, as all toad's minions look up to him in pain ...
[21:00] <iip_i2p> <Newsbyte> that said, he should just go the fuck on with other stuff, for the moment
[21:00] <KenMan> the moment has arrived, Lord Toad.
[21:00] <toad_> Newsbyte: other stuff LIKE WHAT?
[21:01] * toad_ is working on the reset, but will need volunteers
[21:01] <toad_> and by that I mean at least half a dozen
[21:01] <KenMan> yeah, i think Newsbyte has a split IRC client personality disorder !!
[21:01] <toad_> ideally more, I can get more by announcing to the list
[21:01] <iip_i2p> <Newsbyte> it puzzles me how one coder can be so single-minded. Aw waste of time doesn't get better with more waste, even if you think it would be a shame of the already wasted time. It's that simple.
[21:02] <toad_> Newsbyte: if we cannot improve freenet beyond the level of 0.5, or 5084, there is no point in my continued employment. NONE! ZERO!
[21:02] <KenMan> a reset it is. You sure that query levels will be okay ? what if they go wrong ? is it worth the risk ?
[21:03] * KenMan plants a seed of doubt
[21:03] <toad_> KenMan: we're going to find out
[21:03] <toad_> if we have to do another reset, then we'll do another bloody reset
[21:03] * ninja_ (im10ninjas@modem-1174.lion.dialup.pol.co.uk) Quit (Nick collision from services.)
[21:03] <KenMan> ah, the confident , logic be damned, toad is back !! woo hoo
[21:03] * ninja_ (im10ninjas@modem-1982.leopard.dialup.pol.co.uk) has joined #freenet
[21:03] <toad_> hopefully not particularly bloody but you never know when Newsbyte's around
[21:03] * verl signs up and can follow simple instructions. :)
[21:04] <iip_i2p> <Newsbyte> it doesn't have to have a drastic improvement. Where does it say 0.6 has to wait 2 years (if ever) before becoming it?
[21:04] * KenMan plans a devious and cold trick for the day toad performs his first revert in life...
[21:04] <iip_i2p> <Newsbyte> I disagree; the 5084 worked reasonably well, and can be expected to be even better with nativethingy and small-chunks.
[21:05] <KenMan> Newsbyte runs the risk of a toadslap. A decision has been made.
[21:05] <toad_> Newsbyte: that's your problem. In fact I suspect that's a negotiating position.
[21:05] * toad_ slaps Newsbyte
[21:05] <KenMan> what must we do, oh lord ? we are your servants !!
[21:05] <iip_i2p> <Newsbyte> are you calling me a nego?
[21:05] <iip_i2p> <Newsbyte> you racist!
[21:05] <iip_i2p> <Newsbyte> ;-)
[21:06] * toad_ calls Newsbyte a human being
[21:06] <iip_i2p> <Newsbyte> how dare you!
[21:06] <KenMan> ouch ! that was harsh. I thought he was super human ...
[21:06] <iip_i2p> <Newsbyte> blasfemy!
[21:06] <KenMan> Toad: blaspheme at will !!
[21:07] <iip_i2p> <Newsbyte> I'm more of a God, actually
[21:07] <KenMan> BE QUIET, or our master will not provide for us ...
[21:07] <iip_i2p> <Newsbyte> in the sense that I'm always right
[21:07] <iip_i2p> <Newsbyte> especially when others think I'm not
[21:08] <toad_> lol
[21:08] <toad_> well, I'm usually wrong
[21:08] <iip_i2p> <Newsbyte> :-)
[21:08] <toad_> but sooner or later you've got to DO SOMETHING
[21:08] <iip_i2p> <Newsbyte> I know...
[21:08] * KenMan be back 8m
[21:09] <toad_> which means your "I'm probably wrong" turns into "I probably flamingoed up"
[21:09] <iip_i2p> <Newsbyte> well, making nativethingy and smallerchunks is doing something too...
[21:09] * toad_ restarts node with 5089
[21:09] <iip_i2p> <Newsbyte> so, it's not really a matter of doing nothing, or working on the NGR/rate-limiting
[21:09] * toad_ can send jar and seednodes to anyone who wants in on the new network
[21:10] <toad_> Newsbyte: are you arguing for the restoration of classic routing?
[21:10] <iip_i2p> <Newsbyte> pff...another reset?
[21:10] * verl shouts me! me!
[21:10] <iip_i2p> <Newsbyte> what with the chunks?
[21:10] <KenMan> where to get them ? I'll serve it if you want me to...
[21:10] <iip_i2p> <Newsbyte> yet another reset, then?
[21:10] <toad_> KenMan: okay, can you accept DCC?
[21:10] <KenMan> no :( just emails...
[21:11] <toad_> Newsbyte: funnily enough when major changes to routing and rate limiting happen, resets seem to help matters
[21:11] <iip_i2p> <Newsbyte> I'm rather arguing to move forth in a more fruitful way
[21:11] <KenMan> shoot the lizard, he is the messenger
[21:11] <iip_i2p> <Newsbyte> so I noticed the past year: all those resets really helped...
[21:11] <KenMan> and the composer, and the navigator, and...
[21:12] <iip_i2p> <Newsbyte> /snip irony
[21:12] <toad_> Newsbyte: how many resets have we done on stable? they're pretty rare
[21:12] <KenMan> bitchX is recommended for here ?
[21:12] <toad_> KenMan: I'm using ksirc
[21:12] <KenMan> oh yeah...
[21:12] * lostlogic is an xchat bitch
[21:12] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[21:12] <iip_i2p> <Newsbyte> well, that would be the summum; a reset every build ;-)
[21:13] <toad_> KenMan: I'll email you
[21:13] <toad_> verl: you accept DCC?
[21:14] <KenMan> bb5 , then i'll serve it out for the followups. okay
[21:14] <iip_i2p> <Newsbyte> look, toad, let me tell you this: you've been at this for a year, and now a year later, you will still be figuring out what's wrong if you keep this up. If you're still a paid coder, that is. My question is simple (and you didn't answer it): WHEN are you going to decide enough is enough, and concentrate on some other stuff, at least for the 0.6?
[21:15] <toad_> other stuff like what?
[21:15] <toad_> that's like saying we can improve windows 95 without dealing with the core problem of it being DOS-based
[21:15] <toad_> what happens when you do that? you get windows 98!
[21:15] <verl> toad: yes!
[21:16] <toad_> verl: want source or jar+ref?
[21:16] <verl> toad: jar+ref
[21:17] <iip_i2p> <Newsbyte> indeed, well, everyone bought 98. And with all the imperfection, it stillgot most people content most of the time
[21:17] <iip_i2p> <Newsbyte> how many times do I have to tell it? nativethingy + smallchunks and based on build 5084. Then testnetwork, THEN go on with NGR and the lot.
[21:18] <iip_i2p> <Newsbyte> it not like I'm talking in esotheric generalities, here.
[21:18] * Pascal (Pascal@c-24-13-17-191.client.comcast.net) Quit (Read error: 104 (Connection reset by peer))
[21:18] <toad_> Newsbyte: with 5084's pathetic routing and only slightly better rate limiting?
[21:18] * Pascal (Pascal@c-24-13-17-191.client.comcast.net) has joined #Freenet
[21:18] <toad_> anyway I thought you didn't want smallchunks?
[21:18] <iip_i2p> <Newsbyte> that's not true
[21:18] <iip_i2p> <Newsbyte> that's not true
[21:19] <toad_> i'm sure you told me to concentrate on the basics and not on smallchunks
[21:19] <toad_> TODAY, no less
[21:19] <iip_i2p> <Newsbyte> if you check your logs, you'll see I am/was always for it. I jsut dispute the 32kb size
[21:20] <iip_i2p> <Newsbyte> what's more, I don't think I've been here online today
[21:20] <toad_> anyone else want in on the new stable network?
[21:21] <toad_> it'd be nice to have more than 2 nodes on it (1 at the moment)
[21:21] <iip_i2p> <Newsbyte> make a webupdate available, I'll see :-)
[21:21] <salahx> I'll join
[21:21] <iip_i2p> <Newsbyte> o, you mean as seednode?
[21:21] <toad_> Newsbyte: I'm trying to minimize disruption by booting a network before announcing it and updating snaps
[21:21] <toad_> salahx: dcc?
[21:21] <salahx> yep
[21:22] <iip_i2p> <Newsbyte> hmm well, it's too late now, I have to sleep
[21:22] <KenMan> News bites the hand that feeds him !
[21:22] <toad_> heh
[21:22] <toad_> nice
[21:23] <iip_i2p> <Newsbyte> I register the man that kens
[21:23] * toad_ commits the newsbyte to the mental institution
[21:23] <toad_> okay anyone else?
[21:23] <iip_i2p> <Newsbyte> no doubt Toad considers them love-bites ;-)
[21:24] <KenMan> do you know what Ken means ? it's an irish term...
[21:24] <lostlogic> toad_: dcc me the ref.
[21:24] <toad_> oops
[21:24] <iip_i2p> <Newsbyte> I ken
[21:24] <toad_> sorry lostlogic
[21:25] <lostlogic> np
[21:25] * KenMan shifts his 7G log file to another partition... and twiddles his thumbs.
[21:25] <toad_> it's in CVS; you have a dev account, right?
[21:25] <toad_> KenMan: got my files?
[21:25] * salahx is backup up my current node now
[21:25] <KenMan> I am making room for them as we type
[21:25] <iip_i2p> <Newsbyte> let me just say, with godly, thundering voice, that I'll be saying: "I TOLD THEE SO!"
[21:25] * toad_ still has no connections...
[21:26] <iip_i2p> <Newsbyte> ;-p
[21:26] * lostlogic fires up
[21:26] <iip_i2p> <Newsbyte> and now I'm off to sleep
[21:26] <KenMan> i'm getting there, 3 more gigs to go...
[21:26] <toad_> Newsbyte: rumours of your divinity have been greatly exaggerated
[21:26] <iip_i2p> <Newsbyte> sleep well (or good coding)
[21:27] <toad_> okay we have two connections
[21:27] <lostlogic> mine's still starting filesystem
[21:27] <iip_i2p> <Newsbyte> my divinity is not questionned by those who worship me, however :-)
[21:27] <KenMan> 2G to go...
[21:27] <toad_> hmmm
[21:27] <toad_> lost one
[21:27] <lostlogic> there it goes
[21:28] * KenMan plans to push toad into a corner...
[21:28] <toad_> 2 nodes... lostlogic, verl, me
[21:28] <KenMan> so toad, if this doesn't work out, will 5084 be the candidate for the next reset ??
[21:28] <toad_> KenMan: please explain your diabolical plan in advance?
[21:28] <toad_> no, if this doesn't work, we'll have a good idea WHY
[21:28] <toad_> and we'll be able to fix it
[21:28] <toad_> a better idea anyway
[21:28] <toad_> even if it's only slightly better
[21:29] <toad_> also it buys us some breathing room
[21:29] <KenMan> you mean, my desire to occupy your throne, if only for a version ??
[21:29] <toad_> KenMan: if you have some good code, commit it
[21:29] <toad_> or at least discuss it
[21:30] <toad_> I don't count getGlobalQuota() { - <delete lots of code> + return 1000; } as good code :)
[21:30] <toad_> for reasons we have extensively discusseds
[21:30] <toad_> discussed
[21:30] <KenMan> well, it basically went "force a fixed qph of around 1000 onto every node" , faster nodes will transfer faster, but everyone can route at an equivalent rate.
[21:30] <toad_> now, maybe we can get somewhere with semi-fixed globalQuota
[21:31] <toad_> KenMan: and slow nodes won't be able to keep up even with 1000qph
[21:31] <toad_> while fast nodes will be massively under-utilized
[21:31] <KenMan> and one or two other concepts, all compressed into the smallest patch I could create.
[21:31] <KenMan> it would be dirty, nasty, and temporary. But possibly revealing.
[21:31] <toad_> KenMan: are you planning on running a node?
[21:31] <KenMan> And it might even make most users happy.
[21:32] <KenMan> Oh yeah. I gotta start it now...
[21:32] <toad_> KenMan: if it's such a great experiment, why not try it on unstable?
[21:32] <salahx> Scrwew the backup, it gonan tkae DAYS to compelte wit ha 10G soter :(
[21:32] <toad_> salahx: no reason to delete the stoe
[21:32] <toad_> store
[21:32] <toad_> okay, who's 82.36.35.68:57285 ? salahx?
[21:33] <KenMan> we should if you are willing. But you will likely choke on your own tongue when you see it...
[21:33] <salahx> not deklelet it, was backign it up incase it trashes it or something...
[21:33] <toad_> no, just some incompatible leach
[21:33] * KenMan finished his 7G copy in about 2 minutes...
[21:33] <toad_> KenMan: if it works, that's more evidence to the fact that we need a better way for globalQuota() to work
[21:33] <toad_> but I doubt it'll make everything work
[21:33] <KenMan> well, we can just try that !!
[21:34] * toad_ suggests you make it a config option
[21:34] <lostlogic> KenMan: what you got, scsi disks in there?
[21:34] <toad_> for stupidly advanced users
[21:34] <KenMan> Seagate 160G , it is fast. 60M/sec i think
[21:34] <KenMan> dunno, but it is done already.
[21:35] <toad_> KenMan: sure, but 60MB/sec sustained COPY i.e. read AND write?!
[21:35] <salahx> ok, starting now!
[21:35] <KenMan> apparently, you do the math while i install
[21:35] <toad_> salahx is freenet.coronya.com:19032
[21:35] <toad_> who is 217.93.54.177:14522 ?
[21:36] <salahx> yep
[21:37] <KenMan> this time i remember to first delete node, the routing tables, and renumber listenPort !!
[21:37] <salahx> well ya don;t have ot delete the node file
[21:37] * toad_ hasn't bothered to
[21:38] <salahx> I did clear the RT and change the lsitenPort though
[21:38] * KenMan spins the dials...
[21:38] <toad_> now we're back down to 3
[21:38] <toad_> well 4
[21:38] <toad_> anyone else?
[21:39] <salahx> mRI is already 17 seconds :)
[21:39] <KenMan> i got none so far. renamed myref.txt to seednodes.ref and it is running, but no connecto !
[21:39] <toad_> KenMan: hmmm
[21:39] <toad_> you DID update the jar, right?
[21:40] <toad_> KenMan: did you say you'd host the jar and seednodes?
[21:40] <KenMan> so --version tells me
[21:40] <KenMan> yeah
[21:40] <toad_> urls please?
[21:41] <toad_> KenMan: and you restarted it? it says 5089 on the interface?
[21:41] <toad_> what's in the routing table?
[21:41] <KenMan> yeah :(
[21:41] <KenMan> empty, boss
[21:41] <toad_> okay
[21:41] <toad_> evidently it missed the seednodes.ref somehow
[21:41] <toad_> grep seednodes freenet.conf ?~
[21:41] <KenMan> it knows you, it just doesn't SEE you.
[21:42] <KenMan> it didn't puke on your seed.
[21:42] <toad_> KenMan: you said the RT was empty
[21:42] <toad_> if so it doesn't know me
[21:42] * KenMan realizes how open that is to bad jokes. Good thing NewsByte be gone.
[21:42] * toad_ ick
[21:42] <KenMan> Contacted and attempted to contact node references 1
[21:42] <KenMan> Contacted node references 0
[21:42] * KenMan restarts. shaking his head
[21:42] <toad_> what's it list my version as?
[21:43] <KenMan> it had said 5089 (still on screen)
[21:44] <toad_> test CHK for salahx, verl or lostlogic : CHK@a7Nre7805TasH1ZWS2G9Vtx1BJUTAwI,KhuBO7Skdwp1lHyYSNJInw
[21:44] * verl just got TFE icon :)
[21:44] <KenMan> now i've got you, you little toad.
[21:44] <toad_> me too
[21:44] <toad_> must be on KenMan's node
[21:44] <salahx> RNF
[21:44] <KenMan> 10G here
[21:45] <KenMan> RNFs are good when you only have 4 hops to live
[21:45] <verl> toad: it keeps shifting between 1 backed off and 0, with only your node contacted. perhaps that is normal behaviour?
[21:45] <toad_> verl: means it hasn't picked up the other nodes yet
[21:46] <toad_> anyone else want in on the new stable fork?
[21:46] <jabawok_w> yep
[21:46] * jabawok_w lurks and watches
[21:46] <toad_> jabawok_w: can you receive DCC?
[21:46] <jabawok_w> hmm not here..
[21:46] <toad_> KenMan: where's the downloads?
[21:46] <toad_> jabawok_w: one sec...
[21:46] <jabawok_w> what port does DCC use i can port forward company router
[21:47] <toad_> KenMan: filez please?
[21:47] <verl> toad: will you release a new seed with all your known nodes, or will we start to integrate with each other soon enough?
[21:47] <KenMan> http://mywebpages.comcast.net/jkcorson/5089/freenet.jar and seednodes.ref
[21:47] <toad_> verl: the latter
[21:47] <toad_> 404
[21:48] <toad_> on both :|
[21:48] <toad_> KenMan: 404 on both
[21:48] <toad_> please fix
[21:48] <KenMan> done
[21:48] <KenMan> sorry for the 1 minute delay :)
[21:49] <jabawok_w> downloading
[21:49] <jabawok_w> don't need a new freenet-ext.jar since last update?
[21:49] <KenMan> last time, it took me FOREVER to get a second node.
[21:49] <KenMan> I'll put it there just in case ?
[21:49] <toad_> verified above
[21:49] <toad_> jabawok_w: when did you last update?
[21:50] <KenMan> okay, freenet-ext.jar is there too...
[21:50] <jabawok_w> 5088 , got quoted freenet-ext.jar "just in case" i think it was for k6 updates etc.. my node is P3 though..
[21:51] <jabawok_w> seednodes.ref only 1,139 bytes?
[21:51] <jabawok_w> assuming this is ok.. fireing up node
[21:52] <salahx> I goti t finally
[21:52] <salahx> uits an tar of FReeMail
[21:52] <KenMan> hey quit backing me off. I didn't do anything to you !!
[21:52] <jabawok_w> hopefully freenet won't make a thrashing mess of my 384Mb ram server this time.. (went 400M into swap.. doh)
[21:52] <KenMan> freenet is not designed for uits. Don't hurt yourself !!
[21:53] <KenMan> jabawok_w: set -Xmx128m in start-freenet.sh (if you are on linux)
[21:53] <toad_> freenet is not designed for whats?
[21:53] <KenMan> i dunno, he typed it, not me.
[21:54] <jabawok_w> Thanks KenMan, will do.. will this make it OOM later though?
[21:54] <KenMan> not for a long long time.
[21:54] <KenMan> until lots of others join the net, at a minimum...
[21:55] <jabawok_w> ok i'll try it.. running at default for now with this 5089
[21:55] <toad_> woah
[21:55] <jabawok_w> (which i think is -Xmx256m
[21:55] <toad_> From: Ian Clarke <ian@locut.us>
[21:55] <toad_> To: tech@freenetproject.org, thong@post.harvard.edu
[21:55] <toad_> Subject: [Tech] The travelling salesman problem and Freenet
[21:56] <KenMan> i can't see it on gmane... maybe it is fresh.
[21:56] <toad_> Mute's routing is not based on ant behavior, it is based on a broadcast
[21:56] <toad_> search (AFAICS they use the same type of ultra-inefficient broadcast
[21:56] <toad_> search that led to the downfall of the original Gnutella network).
[21:56] <toad_> The file transfer employs some kind of ant-inspired protocol, but this
[21:56] <toad_> won't help alleviate the problems caused by their broadcast-based search
[21:56] <toad_> approach.
[21:56] <toad_> hehe, is this true?
[21:56] <toad_> anyway the main post is interesting
[21:57] <KenMan> it is small scale broadcast , IIRC . very few hops worth.
[21:58] <toad_> KenMan: what's the horizon?
[21:58] <toad_> how many nodes?
[21:58] <KenMan> 5 peers max. I'm looking for the graph again...
[21:58] <jabawok_w> mine should be fired up by now
[21:59] <toad_> cool
[21:59] <salahx> still jsut a toad node here
[21:59] <verl> same here, up 50 mins
[21:59] <toad_> 6 nodes, including one shadowed (I don't know what that means on the conns page)
[21:59] <verl> err, 40 :)
[22:00] <KenMan> http://mute-net.sourceforge.net/howAnts.shtml
[22:00] <KenMan> 50 minutes ? already ? resets are so anti-climactic :o
[22:01] <toad_> well... we need more node ops
[22:01] <ShaunMacPherson> THere is a new Mute-net like java based network called Ants P2P also that mentions mute, not sure if Mute or Ants came first
[22:01] * toad_ will not pull the switch until we have at least 10
[22:01] <toad_> preferably 20
[22:01] <toad_> or 100
[22:01] <toad_> ShaunMacPherson: you want in on the new stable network reset?
[22:02] <KenMan> switch tomorrow noon, when you rise from the dead...
[22:02] <jabawok_w> i've only got one node in the connections page it's ip ends with 16.91 if that helps
[22:02] <toad_> yup, that's me
[22:02] <KenMan> toad node ! toad node !
[22:02] <TheSeeker> Toad: I'e been up for 4 days 19 hours on 60174... you want to gather any stats or anything off it before I update? (Checking in, still on vacation)
[22:02] <thelema|away> ShaunMacPherson: mute was first
[22:02] * thelema|away is now known as thelema
[22:02] <KenMan> evening, thelema
[22:03] <thelema> hi ken
[22:03] <toad_> TheSeeker: no
[22:03] <thelema> what's up?
[22:03] <toad_> thelema: you got a stable node?
[22:03] <toad_> we're resetting stable
[22:03] <toad_> another test CHK for anyone on newstable: CHK@m7FBfLrNEStUVb3jt1koLg2nS-ITAwI,gmV2Hqyl4NPLzQTsFWpn8w
[22:03] <TheSeeker> alright *kills it*
[22:03] <thelema> toad_: sorry, can't help you there.
[22:04] <jabawok_w> RNF, 3 backed off - on that CHK above.. i'll keep trying
[22:05] <KenMan> oh goodness. I have a database of 1250 keys I inserted the other week... but they are tiny nonsense files...
[22:06] <salahx> I
[22:06] <jabawok_w> does this reset of stable reset the datastore?
[22:06] <salahx> i"m not getting it,. Ether get RNF or the node hangs
[22:06] <KenMan> toad is backing me off on purpose !!!
[22:07] <thelema> toad_: I don't think haing standardized routing and rate limiting will matter at all
[22:07] <KenMan> i get working on it, but it doesn't come back. Unless it is a big honker. But then, toad wouldn't publish honkers... :o
[22:07] <thelema> toad_: having each node's limiting work by itself is more important
[22:07] <toad_> thelema: perhaps
[22:07] <toad_> jabawok_w: no
[22:07] <jabawok_w> righteo thanks
[22:08] <toad_> KenMan: what's my MRI
[22:08] <toad_> ?
[22:08] <KenMan> 38s , no problem, right ??
[22:08] <KenMan> it was 25...
[22:08] <toad_> it is if you make requests more often than every 38 seconds :)
[22:08] <salahx> its 42s here
[22:08] <KenMan> ah, RNF.
[22:08] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[22:09] <KenMan> it is unless I make my request at the exact right moment...
[22:09] <KenMan> :p
[22:09] <salahx> there number can't be right
[22:09] <jabawok_w> back to one node (toads) in the conn page now.. (i had 2 for a while)
[22:09] <jabawok_w> still RNF
[22:10] <KenMan> daggonit. People keep stealing my opportunity to meet new peers. I never get no new peers :(
[22:10] <toad_> hmmm
[22:10] <toad_> what's using all the cpu time?
[22:10] <KenMan> not a thing, here
[22:10] <TheSeeker> I guess the snapshots haven't updated yet...
[22:11] <KenMan> oh, that bug in globalQuotaPerHour calcs. You know, the one we haven't found yet ??
[22:11] <TheSeeker> got 60178 from unstable-latest
[22:11] * KenMan apologizes for the dry humour
[22:12] * toad_ restarts node
[22:12] <jabawok_w> ok just now got the filter warning page for your chk above
[22:12] <toad_> (getting rid of old rt)
[22:12] <KenMan> http://mywebpages.comcast.net/jkcorson/5089/freenet.jar
[22:12] <KenMan> http://mywebpages.comcast.net/jkcorson/5089/freenet-ext.jar
[22:12] <KenMan> http://mywebpages.comcast.net/jkcorson/5089/seednodes.ref
[22:12] <jabawok_w> oh.. that reduces the chances of me downloading the CHK then lol
[22:12] <jabawok_w> (toad restarting node that is)
[22:13] <KenMan> at least he fell off my RT. That is a sign that SOMETHING is working, anyway
[22:13] <jabawok_w> Connections open (Inbound/Outbound/Limit) 0 (0/0/256)
[22:13] <jabawok_w> yep same here
[22:13] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[22:14] <toad_> uh
[22:14] <KenMan> 256 ? you must be mad, boy !!
[22:14] <toad_> falling off your RT is not desirable
[22:14] * toad_ back up
[22:14] <toad_> your nodes should reconnect
[22:14] <jabawok_w> KenMan what should it be?
[22:14] <toad_> automatically
[22:14] <toad_> please? :)
[22:14] <KenMan> jaba - reasonable. But for now, 100 will do.
[22:14] <toad_> KenMan: am I no longer in your RT? that should never happen with so few peers...
[22:15] <jabawok_w> default is 512 isnt' it?
[22:15] <KenMan> I meant OCM. sorry, i don't always get the lingo right
[22:15] <toad_> jabawok_w: not any more
[22:15] <toad_> KenMan: okay, I'm in the RT?
[22:15] <toad_> so it will reconnect in a few mins?
[22:15] <jabawok_w> ok i'll just comment my change out of the config so defaults are used again
[22:15] <thelema> toad_: variant mRI implementations could/should have the effect of killing any resonance across the network.
[22:15] <KenMan> toad: yah, just not active
[22:15] * KenMan speeds up the process
[22:16] * toad_ hrrm
[22:16] <toad_> I am still waiting...
[22:16] <KenMan> well, *that* didn't work
[22:16] * toad_ wonders what the initial backoff is...
[22:17] <KenMan> new feature. User must kill and restart the node before it will connect to anything.
[22:17] <KenMan> twice
[22:17] <toad_> well, I have wiped my RT
[22:17] <toad_> but I should still be in yours
[22:17] <toad_> therefore your node should connect to me
[22:17] <toad_> right?
[22:17] <salahx> itsn to reconning :*
[22:17] <KenMan> thrice ...
[22:18] <jabawok_w> restarting
[22:19] <toad_> and it didn't even wipe my RT!
[22:19] <toad_> what the bloody hell is wrong with the connections?
[22:19] <salahx> something is horribly malfuction here
[22:19] <toad_> salahx: anything in your logfile?
[22:19] <KenMan> it just refuses to git up and go. I'll try kicking it forcefully ....
[22:20] <toad_> what is running according to the Env page?
[22:20] * verl got a Caught java.io.IOException: Value out of range: 4500000.0 deserializing
[22:20] <salahx> checking...
[22:20] <toad_> verl: shouldn't matter much
[22:21] <salahx> jsut a bunch of RNF
[22:21] <KenMan> the errors are not, and the warnings mean deep-doo-doo
[22:21] <toad_> i tested it on a localtestnet not long ago
[22:21] <salahx> My node can't contact your anymore though
[22:21] <toad_> i'm sure connecting works!
[22:22] <jabawok_w> Connections open (Inbound/Outbound/Limit) 0 (0/0/200)
[22:22] <KenMan> everyone count down from nine, while standing on one leg. Then retry.
[22:22] <toad_> well, there's nothing I can do
[22:22] <toad_> BECAUSE THE BUG IS ON YOUR END!
[22:22] <toad_> ALL OF YOU!
[22:22] <toad_> hahahahahahahaha! :<
[22:23] <KenMan> nasty. The port responds to a telnet... toad, sometimes you really just bug me.
[22:23] * toad_ restarting again
[22:23] * toad_ thinks something wierd happened
[22:23] <toad_> esp as the rt wasn't deleted
[22:24] <toad_> okay started
[22:24] <toad_> properly
[22:24] <toad_> 0 nodes known
[22:25] * toad_ got a connection...
[22:25] <toad_> 80.46.53.9:4477
[22:25] <toad_> who's that?
[22:25] * KenMan goes up to heat a burrito - he left his node running during toadnode-reload
[22:25] * verl got some people from iip
[22:25] <toad_> verl: your node isn't through yet though?
[22:26] <toad_> aha
[22:26] <toad_> salahx is through
[22:26] <toad_> salahx: did you restart?
[22:26] <jabawok_w> got an incoming connection from x.x.232.181
[22:26] <toad_> or did it just retry?
[22:26] <jabawok_w> yay
[22:26] <salahx> i did , but it 7 restries before it connected
[22:26] <verl> toad: no, i wiped my rt and restarted with announcment on
[22:26] <toad_> you did what? restart?
[22:27] <toad_> robhome.ath.cx:4637 - new one...
[22:27] <salahx> I rrestarted, but it still would'nt reconnect to your node
[22:27] <jabawok_w> thats me
[22:27] <jabawok_w> Connections open (Inbound/Outbound/Limit) 4 (3/1/200)
[22:27] <salahx> My node tries 7 times, it finally connected on the 7th try
[22:27] <toad_> okay
[22:27] <toad_> bbl#
[22:28] <toad_> bbl
[22:28] <toad_> zzz
[22:30] <toad_> talk to you all tomorrow, especially KenMan
[22:30] * thelema is now known as thelema|away
[22:32] <jabawok_w> night toad
[22:32] <jabawok_w> Connections open (Inbound/Outbound/Limit) 1 (0/1/200)
[22:32] <jabawok_w> dropped back to just an outbound to toads node
[22:42] <KenMan> night. Thanks for staying up so late...
[22:44] * KenMan and so it begins
[22:45] <KenMan> unstable took at least a week until queries were 'regular' and steady...
[22:45] <KenMan> frost: smoke em (queries) if you got 'em
[22:47] <KenMan> actually, unless people post messages, i don't think that will do much of anything, except possibly generate some unsuccessful traffic.
[22:48] <KenMan> anybody (on linux) want a script to fetch random keys from my node ??
[22:55] <jabawok_w> yeah why not
[22:56] <KenMan> http://mywebpages.comcast.net/jkcorson/5089/bangOn5089.sh
[22:56] <KenMan> then just 'sh bangOn5089' and cross your fingers. Run it in your freenet directory.
[22:57] <KenMan> Don't hesitate to repeat running it several times if it expires... and for any iip'ers , this is hosted by my ISP. So they can log your addresses, but I certainly am not collecting them.
[22:58] <KenMan> duh . obviously if jabawok_w is configuring routers, he'll know how to make it work. (should have been 'sh bangOn5089.sh')
[22:59] <jabawok_w> running btw
[23:00] <KenMan> excellent. This should drive all future traffic to my node, as the routing tables are formed :(
[23:00] <KenMan> gack !
[23:00] <jabawok_w> getting a fair bit of DNF, quite a bit of RNF and occasinally Download of x bytes started..
[23:00] <jabawok_w> and definitely getting lots of:
[23:01] <jabawok_w> java.lang.NullPointerException
[23:01] <jabawok_w> at freenet.client.SegmentOutputStream.close(SegmentOutputStream.java:225)
[23:01] <jabawok_w> at freenet.client.FCPClient$FCPRequest.doDataFound(FCPClient.java:957)
[23:01] <jabawok_w> at freenet.client.FCPClient$FCPRequest.doit(FCPClient.java:880)
[23:01] <jabawok_w> at freenet.client.FCPClient$FCPInstance.run(FCPClient.java:262)
[23:01] <jabawok_w> at java.lang.Thread.run(Thread.java:534)
[23:02] <KenMan> ugh, i don't know about that one. Anyway, if you see some "State DONE reached." then things are set up right...
[23:02] <KenMan> and as you download these files, other nodes would begin to search from both of our nodes... it's just a good way to create some initial traffic.
[23:02] <jabawok_w> hmm plenty of: State FAILED reached.
[23:03] <KenMan> due to backoff, most likely. Don't worry about it.
[23:03] <KenMan> and for you conspiracy nuts, each of these items contains a specially crafted FreeNet virus, that makes your node phone... YOU ! hahaha
[23:04] <jabawok_w> quite a fair bit of :The query was restarted somewhere on Freenet after a node failed to reply, waiting another 50 seconds before I give up
[23:04] <KenMan> those are good, at least you are stimulating the wires...
[23:05] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Nick collision from services.)
[23:05] * Ash-Foxeus (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[23:05] * mazzanet (~mazzanet@mazzanet.registered) Quit (Read error: 54 (Connection reset by peer))
[23:05] * ninja_ (im10ninjas@modem-1982.leopard.dialup.pol.co.uk) Quit (niven.freenode.net irc.freenode.net)
[23:05] * hobx (~hobx@h196n1fls21o1077.bredband.comhem.se) Quit (niven.freenode.net irc.freenode.net)
[23:05] * thelema|away (~thelema@adsl-65-65-202-46.dsl.stlsmo.swbell.net) Quit (niven.freenode.net irc.freenode.net)
[23:05] * vsalento (~vsalento@cs136227.pp.htv.fi) Quit (niven.freenode.net irc.freenode.net)
[23:05] * mikeDOTd (~mikedotd@plutonium.nexthop.net) Quit (niven.freenode.net irc.freenode.net)
[23:05] * cehteh (foobar@cehteh.homeunix.org) Quit (niven.freenode.net irc.freenode.net)
[23:05] * mazzanet (~mazzanet@mazzanet.registered) has joined #freenet
[23:05] <KenMan> i am getting over 1Kqph already - queries/min says around 20. So someone is at least doing something. But successes are like maybe 1/minute. That's you. Probably.
[23:05] * ninja_ (im10ninjas@modem-1215.lemur.dialup.pol.co.uk) has joined #freenet
[23:06] <KenMan> come back, bean !!
[23:09] * vsalento (~vsalento@cs136227.pp.htv.fi) has joined #freenet
[23:09] <KenMan> nice, my MRIs out are sitting at 3 to 4 seconds...
[23:09] <jabawok_w> KenMan.. fixed the java npe's, - getting lots of State DONE reached. now
[23:09] <KenMan> cool. What was happening ??
[23:10] <jabawok_w> did s/dummy\.txt/\tmp\dummy\.txt/ to your script lol
[23:10] <jabawok_w> silly problem
[23:10] <jabawok_w> permissions. coudn't write dummy.txt in /usr/lib/freenet
[23:10] * jay (jcl@ool-18bf6dac.dyn.optonline.net) Quit (".")
[23:10] <KenMan> ah, you run ... what ? gentoo ?
[23:10] <KenMan> or debian ?
[23:10] <jabawok_w> affirmative
[23:10] <jabawok_w> doh
[23:10] <jabawok_w> gentoo
[23:10] <KenMan> gentoo
[23:11] <KenMan> bbl
[23:14] * thelema|away (~thelema@adsl-65-65-202-46.dsl.stlsmo.swbell.net) has joined #freenet
[23:14] * mikeDOTd (~mikedotd@plutonium.nexthop.net) has joined #freenet
[23:15] * hobx (~hobx@h196n1fls21o1077.bredband.comhem.se) has joined #freenet
[23:15] * cehteh (foobar@cehteh.homeunix.org) has joined #freenet
[23:16] * Rikane|mowing (Logan@ip142177017115.mpoweredpc.net) has joined #freenet
[23:19] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) has joined #FreeNET
[23:21] * Rikane|mowing is now known as Rikane
[23:21] * Rikane is now known as rikane
[23:31] * Ash-Fox (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[23:32] <salahx> ok, I got some mroe nodes now!
[23:39] * Ash-Foxeus (Hal-9000@aav191.neoplus.adsl.tpnet.pl) Quit (Client Quit)
[23:43] * robilad[afk] is now known as robilad
[23:58] * hirvox (~hirvox@cs181027153.pp.htv.fi) Quit (Read error: 104 (Connection reset by peer))
These logs were automatically created by Jay Oliveri with his gimp hapi on irc.freenode.net.
