Timestamps are in GMT/BST.
[0:18] * Zorix (Brandon@fl-65-40-16-125.dyn.sprint-hsd.net) has joined #freenet
[0:20] * Overand (common@83.100.252.64.snet.net) Quit (Read error: 60 (Operation timed out))
[0:27] * Overand (common@35.96.252.64.snet.net) has joined #freenet
[0:36] * pupok (~janie@81-178-115-22.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[0:36] * sanity (~r00t@81-178-115-22.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[0:36] * pupok (~janie@81-178-115-22.dsl.pipex.com) has joined #freenet
[0:36] * sanity (~r00t@81-178-115-22.dsl.pipex.com) has joined #freenet
[0:36] * ChanServ sets mode +o sanity
[1:45] * salahx (salahx@sc1-24.217.174.147.charter-stl.com) Quit ()
[1:51] <KenMan> stable 5091 - 12 hours worth of data - requestSuccess=3.8% , routingSuccess=2.3% , 10G datastore
[1:53] <KenMan> WOW: 24387 requests received, but only 18916 unique keys
[1:55] <KenMan> ah, it's not that amazing, after looking at the breakdown by number of requests for unique keys
[2:02] <KenMan> average percent of routes backed off over last run (13 hours total, sampled per minute) = 88%
[2:02] * Sugadude (tor@cassandra.eecs.harvard.edu) Quit (Remote closed the connection)
[2:06] * Sugadude (~Sugadude@216.32.201.125) has joined #freenet
[2:13] <mikeDOTd> ouch
[2:32] * KenMan restarts with a new load component that targets 66% backoff...
[2:34] <KenMan> and of course, it was not written correctly the first time out of the gate :)
[2:56] <KenMan> cool, seems to be working now. Time for sleep...
[4:20] <ShaunMacPherson> bye kenman
[5:51] * cbreak (~cbreak@dhcp-32-094.via-eth.ch) has joined #freenet
[6:06] * sanity (~r00t@81-178-115-22.dsl.pipex.com) Quit (Read error: 54 (Connection reset by peer))
[6:06] * sanity (~r00t@81-178-115-22.dsl.pipex.com) has joined #freenet
[6:06] * ChanServ sets mode +o sanity
[6:13] * Iakin5 (~nb@212.105.104.162) has joined #freenet
[6:15] * sanity (~r00t@81-178-115-22.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[6:15] * sanity_ (~ian@81-178-115-22.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[6:15] * pupok (~janie@81-178-115-22.dsl.pipex.com) Quit (Connection reset by peer)
[6:15] * sanity (~r00t@81-178-115-22.dsl.pipex.com) has joined #freenet
[6:15] * ChanServ sets mode +o sanity
[6:16] * sanity_ (~ian@81-178-115-22.dsl.pipex.com) has joined #freenet
[6:16] * pupok (~janie@81-178-115-22.dsl.pipex.com) has joined #freenet
[6:39] * Hory (~Miranda@81.196.25.110) has joined #FreeNet
[6:45] * guido^pe (~unknown@dsl-082-082-155-213.arcor-ip.net) has joined #freenet
[7:03] * bnmnetp (~bnmnetp@12-214-10-145.client.mchsi.com) Quit (Remote closed the connection)
[7:28] * sanity_m (~sanity_m@future-is.orange.co.uk) has joined #freenet
[7:34] <pupok> hi sanity_m
[7:42] * sanity_m (~sanity_m@future-is.orange.co.uk) Quit (Read error: 54 (Connection reset by peer))
[7:46] * robilad (~topic@mpiat2313.ag2.mpi-sb.mpg.de) has joined #freenet
[7:47] * Ash-Fox (Hal-9000@aau163.neoplus.adsl.tpnet.pl) Quit (Read error: 104 (Connection reset by peer))
[7:54] * Ash-Fox (Hal-9000@aat81.neoplus.adsl.tpnet.pl) has joined #FreeNET
[7:56] <ShaunMacPherson> hi pupok
[7:57] <hobx_> Happy Birthday Pupok
[7:57] <hobx_> IIRC
[7:57] <pupok> why thank you
[7:57] <pupok> did you know my phone was on this channel all night
[7:58] <pupok> unlike my DSL connection
[7:58] <hobx_> Your good-for-nothing slacker boyfriend treating you right for the occasion?
[7:59] <hobx_> You getting into the cellphone IRC thing?
[8:00] <pupok> yes he even shaved!
[8:00] <pupok> i get 3 megs of gprs a month and i'm trying to see if i can use them all
[8:00] <pupok> last month i only managed 0.5
[8:02] <hobx_> heh
[8:03] <hobx_> Tried cell phone porn? :-)
[8:04] <pupok> nope
[8:04] <cbreak> try a mud
[8:04] <hobx_> I'd google for it, but I'm at work.
[8:04] <pupok> heh
[8:05] <hobx_> already on probation etc :-)
[8:06] * sanity (~r00t@81-178-115-22.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[8:06] <hobx_> ^^^ what is up with that
[8:06] <hobx_> he pops in and out the whole time
[8:06] * sanity (~r00t@81-178-115-22.dsl.pipex.com) has joined #freenet
[8:06] * ChanServ sets mode +o sanity
[8:07] <pupok> that is weird because i stayed on this time
[8:08] <hobx_> Ian's machine is in a spontaneous reboot cycle.
[8:10] <pupok> the only thing i can think is that his is farther away from the wireless router
[8:10] <pupok> mine is next to it
[8:12] <hobx_> ah wireless
[8:12] * toad__ (toad@82-32-16-91.cable.ubr03.azte.blueyonder.co.uk) Quit (Remote closed the connection)
[8:12] <hobx_> that explains it. What ever happened to the old geek status symbol of CAT-5 running along every wall.
[8:12] * hobx_ has like 100 m of CAT-5 in his one room apartment.
[8:14] <cbreak> now it's better: A Geek has enough EM Waves in his room to power light tubes...
[8:14] <hobx_> Want to know what a quasi-continuous function is?
[8:14] <ShaunMacPherson> calculus? :)
[8:14] * toad_ (toad@82-32-16-91.cable.ubr03.azte.blueyonder.co.uk) has joined #freenet
[8:15] <hobx_> hmm
[8:15] <ShaunMacPherson> probably a function in which you can draw without lifting your pencil but only partially hehe
[8:15] <hobx_> I don
[8:15] <hobx_> t think it is called that on this level.
[8:15] <ShaunMacPherson> continuity is a big thing in calculus, its such a bore :)
[8:16] <hobx_> No f is quasi-continuous, any d > 0 there is a closed set F so that f is continuous on F and cap(F^c) < d
[8:17] <ShaunMacPherson> f the function? only time i see capital F is when you do the intergral of a function :P
[8:18] <ShaunMacPherson> in calculus :)
[8:18] <hobx_> F is a subset of the underlying space, which is usually L^2(X,mu)
[8:20] <ShaunMacPherson> u sure you have the term right?
[8:20] <ShaunMacPherson> http://mathworld.wolfram.com/search/ , wolfram cant find it and it is one of the more comprehensive math sites :)
[8:20] <hobx_> sorry. f is L^2(X,mu), F is a subset of X of course.
[8:21] <ShaunMacPherson> you need to help fix the wikipedia mathematics webpages, they are all horrible, if you like wikipedia that is
[8:22] <hobx_> I've done a couple
[8:22] <ShaunMacPherson> One of my favorite examples is this for torus: a solid of revolution generated by revolving a circle about an axis coplanar with the circle.
[8:22] <hobx_> But it takes a while.
[8:22] <ShaunMacPherson> thats how it *was*, i screamed at them it was the most pedantic thing i've seen, so i made them put in 'doughnut shaped' :)
[8:22] <ShaunMacPherson> it does
[8:22] <hobx_> What is wrong with that?
[8:22] <ShaunMacPherson> its pedantic to the extreme
[8:23] <hobx_> it isn't called pedantic, it's called formal
[8:23] <cbreak> I like the first version more.
[8:23] <hobx_> If you don't define things formally in mathematics, you won't get anywhere
[8:23] <ShaunMacPherson> like saying "I am currently engaged in a behavior using several of my digits in a perpendicular motion to scratch the dermis of my glutismaximus' or you could say im scatching my ass
[8:24] <ShaunMacPherson> the first way just complicates things, esp. for an intro
[8:24] <hobx_> The difference is that you won't have an entire field of discovery based on the construction of your ass scratching
[8:24] <hobx_> but there can be based on the construction of a torus. Therefore it needs to be formal.
[8:24] <ShaunMacPherson> no, but it is an illustration that being pedantic doesnt serve much of a purpose in the intro
[8:25] <ShaunMacPherson> one can be conscise and clear at the same time, it just takes work :)
[8:25] <hobx_> And besides, a torus only looks like a donut in a three dimensional euclidean geometry.
[8:25] <ShaunMacPherson> their picture of a torus = a doughnut
[8:26] <hobx_> well, don't ask me to fix wikipedia's math pages by making them less mathematical.
[8:26] <hobx_> The defenition you gave is exactly what I want to find if I look up torus.
[8:26] <ShaunMacPherson> making them ore clear though
[8:26] <hobx_> That is as clear as it gets
[8:26] <cbreak> a doughnut is not a torus
[8:27] <ShaunMacPherson> i kept the defintion as it was, i added 'doughnut shaped' solid... to the rest
[8:27] <ShaunMacPherson> if the goal of wiki is to tell people what a torus is then that goal is not served by definitions that are not understood by anyone other then mathematics graduates ;0
[8:28] <hobx_> And the term "quasi-continuous" exists. It is just too specialized for mathworld (this is a course for PhD students in mathematical analysis).
[8:29] <ShaunMacPherson> if you say so :)
[8:29] <hobx_> try google
[8:30] <ShaunMacPherson> i did, the term is used
[8:30] <ShaunMacPherson> no definitions though
[8:30] <ShaunMacPherson> http://www.google.ca/search?hl=en&ie=UTF-8&q=%22quasi-continuous+function%22&btnG=Search&meta=
[8:30] <hobx_> I gave the defenition
[8:30] <cbreak> quasi-continuous -> continuous for a limited range of input values.
[8:31] <hobx_> sort of
[8:31] <ShaunMacPherson> cbreak just took abite out of pedanticism with that definition :)
[8:31] <ShaunMacPherson> after it the more technical stuff can be put, that would be a good wiki article
[8:32] <hobx_> The term is of absolutely no use if you can't understand the defenition.
[8:34] <ShaunMacPherson> i dunno, cbreak's definition seems to be more unstandable then yours but i'll like the wikipedians edit the intro to the level that the average person could understand
[8:34] <hobx_> well anyways
[8:35] <hobx_> I try to help wikipedia when I have a chance, but I do it to make it more comprehensive, not to water down the formal defenitions that already there.
[8:35] <hobx_> gotta work now
[8:35] <ShaunMacPherson> as i said, put your junk after the explanitory stuff
[8:35] <ShaunMacPherson> see u
[8:38] * lolo-laptop (~lostlogic@68.251.84.226) has joined #freenet
[9:12] * |UK-Monster| (~bbtt@cpc2-warr1-5-0-cust27.bagu.cable.ntl.com) has joined #freenet
[9:13] * |UK-Monster| is now known as lexx
[9:13] * Janete (~jano@155.210.155.101) has joined #freenet
[9:21] * pupok (~janie@81-178-115-22.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[9:21] * leex-Zzzz (~bbtt@cpc2-warr1-5-0-cust27.bagu.cable.ntl.com) Quit (Read error: 60 (Operation timed out))
[9:21] * pupok (~janie@81-178-115-22.dsl.pipex.com) has joined #freenet
[9:26] * robilad (~topic@mpiat2313.ag2.mpi-sb.mpg.de) Quit (Remote closed the connection)
[9:28] * Hory (~Miranda@81.196.25.110) Quit (Read error: 54 (Connection reset by peer))
[9:28] * sanity_ (~ian@81-178-115-22.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[9:28] * pupok (~janie@81-178-115-22.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[9:28] * sanity_ (~ian@81-178-115-22.dsl.pipex.com) has joined #freenet
[9:28] * pupok (~janie@81-178-115-22.dsl.pipex.com) has joined #freenet
[9:30] * robilad (~topic@mpiat2313.ag2.mpi-sb.mpg.de) has joined #freenet
[9:30] * Zorix (Brandon@fl-65-40-16-125.dyn.sprint-hsd.net) Quit (Read error: 110 (Connection timed out))
[9:32] <toad_> heh
[9:32] * toad_ suggests putting a picture of a torus in 3 dimensions on the wikipedia page on torii
[9:33] * robilad wonders if freenet is in wikipedia
[9:34] <cbreak> it is
[9:34] <robilad> great!
[9:35] <hobx_> toad_: Why stop at three?
[9:35] <toad_> hobx_: how do you show a 4D torus on a 2D web page?
[9:35] <hobx_> Perspective!
[9:36] <cbreak> movie with colours to display one dimension.
[9:36] <toad_> hobx_: example?
[9:36] <toad_> i've seen the 4-cubes...
[9:36] <toad_> they make more sense if they're animated imho
[9:36] <toad_> a LOT more sense
[9:37] <hobx_> I'm just kidding
[9:37] <hobx_> A three dimensional corss section of a 4d torus is two spheres.
[9:38] <toad_> hmm, that kinda makes sense
[9:38] <gregh> but, is wikipedia in freenet?
[9:38] <cbreak> no
[9:38] <toad_> gregh: not afaik
[9:39] <toad_> archive.org guy said "contact us when you can do 1TB"
[9:39] <gregh> heh
[9:39] <toad_> w.r.t. P2P
[9:39] <toad_> (they have more like 400TB)
[9:39] <toad_> 100TB? a lot anyway :)
[9:40] <cbreak> even P2P users can have 40 PB... :)
[9:40] <toad_> they'd run a mile from freenet, of course
[9:40] <toad_> cbreak: hmm?
[9:41] <toad_> reachable content on freenet even now must be in tens of gigs though...
[9:41] <cbreak> a story on /. claimed that US Law Enforcers have confiscated 5 P2P Hubs with 40 Petabyte of Data each.
[9:42] <toad_> ahh. it had been corrected by the time i saw it
[9:42] <toad_> to 1-100GB each
[9:42] <toad_> i.e. very little :)
[9:42] <toad_> i mean on a good day you might be able to get 10GB on FREENET ! :)
[9:42] <toad_> and freenet sucks ;)
[9:43] * sanity (~r00t@81-178-115-22.dsl.pipex.com) Quit (Read error: 54 (Connection reset by peer))
[9:43] <mikeDOTd> lol
[9:43] <cbreak> hmm... sometimes it is like /dev/random over the Internet :)
[9:43] <toad_> (retrievability/performance wise)
[9:43] <toad_> not that I'm not trying to fix it
[9:45] <KenMan> yeah, some of us suspect that you are trying to fix it ;)
[9:46] <KenMan> okay. I added a load component that targets 66% backedoff. The results are depicted here: http://mywebpages.comcast.net/jkcorson/FreeNet/5091c.png
[9:47] <KenMan> normal backoff was 88% before I added the component. Afterwards it only went down to 78% , and so far routingSuccess is NOT improved at all.
[9:48] <KenMan> this would be categorized as "freezedown" if all nodes implemented it, and yet it *would* function, albeit at a level around 42 qph...
[9:49] <KenMan> the secret lies in the distribution of mRI values for the routes...
[9:49] <KenMan> 379 2243 2834 27476 36376 36516 42847 47762
[9:49] <KenMan> 48308 51123 76536 83525 100957 110637 155268 163760
[9:49] <KenMan> 169980 180403 218330 238581 245423 255040 261609 271342
[9:49] <KenMan> 274879 278837 286255 322756 323388 328092 328750 345762
[9:49] <KenMan> 349657 358558 360163 363448 456819 466372 500993 507975
[9:49] <KenMan> 535253 538433 565433 588724 618839 619674 622497 638130
[9:49] <KenMan> 653954 659429 663127 687747 994358 1181030 2501078
[9:49] <KenMan> Average MRI from routes = 377230ms
[9:51] <toad_> KenMan: imho such things are pointless without queueing
[9:52] <toad_> try it from unstable-hacked-to-stable
[9:53] <KenMan> so long as you are comfortable with queueing a request for a time period that is roughly comparable to the average MRI I show above...
[9:53] <toad_> no, we need to do something about them
[9:53] <toad_> i.e. reduce them
[9:53] <KenMan> 8)
[9:53] <toad_> Number of known routing nodes 397
[9:53] <toad_> Number of node references 397
[9:53] <toad_> Number of newbie nodes 42
[9:53] <toad_> Number of uncontactable nodes 0
[9:53] <toad_> Contacted and attempted to contact node references 397
[9:53] <toad_> Contacted node references 101
[9:53] <toad_> Contacted newbie node references 42
[9:53] <toad_> Connections with Successful Transfers 70
[9:54] <toad_> Backed off nodes 61
[9:54] <toad_> hmmmm
[9:54] * KenMan wonders..oO "did he really say that?"
[9:54] <toad_> this is with fairly heavy logging so lowish traffic...
[9:54] <toad_> quite variable traffic actually
[9:54] <toad_> anything from 48qph to nearly 2kqph
[9:55] * ced_ (ced@d213-102-173-225.cust.tele2.fr) has joined #freenet
[9:55] <toad_> KenMan: IMHO we need to reduce the MRIs. I have maintained this for some time.
[9:55] <KenMan> and you still see 60% backoff :(
[9:55] <toad_> KenMan: I'm not sure exactly what the right approach is but we have some ideas
[9:55] <KenMan> that is a start...
[9:55] <toad_> the conn limit is one possibility but with a limit of 200 i only have 100...
[9:56] <toad_> often i have much less than even that
[9:57] <toad_> we could try 100 with debugged queueing and a network reset... but i don't want to go too low because of bootstrapping issues and so on
[9:57] <KenMan> yes, i might possibly have mailed you an explanation for that. I don't know what .shouldReference() really does.
[9:57] <toad_> hmm, i didn't know we still used it?
[9:58] <KenMan> i dunno, but the countUnbackedOffNodes() func is counting nodes to which we have no connection :p
[9:58] <toad_> Current probability of a request succeeding by routing 5.2%
[9:58] <toad_> Current probability of an inbound request causing a transfer outwards 5.3%
[9:58] <toad_> Current target (best case single node) probability of a request succeeding 6.7%
[9:58] <toad_> woah
[9:58] <KenMan> that is a nice improvement !
[9:58] <toad_> requestSuccessRatio over the last some hours looks pretty solid...
[9:59] <toad_> even when there is no local traffic to inflate it
[9:59] <KenMan> cool !!! where did you used to sit just prior to queueing ?
[9:59] <toad_> routingSuccessRatio is less so... 0.01 to 0.03...
[9:59] <KenMan> oh...
[9:59] <KenMan> well, let's stick with the higher numbers... why are they higher ?
[9:59] <toad_> routingSuccessRatioCHKNoRNF is reasonable...
[10:00] <toad_> okay
[10:00] <toad_> requestSuccessRatio
[10:00] <toad_> daily averages...
[10:00] <toad_> hmmm
[10:00] <toad_> looks liike it's been reasonable for some time
[10:01] <KenMan> requestSuccessRatio is mainly a function of your datastore size
[10:01] <toad_> january and april it was pretty bad
[10:01] <toad_> but since then it's been okay
[10:01] <toad_> and yes my ds is huge
[10:01] <toad_> :)
[10:01] <toad_> routingSuccessRatioNoRNF is somewhat impressive...
[10:02] <toad_> ignoring 0200 to 0900, where we had significant local traffic, you get...
[10:02] <toad_> 8/26/04 10:00:00 AM BST 638 13 0.02037617554858934
[10:02] <toad_> 8/26/04 11:00:00 AM BST 555 16 0.02882882882882883
[10:02] <toad_> 8/26/04 12:00:00 PM BST 530 15 0.02830188679245283
[10:02] <toad_> 8/26/04 1:00:00 PM BST 595 14 0.023529411764705882
[10:02] <toad_> 8/26/04 2:00:00 PM BST 435 17 0.03908045977011494
[10:02] <toad_> 8/26/04 3:00:00 PM BST 503 38 0.07554671968190854
[10:02] <KenMan> routingSuccess is a better indicator... the only thing I don't like about routingSuccess is that it doesn't count the true number of outbound request messages... which is hosed due to RNFs
[10:03] <toad_> routingSuccessRatioNoRNF doesn't count RNFs
[10:03] <toad_> but routingSuccessRatio does
[10:03] <toad_> but they're not necessarily fatal, yeah..
[10:03] <KenMan> but it ignores a focal problem - namely the RNFs getting in the way most of the time
[10:03] <toad_> well the idea of making RNFs but not loops fatal is interesting
[10:04] <toad_> but perhaps we should explore other avenues first?
[10:04] <toad_> full request coalescing would be possibly quite nice
[10:04] <toad_> even transfer coalescing
[10:05] <toad_> i don't know if in the current state of the network it would make much global difference
[10:05] <toad_> Number of requests (sent/received) 122484/14393
[10:06] <toad_> that's with ~ 2900 local requests over the period though...
[10:06] * pupok (~janie@81-178-115-22.dsl.pipex.com) Quit (Read error: 54 (Connection reset by peer))
[10:06] <KenMan> here is a breakdown of the number of (re)tries each outgoing request got to perform:
[10:06] <KenMan> 1 5530
[10:06] <KenMan> 2 2517
[10:06] <KenMan> 3 1396
[10:06] <KenMan> 4 779
[10:06] <KenMan> 5 433
[10:06] <KenMan> 6 209
[10:06] <KenMan> 7 94
[10:06] <KenMan> 8 41
[10:06] <KenMan> 9 21
[10:06] <toad_> ooh interesting...
[10:06] <KenMan> 10 6
[10:06] <toad_> QueryRejected/Looped request 70/81827
[10:06] <toad_> QueryRejected/No route found 354/25259
[10:06] <toad_> KenMan: hmmm
[10:06] * pupok (~janie@81-178-115-22.dsl.pipex.com) has joined #freenet
[10:07] <KenMan> in that time, 503 outgoing messages collided / looped somewhere beyond my node
[10:07] <KenMan> are you reading sven's post ? or do you get that displayed in OCM ?
[10:07] <Iakin5> [14:39] <hobx_> sorry. f is L^2(X,mu), F is a subset of X of course.
[10:07] <Iakin5> Of course
[10:07] <toad_> KenMan: my OCM
[10:08] <KenMan> so did Iakin upgrade the OCM to display the multiple classes of QR ? I can't find it with 5091...
[10:08] <toad_> so 3* as many RNFs as loops
[10:08] <toad_> KenMan: yes
[10:08] <toad_> on cur unstable
[10:08] <toad_> DataRequest 122514/14410
[10:08] <toad_> another useful stat...
[10:08] <KenMan> sven hoffman made a post that showed (for unstable) 10 times as many loops as RNFs !!!
[10:09] <toad_> w.r.t. MRIs...
[10:09] <KenMan> I hold pretty steady at 5% of QRs=loops, the rest RNFs ...
[10:09] <toad_> currently we queue for 10s at most for requests, 20s at most of inserts, 60s for any local request or insert...
[10:10] <toad_> so the top 10 of your nodes are probably available to local requests
[10:10] <toad_> the rest aren't
[10:10] <toad_> the top 3 are available to nonlocal requests...
[10:11] <toad_> suppose we create a new network, with some changes to try to reduce MRI
[10:11] <toad_> what would we do?
[10:11] <KenMan> and the rest of your nodes with higher mRIs pop in every once in a while to service about 1% of your incoming traffic
[10:11] <cbreak> is there a reason some unstable builds are not available as snapshot?
[10:11] <toad_> first off, reduce the connection limit to maybe 100?
[10:11] <toad_> cbreak: they are
[10:12] <KenMan> i still haven't done that analysis - I am assuming that nodes with high mRIs stay high... probably wrong about that.
[10:13] <toad_> secondly, return DataNotFound if we can't route a request?
[10:13] * toad_ hmmm
[10:14] <toad_> we'd have to do per node failure tables too
[10:14] <toad_> and that would probably combine to produce something degenerate...
[10:14] <toad_> we'd HAVE to count a fatal QR as a DNF...
[10:14] * toad_ thinks this may not be as clear cut as it seems
[10:16] <KenMan> coming at it from the other direction, just for the sake of "what if" ... suppose every route had an equivalent, constant mRI. Then,
[10:16] * toad_ suggests the following gameplan: reduce max connections to ~ 100. get rid of HTL, and allow inserts to be retried despite collisions. implement new splitfile algo, implement transparent splitting. reduce max file size to 256kB.
[10:16] <toad_> KenMan: that is impossible
[10:16] <toad_> i mean we could just fix them, sure
[10:16] <toad_> but then they'd be wrong
[10:16] <KenMan> every link would be available for an equal amount of queries, but each link would be utilized differently when it came to xfers...
[10:16] <toad_> nodes would be either severely overloaded or severely underloaded
[10:17] <KenMan> i'm just trying to think about the problem in a different light, is all
[10:17] <toad_> you can't just say "cap MRIs". all that will happen is that MRIs will be inaccurate and useless.
[10:18] <toad_> KenMan: what if we could reduce MRIs by a factor of ~ 8?
[10:18] <cbreak> oh, fun: My DS contains more than 13100 Keys. A few minutes ago it contained less than 13000. All new keys seem to be in the same specialisation area.
[10:18] <toad_> factor 2 by cutting the # conns by half
[10:18] <toad_> and factor 4 by reducing max block size to 256kB
[10:18] * Hory (~Miranda@81.196.25.110) has joined #FreeNet
[10:18] <KenMan> no, mRIs provide two purposes, you tend to ignore one of them... 1) it controls rationing resources amongst the requesters 2) it limits the rate of queries across a single route
[10:19] <KenMan> you would still have 2) in my analysis world (which I don't propose we implement, just examine)
[10:20] <KenMan> if we could cut mRIs down by a factor of about 10, things would get very interesting indeed
[10:20] <toad_> if we reduce it by a factor 8, then the first 11 can be used for nonlocal requests
[10:20] <toad_> and the first 27 for local ones
[10:21] <toad_> out of 55
[10:21] <toad_> if we can do a factor of 16, then all but the last 2 are available for local requests
[10:21] <toad_> and the first 15 are available for remote ones
[10:22] <toad_> cbreak: that's good ! :)
[10:22] <cbreak> well... now its 13193 keys.
[10:22] <KenMan> i don't think just "get the mRIs smaller" is a sufficient target - I believe that we need mRIs to wander within a smaller range, one that only allows about 1 degree of magnitude of difference
[10:22] <cbreak> all in Area 51
[10:22] <toad_> KenMan: I don't see why
[10:23] <toad_> if they're closer together and they're all 600,000 that doesn't help very much
[10:23] <cbreak> it seems strange that only that specialisation area grows.
[10:23] <KenMan> because if one route can offer 0.7261 qph and another can offer 241,992 qph worth of quota, things tend to get a little imbalanced
[10:23] <toad_> if we can get MRIs down by factor 32, we get the first 28 for local requests... that's much more interesting... now, how do we do this? reduce # conns by a factor of 2 and reduce avg file size by a factor of 16
[10:24] <toad_> 1MB/16 = 64kB chunks
[10:24] <toad_> maybe need to go to 32kB to be sure
[10:24] <KenMan> you assume that reducing avg file size will not affect the number of queries :p
[10:24] <toad_> KenMan: do i?
[10:24] <cbreak> 4f 43
[10:24] <cbreak> 50 48
[10:24] <cbreak> 51 301
[10:24] <cbreak> 52 56
[10:24] <cbreak> 53 56
[10:24] <toad_> if we have 50qph at 1MB
[10:25] <toad_> reduce it to 512kB
[10:25] <toad_> then we have 100qph at 512kB
[10:25] <KenMan> cbreak - announcements strike again, looks like they are working, huh ?
[10:25] <toad_> with an MRI a factor of 2 smaller
[10:25] <toad_> KenMan: woah
[10:25] <cbreak> hmm...
[10:25] <toad_> cbreak: with NGR?!?!?
[10:25] <toad_> woah
[10:25] <toad_> cool!
[10:25] <cbreak> unstable latest (60208)
[10:25] <toad_> some actual routing specialization artefacts... things must be improving!
[10:25] * toad_ recommends you post to devl
[10:25] <cbreak> It happened a few times before.
[10:26] <toad_> IMHO announcements don't work
[10:26] <toad_> err IIRC
[10:26] <KenMan> they don't help , you mean ??
[10:26] <cbreak> The reason I think that is a bug is: After a restart, the peek is gone.
[10:26] <toad_> KenMan: if we go from 50qph at 1MB to 200qph at 256kB, we will have 4x smaller MRI
[10:26] <toad_> KenMan: even though we have 4x higher qph
[10:26] <toad_> KenMan: right?
[10:26] <toad_> cbreak: hrrm
[10:27] <KenMan> yes, i can't argue... it just seems that the basic problem is still there, even with lower mRIs...
[10:27] <toad_> well the problem would be that smaller requests are queued for shorter periods
[10:28] <ced_> i get only rnf with "Attempts were made to contact 1 nodes" does my node needs more time or there is some files to delete?
[10:28] <toad_> what did i set it to ? 1MB: 10 sec, 256kB: 2.5sec, 128kB: 1.25 sec, 64/32/16/8/4/2/1: 1 sec
[10:28] <KenMan> yeah, that is kind of in line with my intuition, which i am unable to verbalize (analyze i suppose)
[10:28] <toad_> so if we reduce it below 128kB it's useful...
[10:28] <toad_> also we could perhaps increase the minimum to 2 seconds per hop
[10:29] <toad_> ced_: build number? how many nodes are actually connected?
[10:29] <cbreak> ced_: If you have just started a fresh node, you may need to wait. If you have upgraded from before 5089 you need to reseed
[10:29] <toad_> 2 secs/hop tips the balance in our favour at 128kB
[10:29] <toad_> hmmm
[10:29] * hobx_ (~chatzilla@ankh.math.chalmers.se) Quit (Remote closed the connection)
[10:29] <toad_> 10 sec/hop ~= 100 secs on a successful request, 200 secs on a DNF
[10:29] <toad_> probably not acceptable...
[10:30] <toad_> even if the request succeeds in 5 htl
[10:30] <toad_> 2.5sec/hop ~= 25 secs on a successful request, 50 secs on a DNF... given current xfer rates that's okay, but what if xfer rates increase?
[10:30] <KenMan> we want mRI to apply to xfer load, but not necessarily query rates... which is of course impossible, but that's what we want.
[10:31] <toad_> 32kB chunk, average xfer rate of 3kB/sec, makes 30 seconds search + 10 seconds xfer + 25 secs queueing... hrmm
[10:31] <toad_> so 40 secs becomes 65 secs
[10:31] <toad_> perhaps that's a price worth paying?
[10:32] * KenMan avoids voicing an unpopular opinion
[10:32] <toad_> that would make it approx 2.5x gain
[10:32] <toad_> KenMan: which one?
[10:33] <KenMan> well, there are so many unpop opins ;)
[10:33] <toad_> that i'm barking up the wrong tree? that's a popular one!
[10:33] <KenMan> sort of. I mean, you are learning from what you are doing. I don't think the tree is necessarily wrong.
[10:33] <KenMan> choice of tree rather
[10:34] <toad_> well short term: will it help to reduce max file size to 256kB? if so, why? if not, why not?
[10:34] <KenMan> i don't know... stick with your beliefs, and follow them through. That's all I can offer.
[10:35] <KenMan> that is not an easy question to answer, and the impact is MAJOR
[10:35] <KenMan> so far you have avoided applying the sledgehammer to the thick ice
[10:35] <ced_> is there alternate sources for seednodes
[10:35] <ced_> is there alternate sources for seednodes?
[10:35] <toad_> well it WILL make queries finish more quickly on average, which has some useful side-effects...
[10:35] <KenMan> some queries, yes
[10:36] <toad_> ced_: how many actual connections do you have? Web interface -> advanced mode -> open connections ?
[10:36] <cbreak> ced_: Any node.
[10:36] <toad_> KenMan: most queries
[10:36] <toad_> on average
[10:36] <toad_> the average size is ~ 400kB
[10:36] <toad_> if the max was 256kB, the average would be ~ 250kB
[10:36] <toad_> so a nice reduction in average transfer time
[10:36] <KenMan> probably, yes. I bb10
[10:36] <toad_> 10 what? mins?
[10:37] <toad_> hmmmmmmmmmmmmmmmm
[10:37] <ced_> it works now. long to train compared to previous builds
[10:37] <toad_> the best way to draw MRIs together is propagation
[10:37] <toad_> IF we can make propagation work
[10:37] <toad_> which is a big IG
[10:37] <toad_> IF
[10:38] <toad_> making RNFs fatal is another interesting one...
[10:38] * Zorix (Brandon@fl-65-40-16-125.dyn.sprint-hsd.net) has joined #freenet
[10:38] <toad_> if we return DNF, then the node can't distinguish...
[10:39] <toad_> either way it'd have to go into the failure table for security reasons
[10:39] <toad_> the problem is we may get some interesting artifacts of that...
[10:40] <toad_> i dunno, maybe not
[10:40] <ced_> what is a "sane" limit for open connections on windows xp adsl
[10:40] <toad_> what form would they take if so?
[10:40] <toad_> ced_: the default should be reasonable
[10:40] <cbreak> is there a corespondance between the number of files in the folder store/51 and the number of keys in that specialisation area?
[10:40] <toad_> limit to 200, maybe 100 actual if you're lucky
[10:40] <toad_> cbreak: no
[10:40] <toad_> intentionally
[10:41] <toad_> we don't really want the per-node FTs to be doing the routing for us...
[10:42] <Iakin5> http://mywebpages.comcast.net/jkcorson/FreeNet/5091c.png
[10:42] <Iakin5> What are thoose spikes? (the green ones)
[10:42] <toad_> dunno
[10:42] <toad_> your purple line seems to be settling nicely
[10:43] <toad_> that's encouraging
[10:43] <toad_> even if it does take 4 hours to do so...
[10:44] <KenMan> the green line is the average of the active set of mRI values from each route, sampled per minute
[10:44] <toad_> curiously load seems to be more over 1.0 than under it - that ought to make purple drop, but it isn't
[10:44] <toad_> okay
[10:44] <toad_> then the spikes are caused by nodes with really bad MRIs coming online briefly
[10:44] <toad_> expected behaviour :(
[10:45] <KenMan> the greenish-bluish is constantly above 10 (=1.0 load) due to my new backoff component , which is what draws queries down
[10:45] <toad_> ahhh
[10:45] <toad_> without that it still settles?
[10:45] <KenMan> without what ?
[10:46] <toad_> backoff load component
[10:46] <KenMan> well, the settling is less obvious, and less consistent, but i guess you could say that... ;)
[10:46] <toad_> KenMan: you said it would probably cause freezedown?
[10:46] <toad_> if used globally?
[10:46] <toad_> KenMan: how do you determine that?
[10:47] <cbreak> hmm... funny: my store/temp/ folder contains 381 files that end with 51aeda110d641d9eed392170a8ece026ae3fcb950f0203
[10:47] <KenMan> it would force query rates to "nearly on the floor. " It is just my guess...
[10:47] <toad_> cbreak: woah
[10:47] <toad_> my store/temp only has 30 or so files in it...
[10:47] <KenMan> it brought the number of incoming queries down to half or less of what it was without this new component...
[10:48] <KenMan> I have 10 files in my store/temp dir...
[10:49] <Iakin5> [16:34] * toad_ suggests the following gameplan: reduce max connections to ~ 100. get rid of HTL, and allow inserts to be retried despite collisions. implement new splitfile algo, implement transparent splitting. reduce max file size to 256kB.
[10:49] <KenMan> the other thing backoff load component did was to cut number of transfers in half (or less), as well.
[10:49] <Iakin5> ....And fix the transfer timeout bugs
[10:49] <Iakin5> ...and the current memory leaks
[10:49] <KenMan> Iakin, how many QRs are looped and how many are RNFs on your supernode ? or is it still down ?
[10:50] <toad_> Iakin5: obviously there are a hundred million bugs to fix too :)
[10:50] <toad_> Iakin5: but i am of the opinion that the transfer timeout bugs ARE fixed
[10:50] <toad_> and nobody's told me anything about memory leaks, just high churn on fast nodes
[10:50] <Iakin5> [16:42] <KenMan> because if one route can offer 0.7261 qph and another can offer 241,992 qph worth of quota, things tend to get a little imbalanced
[10:51] <Iakin5> One should then show a wide area of specialization whilst the other a narrow one..
[10:51] <toad_> we're just trying to address the core issues, if that is possible... find out if that is possible...
[10:52] * Janete (~jano@155.210.155.101) Quit (leguin.freenode.net irc.freenode.net)
[10:52] * cbreak (~cbreak@dhcp-32-094.via-eth.ch) Quit (leguin.freenode.net irc.freenode.net)
[10:52] * Iakin5 (~nb@212.105.104.162) Quit (leguin.freenode.net irc.freenode.net)
[10:52] * vsalento (~vsalento@cs136227.pp.htv.fi) Quit (leguin.freenode.net irc.freenode.net)
[10:52] * hirvox (~hirvox@cs181027153.pp.htv.fi) Quit (leguin.freenode.net irc.freenode.net)
[10:53] <toad_> KenMan: do you think making RNFs fatal would cause significantly reduce MRIs?
[10:53] <KenMan> what is a leguin, anyway ? it is not a bean, that is a legume :)
[10:53] <KenMan> toad, that is a very complex question!
[10:53] <toad_> s/reduce/reduced
[10:53] * Janete (~jano@155.210.155.101) has joined #freenet
[10:53] * Iakin5 (~nb@212.105.104.162) has joined #freenet
[10:53] * cbreak (~cbreak@dhcp-32-094.via-eth.ch) has joined #freenet
[10:53] * vsalento (~vsalento@cs136227.pp.htv.fi) has joined #freenet
[10:53] * hirvox (~hirvox@cs181027153.pp.htv.fi) has joined #freenet
[10:53] <toad_> :<
[10:54] <toad_> i can see lots of reasons to expect it to suck :|
[10:54] <toad_> and a few reasons to expect it to be a larger problem
[10:54] <KenMan> well, then don't go there :) ummm... i don't know. I wouldn't expect it to affect mRIs, but rather degree of backoff. If load computation was altered.
[10:55] <toad_> well it might allow propagation to work...
[10:55] <KenMan> see, it only encourages more queries when I tried it locally. So actually, it does affect mRI.
[10:55] <KenMan> but i didn't want it to in that case.
[10:55] <toad_> have you tried both combined?
[10:56] <KenMan> what is propagation, exactly ?
[10:56] <toad_> load component due to backoff, probably
[10:56] <KenMan> that's almost worth a try ;)
[10:56] <toad_> well try it
[10:56] <KenMan> okay, i shall.
[10:56] <toad_> how exactly do you make RNFs fatal btw? just send a DNF back where you would have sent a QR?
[10:57] <KenMan> i perform the 'abort this query' behavior that occurs with route exhaustion after the first attempt
[10:57] <cbreak> 50 48
[10:57] <cbreak> 51 498
[10:57] <cbreak> 52 56
[10:57] <toad_> ahhh
[10:57] <toad_> hmmm
[10:57] <KenMan> whatever that is. I believe it is "send back an RNF."
[10:57] <cbreak> Somehow I think data is multiplying in the temp dir.
[10:58] <toad_> cbreak: not sure if it counts at that point...
[10:58] <KenMan> no, don't convert RNFs to DNFs, just don't retry them. A loop is NOT an RNF, it is a QR
[10:58] <toad_> eh?
[10:58] * Sugardude (~Sugadude@216.17.101.36) has joined #freenet
[10:59] <toad_> there is a clear difference between fatal RNFs and nonfatal RNFs
[10:59] <toad_> and I'd strongly prefer this to be clearer than the human-readable Reason field
[10:59] <KenMan> an RNF and a loop are both cases of QR , but a loop is not encoded as an RNF
[10:59] <toad_> and ANY fatal failure should go into the failure table for security reasons
[10:59] <cbreak> now there are 579 duplicate hashes in temp... most interesting :)
[11:00] <toad_> we would have to try it network-wide...
[11:00] <KenMan> just putting back in place a limit that said one retry at maximum would cut down the multiplication factor to 2 at most.
[11:00] <toad_> but would that have any useful effect?
[11:00] <KenMan> that might be worth consideration. Doing it at 3 would not reduce the volume significantly (thus not affect backoff significantly), refer to my table further up.
[11:01] <toad_> actually i'm surprised we still get instant RNFs
[11:01] <toad_> on unstable
[11:01] <KenMan> those are the true exhaustions, where 'request couldn't get off the node' on the first attempt
[11:01] <KenMan> 100% backoff , ick
[11:01] * Sugadude (~Sugadude@216.32.201.125) Quit (Remote closed the connection)
[11:02] <toad_> my requestFailRNFRatio is interesting...
[11:02] * interrupt (~chatzilla@64.122.23.213) has joined #freenet
[11:02] <toad_> varies a lot
[11:02] <KenMan> over minutes or over hours ?
[11:02] <toad_> mostly between 30% and 60% today
[11:02] <toad_> over hours
[11:03] * toad_ wonders if we need a network reset once queueing works :)
[11:03] <KenMan> I'm at 6% with the extra load component
[11:03] <toad_> this is my hacked stable build too... the numbers on unstable would be interesting...
[11:04] <toad_> queueing SHOULD help...
[11:04] <cbreak> what stats do you need?
[11:04] <KenMan> I started out at 40% and drifted down to 5-8 %
[11:04] <toad_> unfortunately the MRIs are too high..
[11:05] <Iakin5> toad, have you seen any sign of the shitload of these:
[11:05] <Iakin5> Aug 26, 2004 5:22:31 PM (freenet.node.ds.FSDataStoreElement$KeyInputStreamImpl, Finalizer, ERROR): Please close() me manually in finalizer: Key: 3784958d15a5f15c44e7794627a0c71c085cf01b0f0203 Buffer: freenet.fs.dir.NativeFSDirectory$ExternalNativeBuffer@1eb2c3:0x1 : 3784958d15a5f15c44e7794627a0c71c085cf01b0f0203:temp:2160:b4c2785bae4339c New: true ( 0 of 1025 read)
[11:05] <Iakin5> at freenet.node.ds.FSDataStoreElement$KeyInputStreamImpl.finalize(FSDataStoreElement.java:404)
[11:05] <toad_> Iakin5: no
[11:05] <toad_> not here
[11:05] <KenMan> none here either. It is some artifact of your rubberhole :o
[11:05] <Iakin5> I get about one per minute..
[11:05] <Iakin5> ..since the intoduction of queueing
[11:06] <toad_> well the load-from-backoff component should prevent his node being a rubberhole :)
[11:06] <Iakin5> What do they mean then?
[11:06] <toad_> i don't know
[11:06] <Iakin5> That something forgets to close something?
[11:06] <toad_> presumably
[11:06] * Iakin5 have no idea what a KeyInputStream is
[11:06] <Iakin5> are they for something important?
[11:06] <KenMan> I will refer to it as a rubbernode in future. Somehow the other name sounds offensive.
[11:07] <Iakin5> ..maybe not
[11:07] <toad_> heh
[11:07] <toad_> KenMan: so what's the next phase, apart from debugging queueing and getting it as widely deployed as possible?
[11:08] <KenMan> that's it. I don't know... what you said a week ago. Which was... not fixed keys, i think.
[11:08] <toad_> why not fixed keys?
[11:08] <KenMan> we had three things, i thought fixed keys was one of them, but maybe not.
[11:09] <toad_> per node failure tables might cause MORE requests by encouraging retrying...
[11:09] <toad_> transfer coalescing is nice locally but i doubt it'll make much difference network level
[11:09] <KenMan> you said queueing first, so keep debugging it. I'll see if I can find the other two in the logs...
[11:09] <toad_> same for client cache
[11:10] <toad_> weren't they queueing, load propagation, and making RNFs fatal?
[11:11] <Iakin5> ok, /me adds this to his local code:
[11:11] <Iakin5> System.out.println("Inited:");
[11:11] <Iakin5> inited.printStackTrace();
[11:11] <Iakin5> System.out.println("LastTouch:");
[11:11] <Iakin5> lasttouch.printStackTrace();
[11:11] <Iakin5> So.. now it should be possible to see who last knew about the stream.. and who created it..
[11:11] <KenMan> okay - queueing, fixed keys, and localReq balancing - please pick ONE. any one. But only ONE.
[11:11] <KenMan> you chose queueing
[11:12] <KenMan> and sort of addressed localReq balancing at the same time :)
[11:13] <KenMan> that's where we were on 08/10/04
[11:13] <toad_> ahhh
[11:13] <toad_> well i partially addressed local req balancing yes
[11:13] <toad_> there is of course more to do on it
[11:13] <toad_> what we've talked about today suggests fixed keys might not have as big an impact as hoped for
[11:13] <KenMan> what would you put on the list today ? i still don't understand 'load propagation' ...
[11:13] <toad_> because of practicalities of queueing times...
[11:14] <toad_> KenMan: well, i'd prefer "reducing the file size profile" to full blown fixed size keys
[11:14] <toad_> and there's a number of things i'd like to do specifically to improve local performance e.g. client-cache
[11:14] * matahari (~matahari@munyoo.cs.mu.OZ.AU) has joined #freenet
[11:15] <toad_> KenMan: load propagation was your idea, wasn't it? using backoff fraction as part of the load
[11:15] <KenMan> reducing max-key-size will have almost the same network impact, though.
[11:15] <KenMan> ah, that's load propagation. Okay, sure, I'll take credit for that one, although it was an eye-opener.
[11:15] <toad_> we can't do load propagation until we have a strong possibility of it not causing a freezedown
[11:15] <toad_> fortunately queueing prevents instant RNFs...
[11:16] <KenMan> Given the size of some of the larger mRIs out there (that stay at the same approx level), load prop would "put the smackdown" on the network
[11:16] <toad_> i dunno, it might make sense to tweak the queue times...
[11:16] <toad_> yeah.. but 10 nodes is better than 2 nodes. it still sucks but IMHO queueing may make a big difference when debugged and widely deployed
[11:17] <toad_> the main reason to expect it not to is that we can't queue for very long on each hop...
[11:17] <KenMan> i'll go try my silly experiment (backoff + single retry max) and report back after some hours of data...
[11:17] <toad_> sendingReplyHTL... hmm...
[11:18] <toad_> day averages are about 10
[11:18] <toad_> for REMAINING HTL when we send the data...
[11:18] <KenMan> i like your optimism re: queueing. You may just cause it to save the day yet...
[11:18] <toad_> now, are most requests sent at 15? or 20?
[11:18] <toad_> if 15, then we can increase the queue times
[11:19] * Janete (~jano@155.210.155.101) has left #freenet
[11:19] <toad_> what do you think is reasonable for a total delay over 10 hops?
[11:20] <toad_> if we assume more or less current network conditions, a 32kB key will transfer at 1kB/sec
[11:20] <toad_> so it's 32 seconds for transfer
[11:20] <toad_> around 30 seconds for search with no queueing
[11:20] <toad_> 10 hops... we can do 2 seconds a hop without it really hurting, can't we?
[11:20] <toad_> maybe even 3 seconds a hop?
[11:20] <KenMan> i would think so.
[11:20] <toad_> which?
[11:21] <KenMan> 3 second.
[11:21] <toad_> that would make the total time ~ 90 seconds...
[11:21] <KenMan> go for the gain, and ignore the pain, at least initially
[11:22] <toad_> if it xfers at 3kB/sec (my personal target), we have 40 secs against 70 secs
[11:22] <KenMan> the longer they wait, the more they hate - those old RNF blues
[11:22] <toad_> so nearly doubling...
[11:22] <toad_> hmmm
[11:24] <Iakin5> Inited:
[11:24] <Iakin5> at freenet.node.ds.FSDataStoreElement$KeyInputStreamImpl.<init>(FSDataStoreElement.java:283)
[11:24] <Iakin5> at freenet.node.ds.FSDataStoreElement.getKeyInputStream(FSDataStoreElement.java:58)
[11:24] <Iakin5> at freenet.node.ds.FSDataStoreElement$KeyOutputStreamImpl.getKeyInputStream(FSDataStoreElement.java:163)
[11:24] <Iakin5> at freenet.node.states.data.ReceiveData.getKeyInputStream(ReceiveData.java:106)
[11:24] <Iakin5> at freenet.node.states.request.InsertPending.receivedMessage(InsertPending.java:225)
[11:24] <Iakin5> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
[11:24] <Iakin5> LastTouch:
[11:24] <Iakin5> at freenet.node.states.request.InsertPending.receivedMessage(InsertPending.java:226)
[11:24] <Iakin5> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
[11:24] <Iakin5> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
[11:24] <Iakin5> at java.lang.reflect.Method.invoke(Unknown Source)
[11:24] <Iakin5> at freenet.node.State.received(State.java:143)
[11:24] <Iakin5> at freenet.node.StateChain.received(StateChain.java:177)
[11:24] <Iakin5> Sooo..
[11:24] <Iakin5> Insertpending creates the stream..
[11:25] <Iakin5> and insertpending forgets to close it
[11:25] <toad_> also... queueing on the network level should REDUCE time taken by requests by having less retries and RNFs...
[11:25] <Iakin5> Sorry, the first line of the second stacktrace should read:
[11:25] <Iakin5> at freenet.node.ds.FSDataStoreElement$KeyInputStreamImpl.setParent(FSDataStoreElement.java:406)
[11:28] <toad_> okay, bbiab
[11:32] <toad_> Iakin5: it then goes to SendData
[11:35] <toad_> aha
[11:35] * toad_ has got that infinite loop back...
[11:40] <toad_> aha
[11:40] * toad_ has found the cause, perhaps...
[11:44] <KenMan> curious, the result of backoff load component is that my average outgoing mRIs have risen from under 100s to around 300s, which is by coincidence, about the average of the mRIs from my routes...
[11:45] <KenMan> we don't need an explicit diag to measure the percent of QRs due to looping, Iakin has already done this in OCM (i assume it was he that broke out the individual QR counts)
[11:50] * toad_ restarts with fix for too-many-ConnectionOpenerManagers-scheduled-cpu-loop bug
[11:50] <toad_> KenMan: i'd say that's more than a coincidence :)
[11:50] <KenMan> adjusts target backoff of 75% , as it never quite reached 66%
[11:50] <toad_> the question is, what happens if you have two nodes running it?
[11:51] <toad_> unfortunately you can't easily simulate this sort of thing on localtestnets...
[11:51] <KenMan> the star warriors come down from the heavens and destroy the network, i guess
[11:51] <toad_> you make all sorts of meaningless assumptions...
[11:51] <KenMan> yeah, but it keeps life fun.
[11:51] <KenMan> I just don't convey my meanings very well, is all.
[11:52] <toad_> well we COULD try it out on some sort of testnet
[11:52] <toad_> KenMan: no i don't mean you
[11:52] <toad_> i mean me
[11:52] <toad_> if i set up a localtestnet
[11:52] <toad_> to test the backoff target thing
[11:52] * ced_ (ced@d213-102-173-225.cust.tele2.fr) Quit ()
[11:53] <KenMan> um, unless you do bw rate limiting , which will slow your progress, it will probably not be a decent "sim" - the LAN doesn't produce the whacky range of mRIs we have in real life
[11:54] <KenMan> does NGRoutingTable.countUnbackedOffNodes do what you want it to do ?? by including nodes that aren't connected ?
[11:55] <toad_> why would i want it to include nodes not connected?
[11:56] * toad_ commits largely untested bugfix...
[11:56] <KenMan> i don't know, but it currently does !! bug ??
[11:57] <toad_> 100% cpu usage after some hours
[11:57] <toad_> doing virtually nothing
[11:57] <KenMan> possible reason for having fewer peers recently ?
[11:57] <toad_> caused by ConnectionOpenerManager
[11:57] <cbreak> http://dhcp-32-094.via-eth.ch/~cbreak/freenet/ds_histogram_data_detail.txt
[11:57] <toad_> doubt it
[11:57] <toad_> i think it only happens on recent unstable builds...
[11:58] <KenMan> <toad> why would i want it to include nodes not connected? - so should I fix it and check it in ?
[11:59] <KenMan> i was getting back a value of something over 100, when I only had about 50 peers. It is counting unconnected nodes.
[12:01] <KenMan> if (ne.isAvailable(false) &&isConnected(ne.id)) rather than if (ne.isAvailable(false)) n would fix it.
[12:02] <toad_> well, what calls it?
[12:02] <toad_> anything?
[12:02] <toad_> if nothing except your code, sure, fix it
[12:02] <KenMan> two other places in NGRoutingTable seem to use it. One is trying to detect more than 10% backoff (which it never will) and the other is something called ".shouldReference()"
[12:03] <KenMan> I wrote my own method, because I didn't know your intentions
[12:03] <toad_> trying to detect more than 10% backoff? huh?
[12:04] <KenMan> // If can only talk to 10% of RT... we're probably in trouble
[12:04] <toad_> hmmm
[12:04] <toad_> fix the method
[12:04] <KenMan> okay :)
[12:05] <KenMan> but what about how .shouldReference() is intending to use it ? I don't understand that one at all.
[12:05] <toad_> hmmm
[12:05] <toad_> where is shouldReference? Node?
[12:06] * jay (jcl@ool-18bf6dac.dyn.optonline.net) has joined #freenet
[12:06] <KenMan> NGRoutingTable
[12:06] <toad_> hmmm
[12:07] <toad_> if we have less than a certain number of ndoes in the routing table, we want to reference the StoreData
[12:07] <toad_> nodes
[12:07] <toad_> this is to speed up bootstrapping
[12:07] <toad_> in that instance it definitely should not count nodes which are not connected
[12:08] <toad_> although it's debatable whether the check is useful
[12:08] <KenMan> now i kind of understand the purpose, but still not the code :o it is beyond me.
[12:08] <KenMan> btw thanks
[12:10] * toad_ sets timeout to at least 3000ms...
[12:15] <toad_> >Your opinion please? KenMan supports this. I'm not entirely sure. Also
[12:15] <toad_> >note that these are worst case - many requests will get sent before the
[12:15] <toad_> >timeout.
[12:15] <toad_> I am a bit concerned that we are making tweaks before we have had a
[12:15] <toad_> chance to properly observe the
[12:15] <toad_> effects of these pretty significant changes.
[12:15] <toad_> Can't we just chill for a while and take stock?
[12:16] <toad_> Ian.
[12:16] <toad_> that's in response to a mail suggesting that we increase the min queueing interval to 3 seconds
[12:16] * toad_ thinks he will do it anyway on unstable...
[12:16] <KenMan> tell him you are attempting to make the stock have value
[12:17] <KenMan> ;)
[12:17] <toad_> huh?
[12:18] <jay> bad joke
[12:18] <KenMan> I didn't think you were quite up to the point where you could take pause and evaluate.
[12:21] <toad_> queue runs seem to complete pretty fast at the moment...
[12:21] <toad_> usually about 10ms, up to maybe 1 second...
[12:24] <KenMan> any better handle on that doRescheule() trouble ?
[12:25] * toad_ hrmm
[12:25] <toad_> still a problem...
[12:26] <toad_> scheduled instances of it seem to multiply
[12:26] <toad_> either I need to add an am-i-scheduled-on-the-ticker method somewhere, and check it, or give it its own dedicated thread
[12:34] * toad_ gives it its own thread
[12:34] * salahx (salahx@sc1-24.217.174.147.charter-stl.com) has joined #freenet
[12:37] * toad_ restarts...
[12:44] * toad_ commits 60211
[12:45] <KenMan> the one nice thing about this backoff load component (because it dominates) is that the startup query spike is minimized
[12:46] <KenMan> targetting 75%, running at 80% with only 23 minutes uptime...
[12:46] <toad_> with no retries after QR?
[12:47] <toad_> 1 retry after QR?
[12:47] <KenMan> yes, 1 retry... time will tell...
[12:47] <KenMan> lots and lots of time :(
[12:47] * salahx waits for the snapshot to get rebuilt
[12:50] * toad_ doing a diff
[12:53] <KenMan> given the current network state it would be foolish to target much below 75% backoff - 50% would likely make incoming queries drop below 100 qph. As it is, this test makes my node less productive, but it shows the threshold of sanity when looking at controlling backoff.
[12:55] <KenMan> assuming, of course, that my set of mRIs for the routes is representative of most nodes :o
[12:58] <KenMan> still , 75% /could/ translate into 4th choice routing on average. But I think this 'formula' is flawed, unless NGR performed even distributions of queries across all routes.
[13:00] * KenMan longs for a routedToChoiceRank metric that doesn't count unconnected nodes...
[13:00] * toad_ changes topic to 'Upgrade to 5091 (5089: mandatory reset, 5090: RNF bug fixed!) | Unstable: Upgrade to 60211 | Logs: http://newton.matcmp.ncc.edu/~lockej/freenet/chanlog/ | #freenet-politics and #freenet-chat are available for offtopic discussions'
[13:00] <toad_> KenMan: so implement one
[13:00] <KenMan> i tried :( I will again...
[13:01] <toad_> maybe i should publish a 5092-prerelease more officially?
[13:01] <KenMan> how so ?
[13:01] <KenMan> oh in snapshots ? good idea. And while you are at it, when are you going to unpack those scripts into snapshots for me ?? :)
[13:03] * KenMan says, "there's no hurry"
[13:06] * galt-away is now known as galt
[13:09] * hobx_ (~chatzilla@ankh.math.chalmers.se) has joined #freenet
[13:11] <toad_> bbl
[13:13] * toad_ is tempted to release 5092 actually
[13:13] <toad_> soon
[13:13] <toad_> but for now bbl
[13:15] <cbreak> hmm... 4641 temp files...
[13:16] * matahari (~matahari@munyoo.cs.mu.OZ.AU) has left #freenet
[13:24] <KenMan> backedOffCount for first attempts averages 7.26 , for second try (1st retry) it averages 17.44
[13:25] <KenMan> i blame the instant RNFs of my routes, but I can't be sure without lots of effort. yuck.
[13:25] <KenMan> i have the data available, it's just a lot of work to calculate...
[13:26] <cbreak> compare the DS Size, before I restarted and after: http://dhcp-32-094.via-eth.ch/~cbreak/freenet/before/ds_histogram_data_detail.txt
[13:26] <KenMan> bucketize the elapsed times between first and second routings for each key that got retried. Are most in the sub-5 second bucket ?
[13:27] * KenMan waits for the after histogram...
[13:27] <cbreak> http://dhcp-32-094.via-eth.ch/~cbreak/freenet/after/ds_histogram_data_detail.txt
[13:28] <cbreak> I lost about 5000 keys
[13:31] <cbreak> Incidentialy, the temp folder contained a few thousand files, most with duplicated Hash Values...
[13:32] <toad_> KenMan: want my stable-unstable-merge source tree?
[13:32] <KenMan> i'm uncertain. Why not share the jar with everyone as a 5092-pre.jar in snapshots ?
[13:32] <KenMan> if you want me to adopt it, then sure.
[13:33] <toad_> well you'd need the source that's the point
[13:33] <KenMan> right now i have 5091 + MRI changes you made + backoff load component + max retries=1
[13:34] <KenMan> oh, and my special logging that tells me what it going on with requests and mRIs :)
[13:35] <toad_> do you want a tarball of my source changes?
[13:35] <toad_> your data is pretty meaningless without queueing
[13:36] <KenMan> I probably should synchronize with you through cvs... hmmm
[13:36] <KenMan> what tags would I need ?
[13:36] <toad_> mostly it's unstable
[13:36] <KenMan> I can wait until you make it officially stable, and then do my thing...
[13:37] <toad_> you could just get unstable and replace Version.java...
[13:37] <toad_> yeah, i'm pondering whether to just release it as is...
[13:37] <KenMan> but there are some unstable things that shouldn't be included yet ? oh, looks like 'no' :)
[13:37] <toad_> i think i've got the MAJOR bugs out...
[13:37] <toad_> i hope i have...
[13:37] <toad_> and IMHO it IS better than 5091
[13:37] <toad_> well, one way to find out anyway
[13:37] <KenMan> put out a prerelease jar for a day or two, see what reports you get
[13:38] <toad_> it doesn't have to be perfect...
[13:38] * toad_ thinks he will put it out as is
[13:38] <toad_> well no
[13:38] <KenMan> no, but we should avoid making major unforeseen impacts on stable . Not that it really matters that much.
[13:38] <toad_> i'll put the prerelease up for a few hours...
[13:39] <toad_> better to have it on snapshots...
[13:39] <KenMan> give it at least a day - you are in a hurry to get feedback, users are in a hurry to 'not break things with an upgrade' :)
[13:39] <toad_> than on my server
[13:39] <toad_> more credible :)
[13:40] <KenMan> more amiable to your node as well
[13:40] * KenMan says, "let my suck your lifeblood" ie. your outgoing bandwidth , for a while
[13:41] <toad_> you think that many would use it?
[13:41] <toad_> i doubt it...
[13:41] * toad_ will be busy tomorrow...
[13:41] <toad_> and away saturday to monday...
[13:41] <KenMan> just one transfer will take... how long ? 2M at ... how much bw ya got ?
[13:43] <KenMan> cbreak lost many keys at just a few points. Probably related to temp keys used with announcements.
[13:43] <mikeDOTd> i say release 5092. i've been running unstable with the stable Version.java file for days now (syncing periodicly), and it's more just as stable and useable as 5091
[13:43] <KenMan> fair enough
[13:43] <toad_> cool, and even with the bugs :)
[13:45] <KenMan> then i can cvs update against "-r stable"
[13:45] <cbreak> KenMan: My node recieves requests all the time, and its outbound limit is used up most of the time. Should it realy anounce?
[13:46] <KenMan> I don't know, but my moderate level of knowledge would blame the announcement mechanism. The biggest lost was at 51 - 777 keys went away.
[13:46] <KenMan> A2 lost 528 keys... 47 lost 493 keys...
[13:47] <toad_> hmmm
[13:47] <KenMan> but typical is 0-5 key loss in any one bucket.
[13:47] <toad_> we need to ensure that the QueryRejected reasons don't cause a space leak in OCM's thingy...
[13:49] <KenMan> securing against a potential overflow of number of buffers thing ?
[13:49] <cbreak> I have %useDSIndex=true
[13:49] <KenMan> so try setting it to false and see if you still get that weird behavior
[13:50] <toad_> KenMan: possible remote DoS via memory exhaustion, OOM on long running nodes
[13:50] <KenMan> cbreak: i don't know what that will tell us, but it would mean something one way or the other
[13:51] <cbreak> I will try it. But that kind of key duplication is not generaly reproducable.
[13:51] <KenMan> heh, good to know :)
[13:52] <toad_> bbiab
[13:54] * j4n (~j4n@aum.inter-i.wohnheim.uni-mainz.de) has joined #freenet
[14:00] <KenMan> cbreak - give it a few days with index still enabled, and see if it happens again. If so, then give a few days without indexing.
[14:00] <cbreak> ok.
[14:01] <KenMan> i suppose temp file creation doesn't really interact with the index, but give it a shot
[14:01] <KenMan> and REPORT BACK ! one way or the other...
[14:02] <cbreak> maybe the node somehow could not move a file into the store...
[14:02] <cbreak> well... I will :)
[14:02] <KenMan> i'm pretty sure it has something to do with providing sample keys for another node to search for. I don't understand it at all, but I know it exists.
[14:02] <KenMan> and it must be related to announcements. At least, that is my best guess.
[14:03] <cbreak> I wonder if this happened to other users before.
[14:04] <cbreak> Maybe it is uniqe to Sun Java for Mac OS...
[14:04] <KenMan> probably... someone somewhere, who didn't go looking for it.
[14:05] <KenMan> maybe it happens to lots of people... but very infrequently. who knows ?
[14:06] <KenMan> Ooh, i like what 75% backoff target and single retry max is doing to my graphs.
[14:09] <KenMan> first try average backoff is 7.7 of 50 routes, retries are averaging 17.6 (of 50) backoff. Total average backoff at any one moment is 81% . 1003 qph for first full stablized hour. And not as many incoming mRI spikes.
[14:10] <KenMan> I wonder what it would do with NO retries...
[14:11] <KenMan> backoff target sets a reasonably well behaved cap on the incoming query rate. Average mRI out is 140644ms
[14:15] <jay> ugh Outlook Express is such a piece of shit
[14:16] <jay> there should be jail time for it's users
[14:16] <KenMan> or its creator, anyway
[14:16] <cbreak> how unfortunate jails only work on BSD... :)
[14:17] <KenMan> heh. but even some criminals manage to perpetrate more crimes from behind the bars of a chroot jail...
[14:17] <jay> anyone has the right to create something like a pet rock
[14:18] <jay> it's those who buy it who need their heads examined
[14:18] <jay> im trying to get Newsbyte to ditch using OE to post his messages
[14:18] <KenMan> some pet rocks have immense artistic value, in the eye of the be-payer .
[14:21] <KenMan> this graph is rock steady - http://mywebpages.comcast.net/jkcorson/FreeNet/5091c.png
[14:22] * KenMan thinks it's a little spooky
[14:23] <KenMan> I would get better volume of successful routings if I allowed unlimited retries, but that's just because routing is fairly random.
[14:24] <KenMan> s/fairly/quite
[14:33] <KenMan> I saw the coolest thing the other day- a jumping spider took down a small bee, while it was flying. I don't think they are bees, but they look like them, only smaller.
[14:33] <KenMan> the spider used a strand as an anchor, and the bee kept them in the air for several seconds before they fell onto the ground. Then the spider ate him.
[14:34] <KenMan> I've never seen *anything* like it before.
[14:35] <cbreak> hmm... Mutant Spider. If it had bitten you, imagine the possibilities...
[14:35] <KenMan> the spider positioned himself only inches from where the 'bee' kept hovering before making his deadly strike.
[14:35] <salahx> you might get you own movie series!
[14:36] <KenMan> this particular spider looks more like a crab, with big pincher-looking things on the front. Those are its sexual organs, if i recall. neither one was larger than your fingernail.
[14:36] <salahx> naah. A human with spider-like powers. That would enver sell...
[14:36] <KenMan> heh
[14:37] <KenMan> these 'bees' like to hover very close to me, and usually will land on my skin. Attempts to wave them away are futile. But they can't sting.
[14:37] <KenMan> i don't know exactly what they are, but they are mildly irritating.
[14:37] <cbreak> I wonder if those animals are preprogrammed (DNA) or if they learn such things...
[14:37] <KenMan> which, the spider or the bee ?
[14:38] <cbreak> the spider. Being iritating is not hard...
[14:38] <toad_> actually it's more likely to be related to iakin's bug...
[14:38] <salahx> the spider ate iakin's bug ?
[14:39] <KenMan> well, all spiders have the most amazing vision - even the smallish ones can detect you from 10 feet away. They have 8 eyes (or is it 10 or 12) that see in all directions at once.
[14:41] <KenMan> to clarify, both critters were about the size of your fingernail. I didn't mean the spider's 'thing's
[14:41] <toad_> no, the tempfile leak
[14:42] <KenMan> woops, there goes my theory that I was affecting the spikes in mRI values from my routes... i just republished the graph
[14:43] <toad_> huh?
[14:43] <hirvox> wierd. the JVM-crashes-silently-on-System.runFinalization() bug is gone
[14:44] * KenMan decides to be patient and let the node run for many hours before casting a curious eye upon the data...
[14:49] <KenMan> i just checked the load component into cvs (disabled/commented out inside Node.java) to replace the old one for backoff
[14:49] * toad_ making changelog, checking over changes, fixing any minor issues...
[14:50] <toad_> PRI 26/08/2004 20:08:58.128 (freenet.support.SimpleDataObjectStore, main, ERROR): Size was wrong reading in SimpleDataObjectStore (Got EOF after 26/394 items read). Truncating to 26 items. Datafile name: '/var/lib/freenet-stable/rtprops_b', datafile size: 205357
[14:50] <toad_> yikes
[14:50] <toad_> does that mean i just lost my routing table?
[14:51] <toad_> apparently not...
[14:51] <KenMan> only the left half of it, huh ?
[14:51] <KenMan> j/k
[14:52] <toad_> KenMan: that code is likely to get deleted if it's just commented out...
[14:53] <KenMan> i replaced the last attempt, which was still around. I just wanted it to be available, in case you or someone else got curious about playing with it.
[14:53] <KenMan> *still around but commented out
[14:54] <KenMan> it would only take 5-10 minutes to code it up again anyways...
[14:54] * gregh (~greg@defender.hewgill.net) Quit (Read error: 54 (Connection reset by peer))
[14:54] <KenMan> that Ayatollah in Iraq just risked his life by sitting down to talk with Al-Sadr - the guy who murdered the other ayatollah some time ago in order to gain more followers
[14:55] <toad_> cool
[14:56] <toad_> [ot] /me has heard good things about him... e.g. his mob appears to have largely kicked al-sadr's mob out of some area or other
[14:56] <toad_> but massively ot :)
[14:56] <KenMan> sometimes religion can avoid bloodshed, other times it can cause it.
[14:56] <toad_> [ot] religion is used by people as an excuse, just like the freedom, the nation state and any number of other things
[14:56] <hobx_> I wouldn't idolize Al-Sistani. I think he is after his own ends just like everybody else.
[14:57] <toad_> [OT] absolutely
[14:57] <hobx_> Perhaps they managed to put a chip in his brain during the surgery or something though.
[14:57] <toad_> hehe
[14:57] * toad_ doubts that's possible yet, but nice idea
[14:58] <hobx_> (Seriously, if you were the Americans wouldn't you? I mean, I guess the don't have brain control, but if you have somebody who could ostensibly become a troublesome leader under surgery, wouldn't you place a small radio controlled kill chip somewhere?)
[14:59] <toad_> when was he under american surgery?
[14:59] * gregh (~greg@defender.hewgill.net) has joined #freenet
[14:59] * toad_ hrrm
[14:59] <hobx_> British in London during the last several weeks
[14:59] <toad_> i announce my intention to release 5092 soon... and the node starts spamming log messages...
[14:59] <toad_> mostly Unrecognized trailer ID
[15:00] <toad_> rather odd as I'm sure we've sent the kill signal many many times...
[15:06] <KenMan> is "announcing your intention to update stable" = "send the kill signal many many times" ? :p
[15:06] <toad_> heh
[15:06] * interrupt (~chatzilla@64.122.23.213) Quit ("Chatzilla 0.9.64f [Mozilla rv:1.7/20040707]")
[15:12] <toad_> KenMan: the kill signal for the trailers
[15:14] * Hory (~Miranda@81.196.25.110) Quit ("Visit http://CyberLore.net for recommendations on the best games, websites and freeware. Don't settle for anything else!")
[15:14] * toad_ has said too much!
[15:15] <cbreak> ok. again unusual key numbers: http://dhcp-32-094.via-eth.ch/~cbreak/freenet/ds_histogram_data_detail.txt
[15:16] <toad_> cbreak: strange...
[15:16] <cbreak> KenMan: Consider this a report :)
[15:16] <toad_> very strange
[15:16] <cbreak> I restart my node, and change to no ds index.
[15:17] <toad_> and when you restart they are gone?
[15:17] <cbreak> yes.
[15:17] <toad_> from the histogram?
[15:18] <cbreak> http://dhcp-32-094.via-eth.ch/~cbreak/freenet/
[15:18] <cbreak> take a look at before and after
[15:18] <toad_> are there many many files in temp that correspond to those keys?
[15:19] <cbreak> there where a few thousand files, and a lot of those had the same sha1 hash.
[15:19] * salahx (salahx@sc1-24.217.174.147.charter-stl.com) Quit (Read error: 110 (Connection timed out))
[15:19] <toad_> hmmm
[15:19] * toad_ thinks he will fix after 5092
[15:19] <cbreak> this time there where only few temp files, but also smaller peaks.
[15:26] * j4n is now known as j4n`na
[15:35] <KenMan> average backoff holding at 81% , average outgoing mRI holding at 141s for 3 hours now.
[15:38] <KenMan> average mRI received is 173s, but average mRI per minute averaged -> 378s , because lower mRI values tend to be communicated more often.
[15:41] <KenMan> my outgoing mRIs don't tend to wander as much as the incoming from multiple hosts - after eyeballing the data (some drift below 100s others go slightly above 200s) I stand by the 141s as a reasonably accurate measurement.
[15:42] <KenMan> so the outgoingRequestInterval diag is more believable than the incomingRequestInterval diag, in my book
[15:44] <KenMan> bbmuchlater
[15:59] <toad_> okay, checked out, generated changelog for cvs
[16:16] * guido^pe (~unknown@dsl-082-082-155-213.arcor-ip.net) Quit (Read error: 110 (Connection timed out))
[16:17] * guido^pe (~unknown@dsl-213-023-244-193.arcor-ip.net) has joined #freenet
[16:19] * toad_ changes topic to 'Upgrade to 5092 (5089: mandatory reset, 5090: RNF bug fixed!, 5092: lots of changes including queueing) | Unstable: Upgrade to 60211 | Logs: http://newton.matcmp.ncc.edu/~lockej/freenet/chanlog/ | #freenet-politics and #freenet-chat are available for offtopic discussions'
[16:19] <toad_> 5092 is out1
[16:19] <toad_> 5092 is out!
[16:20] <toad_> ARE YOU LISTENING CofE ?! :)
[16:20] <cbreak> with Queueing?
[16:20] <toad_> yup
[16:25] * interrupt (~interrupt@64.122.23.213) has joined #freenet
[16:26] <toad_> hi interrupt
[16:27] <interrupt> hi
[16:28] <vsalento> I get exception when trying to get autoRequester.doComputeCHK(bucket) to work ... NPE at freenet.support.io.NIOOutputStream.write(NIOOutputStream.java:118)?
[16:29] <toad_> vsalento: hmmm
[16:29] <toad_> interrupt: running freenet?
[16:29] <toad_> vsalento: show me the trace
[16:29] <toad_> privmsg
[16:30] <interrupt> toad: yes, I am
[16:30] <toad_> interrupt: stable?
[16:30] <vsalento> on freenet server side the NIO input stream is closed: waited more than 120000ms
[16:30] <toad_> interrupt: upgraded to 5092 yet?
[16:30] <interrupt> toad: upgrading right now
[16:30] <toad_> vsalento: hmm, not surprising...
[16:30] <toad_> vsalento: privmsg me with the stack trace
[16:32] <vsalento> java.lang.NullPointerException
[16:32] <vsalento> at freenet.support.io.NIOOutputStream.write(NIOOutputStream.java:118)
[16:32] <vsalento> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:66)
[16:32] <vsalento> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:124)
[16:32] <vsalento> at freenet.client.FCPClient$FCPInstance.negotiateConnection(FCPClient.java:312)
[16:32] <vsalento> at freenet.client.FCPClient$FCPInstance.run(FCPClient.java:258)
[16:32] <vsalento> at java.lang.Thread.run(Thread.java:534)
[16:32] <toad_> ok
[16:32] <toad_> what build?
[16:33] <vsalento> toad_: unstable newest from anon-cvs
[16:34] <toad_> okay
[16:34] <toad_> does that line say:
[16:34] <toad_> if(!wsl.send(b,off,len,chan,this,WriteSelectorLoop.NEGOTIATION))
[16:34] <toad_> throw new SocketException("Could not send data: WSL.send returned false");
[16:34] <toad_> ?
[16:34] <toad_> if so, the only way that can NPE is if WSL is null...
[16:34] * Zorix (Brandon@fl-65-40-16-125.dyn.sprint-hsd.net) Quit (Read error: 110 (Connection timed out))
[16:35] <toad_> suggest you put an if(wsl == null) throw new NullPointerException() in the constructor...
[16:35] <toad_> after it's assigned
[16:35] <toad_> and then try again
[16:35] <vsalento> yes
[16:35] <toad_> that'll show where the real problem is
[16:35] <toad_> what are you running btw?
[16:35] <toad_> your own code or freenet.client.cli.Main ?
[16:37] <vsalento> my own code that tries to use freenet
[16:38] <toad_> ok
[16:38] <toad_> well do what i said above and get the stack trace from that (it will NPE earlier)
[16:39] <cbreak> what was the problem with start-freenet.sh on Darwin?
[16:39] <toad_> and that'll probably tell me what the problem is
[16:39] <toad_> cbreak: JRE 1.4.2 on mac doesn't have the direct memory limit setting
[16:39] <toad_> all other 1.4.2 JREs do
[16:39] <cbreak> ah. Every time I used 1.4.2 I got Kernel Panics with Freenet.
[16:40] <cbreak> well... last time I tried was about one year ago...
[16:41] <toad_> on what os?
[16:41] <cbreak> Mac OS X
[16:41] <toad_> ahhh
[16:41] <toad_> hmmm
[16:42] <toad_> bbiab
[16:42] <toad_> vsalento: i'm toad@amphibian.dyndns.org if you need to email me
[16:42] <toad_> but i'll probably be back later
[16:47] * jay (jcl@ool-18bf6dac.dyn.optonline.net) Quit ("bbl")
[16:51] * Sugardude (~Sugadude@216.17.101.36) Quit ()
[16:57] * Zorix (Brandon@fl-65-40-16-125.dyn.sprint-hsd.net) has joined #freenet
[16:58] * Glash (John@myw-stp-196-34-113-48.sentechsa.net) has joined #freenet
[16:58] <Glash> OMG! unstable is actually working rather well!
[17:02] <verl> wish i could say the same for stable...oh well, only 4 ppl on 5092 yet
[17:02] <cbreak> already? nice :)
[17:04] <Glash> is DFI still available?
[17:04] <cbreak> On Unstable? No. Maybe an old version
[17:05] <Glash> what a shame, well the way unstable is working now things should start looking up.
[17:06] <cbreak> The last Version I ever got: http://localhost:8888/SSK@sPolFAwS69lG2Le~TTRzNEVarcEPAgM/40b28c80-DFI//
[17:06] <cbreak> (Note the static DBR Key)
[17:06] <cbreak> Its from 2004.5.24
[17:19] <verl> lots of these: 2004-aug-26 23:37:33 (freenet.node.rt.EdgeKludgingBinaryRunningAverage, YThread-12, ERROR): Too high probability: 1.028 while kludging freenet.node.rt.EdgeKludgingBinaryRunningAverage@edc88b (-28 0s, 1028 1s, 1000 total), init=0.5, index=139
[17:19] <verl> java.lang.Exception: debug
[17:19] <verl> cpu stuck at 100% usage
[17:27] <iip_i2p> <Geoff> Ok. Does anyone have the problem when using Freenet of it just taking ages to load up all the files. Then if you go to configure anything, it wont allow you to Open Gateway again? Windows XP os we're talking here.
[17:28] <cbreak> The Gateway is just http://localhost:8888/
[17:28] <cbreak> a simple web page on freenets local HTTP Server "FProxy"
[17:29] <lexx> toad_ is updateing unstable alot heh
[17:32] <iip_i2p> <Geoff> Yeah but I mean when I've tried configuring anything - actually even if i dont - but instead simply click Ok, the freenet node software doesnt work.
[17:33] <cbreak> but you still can open the gateway yourself? If yes, freenet works.
[17:34] <cbreak> Note: If you have changed the config file, you need to restart the node in most cases.
[18:03] <Glash> I've alway wondered about they issue of some images that are already in the store etc. not loading on browser refreshes, does anyone know what is going on?
[18:04] <Glash> They may load, but can really take a while.
[18:05] <cbreak> some browsers only open a few connections to fproxy
[18:05] <Glash> fproxy seems very slow, is that normal?
[18:05] <cbreak> if all are used to retrieve files not in your DS, they may block until sucessfull or failure
[18:06] <cbreak> fproxy is not slow here, until I actualy want to retrieve something out of freenet.
[18:07] <Glash> Ok, I understand, I once had the Firebirds settings increase the number of threads but I can't remember where I got the info.
[18:08] <cbreak> firefox? :)
[18:09] <Glash> Sorry, yeah, Firefox's
[18:21] * interrupt (~interrupt@64.122.23.213) Quit (Read error: 110 (Connection timed out))
[18:22] * Zorix (Brandon@fl-65-40-16-125.dyn.sprint-hsd.net) Quit (Read error: 110 (Connection timed out))
[18:26] <Glash> OK, solved, found my old prefs.js.
[18:28] * dystopia_ (admin@gaia-wan0.ipv6.freeshell.bofx.net) has joined #freenet
[18:28] <cbreak> yes. Seems to be related with network.http.max-connections and max-connnections-per-server
[18:30] * lolo-laptop (~lostlogic@68.251.84.226) Quit ("Client exiting")
[18:33] * Ash-Fox (Hal-9000@aat81.neoplus.adsl.tpnet.pl) Quit ()
[18:34] * Ash-Fox (Hal-9000@aat81.neoplus.adsl.tpnet.pl) has joined #FreeNET
[18:37] <Glash> cbreak, BTW I got DFI
[18:37] <Glash> the one you gave me
[18:38] * sanity_m (~sanity_m@future-is.orange.co.uk) has joined #freenet
[18:38] * sanity_m (~sanity_m@future-is.orange.co.uk) Quit (Client Quit)
[18:39] * dystopia (admin@gaia-wan0.ipv6.freeshell.bofx.net) Quit (Read error: 110 (Connection timed out))
[18:39] * dystopia_ is now known as dystopia
[18:44] <cbreak> Glash: Its an old Edition, and since Dolphin seems to have switched to Stable (not sure), the last which was inserted
[18:45] <Glash> yeah, I gathered, but still usefull.
[18:46] <cbreak> You might also try TFE
[18:47] <Glash> Think freenet would run comfortably on a 256/128 connection?
[18:48] <cbreak> maybe, if it runs constantly. But adjust input and output BW limit acordingly.
[18:49] <Glash> And anything lower is probably a wast of time, the node would probably just get swamped?
[18:51] <cbreak> I don't know. If the node is bad, it should just be dumped, and not used by others.
[18:51] <Glash> ah, yes.
[18:52] <cbreak> It should not hurt the network, at worst, it just doesn't work :)
[18:53] <Glash> I'm amazed with how well 60211 is working, haven't seen this performace in ages.
[18:54] <Glash> I'll give my node some time to catch-up, it's at 130% load.
[18:55] <Glash> Need more BW.
[18:55] <cbreak> hmm... fun: local news report that a barrel of highly poisonous stuff was lost...
[18:55] <Glash> go help look for it.
[18:56] <Glash> Now, that would be fun.
[18:56] <cbreak> I don't intend to leave my house in the next few days... :)
[18:57] <Glash> smart
[18:58] <Glash> Gee, and I thought Freenet was dieing.
[18:58] <Glash> far from it.
[18:59] <cbreak> its like with BSD, Apple and all the others...
[19:00] <Glash> I'm just dreading the next bit experimental code :)
[19:00] <cbreak> after queueing, there might come stuff-coalescing and fixed size keys.
[19:01] <Glash> stuff-coalescing? any URLs?
[19:01] <cbreak> you may investigate on the dev list
[19:02] <Glash> ok, missed that.
[19:02] <cbreak> that stuff is requests and stuff :)
[19:02] <cbreak> (or was it messages? Don't know...)
[19:03] <Glash> ah, yes, I see.
[19:04] <Glash> Wow, 60211 is working like a champ, back down to 100% load, whoohoo.
[19:07] <Ash-Fox> I had a visit from the police today
[19:07] <Ash-Fox> They threatened to take away all my hardware
[19:07] <cbreak> reason?
[19:07] <Ash-Fox> Some idiot ran a ebay scam site on my free webhosting
[19:08] <Glash> Nasty
[19:08] <cbreak> I hope you could settle that better
[19:08] <Ash-Fox> I am going to write a large article about why we shouldn't support ebay, especially when they are calling the cops on people before investigating!
[19:09] <Ash-Fox> Hmm, I think I would of fought the cops if they tried to take anything
[19:09] <cbreak> not healthy. what country are you in?
[19:10] <Ash-Fox> Poland, at least it's not as bad as in the states
[19:10] * Ash-Fox has a homemade handtaser he built :P
[19:10] <Ash-Fox> always in my pocket
[19:10] * galt is now known as galt-away
[19:11] * ornge (~ornge@h131n2fls34o879.telia.com) has joined #freenet
[19:11] <cbreak> ok, not healthy for them :)
[19:11] * ornge (~ornge@h131n2fls34o879.telia.com) Quit (Client Quit)
[19:11] <Ash-Fox> Not to mention I know how to torcher information out of people using one finger :P
[19:11] <cbreak> Ever tested that thing on yourself? If you make mistakes, you can kill someone.
[19:12] <Ash-Fox> cbreak, well, I know it's enough to send someone to hospital
[19:12] <Ash-Fox> Which suits my needs
[19:13] <cbreak> Hmm... I for one would cooperate... Violence only makes the situation worse.
[19:13] <Ash-Fox> I would cooperate, to a limit
[19:13] <Ash-Fox> I would not, under no circumstances let them take the computers
[19:14] <Ash-Fox> I have nothing to hide
[19:15] <Glash> I'm sure the Nips would agree.
[19:15] <cbreak> Hmm... I have lots of crypto containers. For all my stuff... Homework, Programms, Source Code... And such a bad memory for Pass Phrases... (Maybe that works...)
[19:16] <Ash-Fox> Glash, Nips?
[19:16] <Ash-Fox> in a raid, I would effectivly attempt to disable the hostiles
[19:16] <Glash> Nipponese
[19:16] <Glash> Japanese
[19:16] <Ash-Fox> Glash, hmm?
[19:18] <Glash> I lost Gig & Gigs from lost passphrases and keys, etc, used to keep many keys on a floppy, bad idea.
[19:19] <Glash> I need a battery powered memory stick.
[19:19] <cbreak> I saw USB Sticks with Fingerprint reader.
[19:20] <Glash> Batteries out, data gone.
[19:20] <cbreak> The only USB Memory Sticks I have seen worked with EEPROM
[19:20] <cbreak> (Or some kind of Flash Memory)
[19:20] <Glash> Yeah, I know, it sucks.
[19:20] <toad_> just use flash
[19:21] <toad_> <Glash> OMG! unstable is actually working rather well! - how so?
[19:21] <Glash> Won't really help though, distroying evidence is also a crime.
[19:21] <toad_> i mean, how can unstable work well? how is that possible with its sparsity of content?
[19:22] <Glash> road_, well compared all my other recent attempts it's keeping up well and running smoothly.
[19:22] <toad_> <verl> lots of these: 2004-aug-26 23:37:33 (freenet.node.rt.EdgeKludgingBinaryRunningAverage, YThread-12, ERROR): Too high probability: 1.028 while kludging freenet.node.rt.EdgeKludgingBinaryRunningAverage@edc88b (-28 0s, 1028 1s, 1000 total), init=0.5, index=139
[19:22] <toad_> <verl> java.lang.Exception: debug
[19:23] <toad_> <verl> cpu stuck at 100% usage
[19:23] <toad_> on what?
[19:23] <Glash> Very few RNFs, buffers well, etc
[19:23] <Glash> XP sp1, SMP athlon 2k+
[19:23] <Glash> latest JRE
[19:24] <Glash> In fact, I don't think I've seen one RNF so far.
[19:25] <toad_> <Glash> I'm amazed with how well 60211 is working, haven't seen this performace in ages. - oh?
[19:25] <Glash> Running Fuqid, and it''s working great, slow but steady.
[19:25] <toad_> <Glash> Wow, 60211 is working like a champ, back down to 100% load, whoohoo. - hmm, /me is skeptical ;)
[19:26] <Glash> I'm BW limited, heavly, and yet it recovers from load.
[19:26] <toad_> Ash-Fox has a homemade handtaser he built :P - umm, firing weapons at the police is not a good idea!
[19:26] <Ash-Fox> toad_, it's not a gun
[19:26] <Ash-Fox> toad_, you basically have to stab it into the hostile
[19:27] <toad_> <cbreak> Hmm... I have lots of crypto containers. For all my stuff... Homework, Programms, Source Code... And such a bad memory for Pass Phrases... (Maybe that works...)
[19:27] <toad_> use a stego fs
[19:27] <toad_> then you can tell them the key to your porn collection
[19:27] <toad_> and say that's all their is
[19:27] <Ash-Fox> toad_, I don't have any porn
[19:27] <Glash> Yeah, those Polish police can get a bit rough, gotta show 'em who's boss.
[19:27] <toad_> make sure some of it's goat porn and they won't bust you for the Important Stuff ;)
[19:27] <Ash-Fox> toad_, I pretty much don't store anything here that is valuable
[19:28] <toad_> Glash> I lost Gig & Gigs from lost passphrases and keys, etc, used to keep many keys on a floppy, bad idea. - scary
[19:28] <Glash> Yeah, I'm REALLY into MILFS.
[19:28] <toad_> <Ash-Fox> toad_, you basically have to stab it into the hostile - then how are you safe?
[19:29] <toad_> MILFS? is that like Rubberhose.org?
[19:29] <Ash-Fox> toad_, I don't have a fear for endangering my own life
[19:29] <cbreak> Ash-Fox: Still Alive? Surprising...
[19:30] <Glash> MILFS=Old hags
[19:30] <Ash-Fox> cbreak, Eh, I've been shocked, bones broken, stabbed, beaten
[19:30] <Ash-Fox> And pretty much I come out fine everytime
[19:31] <cbreak> scary, realy... :)
[19:31] <Ash-Fox> cbreak, all of them were caused by people stepping over my boundries
[19:31] <Ash-Fox> I clearly warn them... and yet they think I am bluffing...
[19:32] <toad_> okay, anyone have any comments on 5092?
[19:33] <toad_> SHIT
[19:33] <toad_> :<
[19:33] <toad_> all my queue runs are taking AGES
[19:33] <toad_> hmmm
[19:33] <toad_> they were
[19:33] <toad_> they came down for a bit..
[19:33] <toad_> now they're up again but not as much mostly... hrrrm
[19:33] <Glash> Last I tried stable is was useless.
[19:34] <toad_> well I'll just have to implement the faster queue run code when i get back on monday
[19:34] <toad_> Glash: what was the problem?
[19:34] * Zorix (Brandon@fl-65-40-16-125.dyn.sprint-hsd.net) has joined #freenet
[19:34] <toad_> last time i tried unstable it was useless
[19:34] <toad_> because nobody uses it, no content
[19:35] <Glash> Lots of RNFs, can't get of the node messages etc.
[19:35] <toad_> that's largely fixed in 5092
[19:35] <toad_> because of queueing
[19:35] <Glash> very few nodes connections, even with a good seed file
[19:35] <Glash> Ok
[19:36] <Glash> If noone is using stable why is the seedfile so large?
[19:36] <toad_> nobody is using UNstable
[19:36] <toad_> that's what I said :)
[19:36] <Glash> ah, sorry
[19:36] * toad_ hrrm
[19:37] <cbreak> unstable is used, I think. At least by someone who is using up my bandwith...
[19:37] <toad_> just for tonight i think i'll knock off around now (1:00) and play until 2:00, and then go to bed. especially as i still have that headache.
[19:37] <toad_> which may not get better with warcraft, but we'll see :)
[19:37] * toad_ doesn't have the energy to go for a walk
[19:37] <Glash> This new code in unstable seems to be working well, will it make it into stable?
[19:37] <toad_> is there any urgent business here?
[19:37] <toad_> Glash: 5092 ~= 60211
[19:37] <toad_> it's very close
[19:37] <cbreak> Has Freenet idle traffic, or is all originated by requests and inserts?
[19:38] <toad_> 5092 should work pretty well based on what I've seen here
[19:38] <toad_> cbreak: ultimately yes, mostly
[19:38] <toad_> Glash: I merged 99% of 60211 into 5092
[19:38] <Glash> I'll give stable a bash soon.
[19:38] <toad_> okay
[19:38] <toad_> well bbl
[19:39] <Glash> Excellent.
[19:39] <toad_> i may not be back tomorrow
[19:39] <toad_> otoh i may be
[19:39] <toad_> it shouldn't take all day to pack etc
[19:39] <toad_> I will definitely not be here on saturday
[19:40] <toad_> i'll be back on tuesday 31st probably
[19:40] <toad_> so if 5092 is abysmally bad despite first appearances, it's my fault, and i apologize in advance
[19:40] <toad_> bye...
[19:40] * toad_ (toad@82-32-16-91.cable.ubr03.azte.blueyonder.co.uk) Quit (Remote closed the connection)
[19:40] <Glash> cheers, thanks
[19:43] * cbreak (~cbreak@dhcp-32-094.via-eth.ch) Quit ("leaving")
[19:44] * puphone_ (~puphone@future-is.orange.co.uk) has joined #freenet
[19:55] * puphone_ (~puphone@future-is.orange.co.uk) Quit ("Virca 1.1.18")
[20:18] <Ash-Fox> haha http://127.0.0.1:8888/servlet/nodeinfo/networking/ocm/combined?identity=bda9d830e494847726c70e11b1d3a4745b1e8ae8d2f65e6cac92bbe219ff7d2a34e08713f3e839b2a92c340da97ee5a9397511b588ce42439df27fe2cc98aa6b44f9fdae5ccbb703073d9cfc72bd1cd1b7b277abd35669231e51b5036bb9d004c77b50038fa3483274f502526ba8da560e60eb6d00e91f16736c46bd62469867
[20:18] <Ash-Fox> hmm
[20:19] <Ash-Fox> I'm looking for that guy who had modified his peer pic to a smiley face
[20:34] * osh (~chatzilla@h88n2fls307o1039.telia.com) has joined #freenet
[20:36] * Zorix (Brandon@fl-65-40-16-125.dyn.sprint-hsd.net) Quit (Read error: 110 (Connection timed out))
[20:37] <osh> Hmm, I get a virus warning about "downloader-og" when trying to update to the latest snapshot. Is this intentional?
[20:38] <osh> Some adware/spyware thingy according to NAI.
[20:40] <lexx> lol
[20:42] * osh doesn't feel like installing spyware. It seems to find it's way there anyway. :-/
[20:44] <lexx> it ant spyware
[20:44] <lexx> need to know so i am not deling lots of stuff
[20:44] <lexx> going from unstable to stable
[20:44] <lexx> apart from chageing ports
[20:44] <lexx> what i need to delete
[20:46] * lexx any one
[20:46] <osh> lexx: You are correct. PandaAV calls it a Trojan. http://www.pandasoftware.com/virus_info/encyclopedia/overview.aspx?IdVirus=50445&sind=0
[20:48] <Glash> Just tried stable, didn't work for me as well as unstable, perhaps unstable hardcodes some settings.
[20:49] <lexx> for how long
[20:50] <lexx> Glash
[20:50] <Glash> hour or so
[20:51] <lexx> heh
[20:51] <lexx> try an day
[20:52] <lexx> woho FIND is working
[20:52] <Glash> Why is unstable sa fast at intergrating, fewer nodes
[20:52] <lexx> i found unstable was a bit unstable
[20:53] <osh> None of You guys get this virus-warning when trying to update snapshot to latest version?
[20:54] <lexx> i use avg nope
[20:54] <lexx> ignore it
[20:54] <osh> lexx: I take my paranoia seriously. That's why I run freenet... =)
[20:54] * guido^pe (~unknown@dsl-213-023-244-193.arcor-ip.net) Quit (Remote closed the connection)
[20:55] <lexx> heh
[20:55] <lexx> i had windows on my pc for an year and i ant got no adware spyware on my pc
[20:56] <lexx> most users dount know what thay are clicking on
[20:58] <lexx> any way my stable is now up
[20:58] * lexx is now known as lexx-Zzzz
[20:58] <lexx-Zzzz> nn
[21:05] * qwerty (John@myw-stp-196-34-112-106.sentechsa.net) has joined #freenet
[21:29] * Glash (John@myw-stp-196-34-113-48.sentechsa.net) Quit (Read error: 110 (Connection timed out))
[21:55] * Zorix (Brandon@fl-65-40-16-125.dyn.sprint-hsd.net) has joined #freenet
[22:06] * E1ven (~no@pool-151-203-126-250.bos.east.verizon.net) has joined #freenet
[22:08] * E1ven (~no@pool-151-203-126-250.bos.east.verizon.net) has left #freenet
[23:50] <qwerty> can someone please send me a default config file for unstable
[23:52] <mikeDOTd> 'java -jar freenet.jar --config' will generate a default config
These logs were automatically created by Jay Oliveri with his gimp hapi on irc.freenode.net.
