Timestamps are in GMT/BST.
[0:15] * icekey (~yosukey@cat.cs.mu.OZ.AU) has joined #freenet
[0:16] <icekey> Is anyone a freenet developer?
[0:27] <mikeDOTd> toad is the main, paid developer
[0:27] <mikeDOTd> there are a few people in here that assist him though
[0:37] <icekey> i guess i'll wait for toad
[1:16] * plixed (~plixed@pD9551513.dip.t-dialin.net) Quit (Read error: 110 (Connection timed out))
[1:49] <KenMan> hee hee, a kiddie-porner got busted in my state after a tornado ripped up his house. Construction workers discovered the huge porn cache while making repairs to the suspect's home.
[1:49] * mikeDOTd (~mikedotd@plutonium.nexthop.net) Quit (Read error: 54 (Connection reset by peer))
[1:49] <KenMan> So presumably, they felt their civic duty was to turn him in. See, sometimes good things happen to bad people :)
[1:52] <KenMan> here in USA, even when they catch the criminal and he confesses to a crime, he is still called a 'suspect' for legal reasons, until after he is sentenced.
[1:53] <KenMan> It covers the possibility that the man avoids a conviction by claiming "the tornado must have put the material INTO my house, rather than sucking them out and strewing them about my neighborhood. It was my neighbor's porn"
[1:54] <KenMan> because stupid things have happened: after criminals get away with a crime, they can then sue people who referred to them as anything other than "suspect"
[2:05] <Ash-Fox> "He called me his 'bitch'!"
[2:20] <icekey> Where do you get this stuff :)
[2:35] <Ash-Fox> http://www.livejournal.com/users/ash_fox/8538.html
[3:48] * qumqats (qumqats@outel.org) has joined #freenet
[4:07] * icekey (~yosukey@cat.cs.mu.OZ.AU) Quit ("Leaving")
[4:25] * qumqats (qumqats@outel.org) Quit ("Download Gaim: http://gaim.sourceforge.net/")
[4:44] <toad_> KenMan: no, it's more to do with "they forced me to sign the confession; they forged the confession; etc"
[4:44] <toad_> which does happen occasionally
[4:45] * toad_ got some new nice graphs
[4:49] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-100x25.png
[4:49] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-100x25-zoom.png
[4:59] * Hory (~Miranda@81.196.25.110) has joined #FreeNet
[5:00] <toad_> hi
[5:09] * tyesinclair (~tyesincla@170.252.64.1) has joined #freenet
[5:23] * tyesinclair (~tyesincla@170.252.64.1) has left #freenet
[5:33] * toad_ back
[5:42] <ShaunMacPherson> wb
[5:56] * _kno_ (~kvirc@212.145.186.157) has joined #freenet
[6:04] <_kno_> hi, why cant I see nothing in freenet?
[6:11] <ShaunMacPherson> hi
[6:11] <ShaunMacPherson> Did you connect to local host?
[6:22] <_kno_> ShaunMacPherson: yes, but or it takes a lot to load a page. I load find very fast, but doesnt load the images
[6:40] * plixed (~plixed@p50823A59.dip.t-dialin.net) has joined #freenet
[6:53] <ShaunMacPherson> It takes a while usually
[7:11] * _kno_ (~kvirc@212.145.186.157) Quit (Read error: 110 (Connection timed out))
[7:52] * Hory (~Miranda@81.196.25.110) Quit ("Visit http://CyberLore.net for recommendations on the best games, websites and freeware. Don't settle for anything else!")
[7:53] * yonkeltron (~yonkeltro@pcp04665066pcs.wilog501.pa.comcast.net) has joined #freenet
[9:26] * tyesinclair (~tyesincla@170.252.64.1) has joined #freenet
[9:27] * tyesinclair (~tyesincla@170.252.64.1) has left #freenet
[9:48] * guido^pe (~unknown@dsl-213-023-245-024.arcor-ip.net) has joined #freenet
[9:51] * ShaunMacPherson (~ShaunMac@Sudbury-HSE-ppp3977356.sympatico.ca) has left #Freenet
[9:53] * FallingBuzzard (~srademach@207.152.112.129) has joined #freenet
[10:42] <toad_> hi
[10:42] * toad_ back
[10:44] <toad_> anyone here?
[10:44] * mikeDOTd (~mikedotd@plutonium.nexthop.net) has joined #freenet
[10:45] <toad_> hi mikeDOTd
[10:45] <sanity> me
[10:46] <toad_> HELLO SANITY !!
[10:46] <sanity> HELLO TOAD!!!!!
[10:46] <toad_> seen my latest graphs?
[10:46] * sanity isn't feeling very well
[10:46] <toad_> well i just updated them..
[10:46] <toad_> sanity: flu?
[10:46] <sanity> dunno yet
[10:47] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-100x25.png
[10:47] <sanity> kinda feel lethargic and light headed :-/ came on quite suddenly this morning
[10:47] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-100x25-zoom.png
[10:47] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-100x25-load.png
[10:47] <toad_> all very interesting graphs
[10:47] <sanity> toad: what is the bottom line? (i mean metaphorically, not literally ;)
[10:47] <toad_> albeit from only 100x25
[10:48] <toad_> well, the estimators definitely learn at HTL 12, 11, and probably 10
[10:48] * sanity needs his observations pre-chewed today
[10:48] <sanity> ok
[10:48] <toad_> it's debatable below that
[10:48] <toad_> AND
[10:48] * d-ArkAngel (~Rob@213-131-104-86.onyx.net) has joined #freenet
[10:48] <toad_> the load ratio is increasingly favourable
[10:48] <sanity> well, higher HTLs => more requests => more samples
[10:48] <d-ArkAngel> hi all
[10:48] <sanity> d-ArkAngel: howdy
[10:48] <toad_> on all lines but especially on the top htl
[10:49] <toad_> d-ArkAngel: hi
[10:49] <toad_> it reaches at least 0.68
[10:49] <sanity> so are we thinking that NGR works provided we support estimator passing?
[10:49] <toad_> this is the ratio of #hits on the most loaded node to the least loaded node
[10:49] <toad_> sanity: well we haven't shown its scalability yet
[10:49] <toad_> but it looks very promising
[10:49] <sanity> cool
[10:50] <sanity> any lessons for Freenet IRL yet, or does this just confirm our suspicions?
[10:50] <toad_> well.. there have been a few things
[10:50] <toad_> e.g. LRU
[10:51] <toad_> the importance of estimator passing, is pretty significant imho
[10:51] * pupok (~r00t@81-178-94-146.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[10:51] <d-ArkAngel> one thing we might need to look at is how much impact "evil" nodes sending false estimator data have on the network.
[10:51] * sanity (~ian@81-178-94-146.dsl.pipex.com) Quit (Read error: 232 (Connection reset by peer))
[10:51] * pupok (~r00t@81-178-94-146.dsl.pipex.com) has joined #freenet
[10:52] <toad_> doh
[10:52] <d-ArkAngel> if the routing performance depends on the estimator passing, then that could lead to a DOS with deliberate bad estimator passing.
[10:52] * sanity (~ian@81-178-94-146.dsl.pipex.com) has joined #freenet
[10:52] <toad_> hi again :)
[10:52] <toad_> d-ArkAngel: i doubt it, we treat them with some skepticism
[10:52] <toad_> but maybe
[10:52] <toad_> <d-ArkAngel> if the routing performance depends on the estimator passing, then that could lead to a DOS with deliberate bad estimator passing.
[10:53] * lolo-laptop (~lostlogic@68.251.84.226) has joined #freenet
[10:53] <toad_> but to some extent I suppose it is a safety-in-numbers thing...
[10:53] <toad_> hi lolo-laptop
[10:53] <lolo-laptop> heya
[10:54] <d-ArkAngel> if passing estimators is just the iceing on the cake then that's fine, but if it becomes the linchpin then there could be problems.
[10:54] <toad_> d-ArkAngel: yeah...
[10:54] <toad_> sanity: comments?
[10:54] <toad_> it does rather look like the lynchpin...
[10:55] <sanity> it doesn't surprise me how important estimator passing is, the speed with which estimators start producing accurate estimates is fundamental
[10:55] <sanity> we should also consider other ways to speed up estimator learning if we can
[10:55] <toad_> sanity: fixed file sizes
[10:55] <toad_> but the point here is not just the speed of learning
[10:55] <sanity> indeed, well I support that
[10:56] <toad_> it's the balance between learning and connection churn
[10:56] <d-ArkAngel> estimator passing must help be increasing the speed at which nodes learn right? maybe if we recude the among of connection flux we can acchive the same long term performance increases, without the network growing as dependant on it's friends.
[10:56] <toad_> the rate at which data that is LOST
[10:56] <sanity> toad: they are two sides of the same coin
[10:56] <sanity> it all comes down to how smart the routing decisions are
[10:56] <toad_> well, it's not learning in terms of requests... the network without estimator passing sucks, it doesn't slowly learn, it slowly gets worse
[10:56] <d-ArkAngel> oooo, how about rather than a node telling us about thier estimates, the peer that they pass it to tells us...
[10:57] <sanity> toad: well, its a feedback loop, and feedback loops either tend to get progressively better or progressively worse
[10:58] <d-ArkAngel> we're always meeting new nodes when they answer our questions right? how about if your node is passed a node from a data source to pass back to someone you tag your estimator to it. hmmm, I don't think that would solve the problem would it... it would be fairly easy to forge such a relation....
[10:59] <sanity> perhaps using a "newbie routing" phase where newbies route on the basis of known specialisations (determined through the announcement mechanism to make chosen specialisation attacks more difficult), and only switch to estimators once they have collected sufficient samples...
[10:59] <hobx_> you can have a feedback loop that oscillates.
[10:59] <toad_> d-ArkAngel: that's what we do
[10:59] <d-ArkAngel> ahh :-)
[10:59] <sanity> hobx: yes, forgot that one
[10:59] <toad_> d-ArkAngel: it's not from the source node, it's from the node that routed to it
[10:59] <sanity> hobx: but only if its a negative feedback loop
[11:00] <toad_> sanity: iirc newbie routing sucks in simulation...
[11:00] <sanity> oh? ok
[11:00] <sanity> well that is probably good news
[11:00] <toad_> anyway it would not help sufficiently
[11:01] <toad_> the only way to get the same result as estimator passing is via drastically lowering connection churn
[11:01] <sanity> the bottom line is that whatever is going on in the real network that makes it suck, may not be happening in our simulation yet
[11:01] <toad_> and that would increase the time to establish the 1/N connection distribution
[11:01] <toad_> sanity: well there's plenty going on in the real network that is somewhat promising e.g. specialization
[11:01] <sanity> yes
[11:02] <sanity> toad: good, to be honest i haven't tried it in a while
[11:02] <sanity> is that happening in stable or unstable?
[11:02] <toad_> well, inserts REALLY suck
[11:02] <d-ArkAngel> why connection chrun is good?
[11:02] <toad_> we may need to reset stable to prove that the current fixes didn't fix inserts
[11:02] <sanity> d-ArkAngel: allows network to reorganise
[11:02] <toad_> requests are okay though
[11:02] <sanity> toad: ok
[11:02] <sanity> what is the symptom with inserts?
[11:03] <toad_> stuff related to timeouts
[11:03] <d-ArkAngel> how about each node having one set of high churn peers, and a set of low churn peers. rather then trying to change the overall rate of churn.
[11:03] <toad_> inserts taking forever and the client timing out, or inserts perpetually restarting
[11:03] <toad_> d-ArkAngel: eww...
[11:03] <sanity> ouch
[11:03] <sanity> well, that sounds like a pretty good thing to focus on
[11:03] <sanity> d-ArkAngel: establishing new connections is expensive
[11:04] * toad_ thinks we need the change i proposed w.r.t. bidirectional connections...
[11:04] <d-ArkAngel> what change is that toad?
[11:05] <toad_> d-ArkAngel: RT of size twice that of open connections
[11:05] <toad_> if we get a StoreData pointing to a node in the offline RT, and we would have referenced it otherwise, then connect to it
[11:05] <toad_> but keep the old estimators
[11:05] <toad_> basically
[11:05] <toad_> promote it to the top of both LRUs
[11:06] <toad_> so i suppose we need another hashtable and lru for each node...
[11:06] <d-ArkAngel> so basicaly the connection pool is larger than the number of active connections.
[11:07] <toad_> yes, but it's slightly more than what we do on the real network
[11:07] <toad_> d-ArkAngel: not connection pool
[11:07] <toad_> route pool
[11:07] <toad_> well
[11:07] <toad_> pool of nodes we know about
[11:07] <toad_> we only route to those we're actually connected to
[11:07] <d-ArkAngel> because you don't get direct conenctions to some nodes, we just know them as a friend of a friend
[11:08] <toad_> you could connect to them
[11:08] <toad_> but you don't route to them until you're actually connected to them
[11:08] <d-ArkAngel> ahhh right, i see.
[11:08] * toad_ wonders what oskar thinks of the estimator passing results which are pretty impressive imho :)
[11:09] <toad_> okay, i think the bigserver is still not back online
[11:09] <toad_> so i will hack on the bidi thing
[11:09] <toad_> when the bigserver is back online, can do some scaling tests
[11:10] <d-ArkAngel> how hard would it be to code up a single node to lie about the estimator passing? either pass a random one, or pass the inverse or something. to see how much impact that has on the networks ability to route?
[11:10] <d-ArkAngel> bigserver?
[11:10] <toad_> not your dp opteron
[11:10] <toad_> the other one
[11:10] <toad_> the one in russia
[11:10] <toad_> d-ArkAngel: not too hard
[11:11] <toad_> okay, what we want here... we don't even check the offline RT until we get to the point of connecting...
[11:11] <toad_> d-ArkAngel: IRL we have some mechanisms for skepticism on passed estimator data...
[11:11] <toad_> d-ArkAngel: anyway you can hack on it yourself if you like
[11:12] <toad_> e.g. if it's absurdly specialized, we reject it
[11:12] <toad_> also we only use the first 200 bits (of up to 1000) on binary running averages
[11:12] <toad_> there may be others...
[11:13] <toad_> these aren't simulated; they may not be enough
[11:13] <d-ArkAngel> with all the house moving I'm doing I'm hardly getting time to work on the parael sim and come in and worry your guys in here with inconvinient questions, let alone play with the simulations my self :-)
[11:14] * nextime (~nextime@danex.i-m-c.it) Quit ("nextime has no reason")
[11:15] <d-ArkAngel> did you get my comment about possibly using a nonSynchronised LRUQueue last night as I was signing off?
[11:16] <toad_> hmmm
[11:16] <toad_> yeah i think so
[11:16] <toad_> i doubt it'd make that much diff...
[11:16] <toad_> anyway LRUQueue uses Hashtable...
[11:16] <toad_> which is also synchronized
[11:17] <d-ArkAngel> is hashtable synchronsied? I thought it was hashtree that was synchronised
[11:17] <d-ArkAngel> just as I've been looking arround at things, I noticed that was synchronised, and if it's used a lot, then it can cause quite an impact.
[11:17] <d-ArkAngel> maybe it was hasMap that's not synced.
[11:17] <d-ArkAngel> hashmap even
[11:17] <toad_> hashtable is synchronized
[11:18] <toad_> everything else isn't
[11:18] <toad_> everything else that starts with hash
[11:18] <toad_> okay, that code was disturbingly easy.. I must have got something wrong...
[11:19] <toad_> hmm, i did, but nothing major yet...
[11:19] <d-ArkAngel> so with LRUQueue we'd be getting two synchonisation hits wouldn't we? as the locks come on the in LRUQueue object, and then again in the hastable object?
[11:19] <toad_> think so
[11:19] <toad_> but isn't uncontended locking really fast?
[11:20] <d-ArkAngel> but how many times per cycle are we hitting one of those double synced methods?
[11:20] <toad_> dunno
[11:20] <toad_> depends if we do a connection
[11:20] <toad_> if not, none
[11:20] <toad_> and conn flux isn't THAT high
[11:20] <d-ArkAngel> :-)
[11:21] <toad_> typical cycle:
[11:21] <toad_> Connections added: 26, tried 1115, force tried 0, passed estimators: 26
[11:21] <toad_> a cycle is 2,500 requests
[11:22] <d-ArkAngel> I've gotta say that I'm glad to be getting to do some java again.
[11:23] <toad_> interesting
[11:24] <d-ArkAngel> I'll stop splitting hairs and get something done :-)
[11:28] <toad_> something freenet-related?
[11:29] <toad_> hmmm
[11:32] <d-ArkAngel> well I've got a couple of work things I need to then I'll do a little work on the threaded sim, then when I get home from work I've got a little contract work I need to do for some guys in the US, then after that I'm back on threaded sims. (assuming I'm feeling up to it at what ever time of night it becomes :-) )
[11:32] <d-ArkAngel> are you finished with them two 36 hour old sims running on my server? just thinking if I'm ready for testing the threaded sim I might kill them off if you've got all you need from them?
[11:34] <d-ArkAngel> those estimator passing graphs sure are impressive
[11:41] * pittaman (~mistery_b@d51A410CA.kabel.telenet.be) has joined #freenet
[11:42] <toad_> d-ArkAngel: well you can kill them off, but kill -STOP would be an alternative
[11:42] <toad_> yeah, kill them off
[11:42] <toad_> no estimator passing
[11:43] <d-ArkAngel> well I won't do it till I need the server anyway.
[11:43] <d-ArkAngel> I wonder how many itterations it's managed to do :-)
[11:43] <toad_> probably about 10 at 800 nodes :)
[11:43] <d-ArkAngel> lol
[11:44] <d-ArkAngel> by the way, I've not been running X or any other appliations on that box, I've got my laptop setup on my desk as dual monitor, and I'm doing everything on that at the moment anyway.
[11:44] <toad_> cool
[11:44] <d-ArkAngel> so you *should* have 100% of the CPU time on there
[11:45] * toad_ waits, and waits, for eclipse...
[11:45] <d-ArkAngel> depending on how well off for money I am after I buy this new house I'm thinking of seeing how cheapely I can get hold of CPU power, either single opteron boards, or duals.
[11:46] <d-ArkAngel> well after I get moved in and settled. (you never know what money you'll need when you move in do you? :-) )
[11:47] <toad_> :|
[11:47] * toad_ stopped his local graphs...
[11:47] <toad_> well local simulations
[11:47] <toad_> compiling with bidi...
[11:48] <d-ArkAngel> as soon as I got ant all working I stoped using eclipse and just edit it using a text editor.
[11:49] <toad_> I like eclipse
[11:50] <d-ArkAngel> I dunno, it just didn't "feel" right. I guess it's cos I've never really used an IDE
[11:54] <d-ArkAngel> I've been trying to learn how to use netbeans, and am on the point of falling out with that too. I guess if they'd taught us java with a IDE back at uni the maybe I'd have picked it up a little better. ahh well.
[11:55] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-100x25-zoom.png
[11:55] * toad_ hrrrmz
[11:55] <toad_> re the purple line
[11:55] <toad_> it seems to be following red just more smoothly
[11:55] <toad_> not enough data yet of course...
[11:56] <d-ArkAngel> true.
[11:57] <toad_> hmmm
[11:57] <d-ArkAngel> and even if it's no clear benefit here, it might be of more benefit in some of the other instances where there is more room for improvment (maybe lower HTL, or more nodes)
[11:57] * toad_ decides to run it on your dp opteron...
[11:57] <d-ArkAngel> after all the line you've asked it to impove on is already running at 97% psuccess :-)
[11:58] * toad_ uploading jar...
[11:58] <toad_> well it goes down to 95.5%
[11:59] <toad_> there's a big question of which success % to target for scalability tests here
[11:59] <toad_> the blue line even works reasonably well
[11:59] <toad_> dark blue i mean
[11:59] <toad_> it rises
[11:59] <toad_> purple on the wider graph isn't so convincing...
[12:03] <d-ArkAngel> on the plus side, you're worried by graphs that have a .9+ psuccess now, rather than the fear, uncertanty and dread that you felt with the first NFC graphs down at .4
[12:03] <d-ArkAngel> ;-)
[12:04] <toad_> :)
[12:04] <toad_> well, pump the HTL high enough and you can get even pretty bad routing to produce 90%+
[12:05] <d-ArkAngel> yeah, but .4 was HTL 10, you've got .95 with HTL 9 now when it's been running a while!
[12:05] <d-ArkAngel> sorry .9
[12:05] <d-ArkAngel> not .95 :-)
[12:06] * Mazah (~Masa@dsl-aur4-p74.dial.inet.fi) has joined #freenet
[12:07] * Mazah (~Masa@dsl-aur4-p74.dial.inet.fi) has left #freenet
[12:08] <toad_> was that the same size network?
[12:09] <toad_> hmmm
[12:09] <toad_> the random element does seem significant...
[12:09] * hobx (~hobx@h196n1fls21o1077.bredband.comhem.se) Quit (burroughs.freenode.net irc.freenode.net)
[12:09] * eMBee (~mbaehr@sirius.iaeste.tuwien.ac.at) Quit (burroughs.freenode.net irc.freenode.net)
[12:10] <toad_> it may be that offlinert doesn't help much...
[12:11] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-100x25-2.png
[12:11] <toad_> it definitely helps with htl 11
[12:11] <d-ArkAngel> are you using high entropy random data in the simulator? or is it psudorandom to improver perfromance?
[12:11] <toad_> the htl 12 results seem different...
[12:11] <toad_> d-ArkAngel: currently using the high entropy data
[12:12] * Ash-Fox (Hal-9000@aaf197.neoplus.adsl.tpnet.pl) Quit (Nick collision from services.)
[12:12] * Ash-Fox (Hal-9000@aag85.neoplus.adsl.tpnet.pl) has joined #FreeNET
[12:13] <d-ArkAngel> in my code alterations for muntil threadding would it be a problem for me to change out some of the high entropy stuff for psudo randoms? like data from Math.random()
[12:15] <d-ArkAngel> the advantage to such a thing is we could always seed it from out high entropy source to start things, but print that, so if we ever want to retrace the same graph we can use the same see.
[12:16] <d-ArkAngel> maybe that'd be down the line when we've got nodes of different sizes comming and going, so we could analise why the network suddenly drops and then comes back up....
[12:16] <d-ArkAngel> I suppose we could easily change it back to using the high entropy data later on.
[12:16] <toad_> d-ArkAngel: sure, change it
[12:17] <d-ArkAngel> by the end of today I'll try and have the multi threaded sim at least working for totaly random routing. and then I can work on getting the other elements like the estimators and the rouging ported over properly after that :-)
[12:20] * yonkeltron (~yonkeltro@pcp04665066pcs.wilog501.pa.comcast.net) Quit (Read error: 110 (Connection timed out))
[12:20] <toad_> cool
[12:22] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-100x25-2.png
[12:22] <toad_> okay it's definitely less wobbly
[12:22] * eMBee (~mbaehr@sirius.iaeste.tuwien.ac.at) has joined #freenet
[12:22] <toad_> it's a bit lower on red, and significantly higher on blue
[12:22] * yonkeltron (~yonkeltro@pcp04665066pcs.wilog501.pa.comcast.net) has joined #freenet
[12:22] <toad_> hi yonkeltron
[12:22] <d-ArkAngel> ahh but is the less wobbly ness random :-)
[12:23] * hobx (~hobx@h196n1fls21o1077.bredband.comhem.se) has joined #freenet
[12:23] <d-ArkAngel> maybe running the same sim a couple of times and averageing the result.
[12:23] <toad_> hmmm no light-blue is against green, not against dark-blue
[12:23] <toad_> so it's lower than green mostly
[12:23] <toad_> and purple is mostly lower than red
[12:23] <d-ArkAngel> would smoth the lines out and make things easire to see what's going on.
[12:24] <d-ArkAngel> tho that makes for yet more CPU time :-)
[12:26] <toad_> they seem to follow each other somewhat
[12:26] <toad_> compare red with purple
[12:27] <d-ArkAngel> indeed.
[12:27] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-100x25-2.png (updated) - hmm, possibly some progress?
[12:28] <d-ArkAngel> could be.... maybe it's helping it learn that little bit faster....
[12:28] * toad_ runs a 200x25 locally
[12:28] <toad_> just one
[12:28] <toad_> at 16 htl
[12:28] <toad_> with both estimator passing and offline RT
[12:28] * bllx (~leot@host217-44-61-160.range217-44.btcentralplus.com) has joined #freenet
[12:29] <toad_> hi bllx
[12:29] <bllx> hi
[12:30] <bllx> do you know fink? you cant get freenet via fink can you
[12:30] <toad_> what's fink?
[12:30] <bllx> package manager for os X
[12:30] <bllx> nm
[12:30] <toad_> ah
[12:31] <bllx> how do you get the unstable v. of freenet?
[12:31] <mikeDOTd> fink doesn't include freenet, AFAIK
[12:31] <bllx> too experimental for fink mebbe
[12:31] <mikeDOTd> http://www.freenetproject.org/snapshots/unstable-latest.jar
[12:32] <mikeDOTd> but you'll also need the unstable seedrefs from that link too
[12:32] <bllx> just wondering if there's a chance the unstable versions might work with current os X java
[12:32] <mikeDOTd> (unstable is seperate from stable)
[12:32] <sanity> bllx: it should
[12:32] <toad_> bllx: yes, quite possibly
[12:32] <toad_> stable should work too
[12:32] <bllx> os X java atm incompatible w freenet
[12:32] <bllx> hmm maybe ill check it out
[12:33] <bllx> tx
[12:33] <sanity> bllx: http://freenetproject.org/index.php?page=tryunstable
[12:33] <bllx> but why d'you say it should work? it seems apples current java is at fault
[12:34] <sanity> bllx: well, i am not aware of any bugs which prevent freenet from running on Apple's JVM, but that doesn't imply that it does
[12:34] <bllx> well its been reported by others, its a known fact
[12:34] <sanity> apple recently upgraded their JRE anyway, so if it didn't work before it might work now
[12:34] <sanity> bllx: what was the symptom?
[12:34] <bllx> no, since then
[12:35] <sanity> you mean in the last week?
[12:35] <bllx> freenet takes up to 90% of CPU, then the comp freezes
[12:35] * tyesinclair (~tyesincla@170.252.64.1) has joined #freenet
[12:35] <sanity> hmmm
[12:35] <bllx> in last coupla weeks since apples 1.4.2_2 java update, yes
[12:35] <bllx> not just me
[12:35] <sanity> well, if that is happening then we need people to help us debug it
[12:36] <bllx> its on freenet site somewhere as reported bug, i saw it
[12:36] <sanity> ...by monitoring CPU usage, disk usage, and memory usage
[12:36] <bllx> i did all that, just the CPU was going crazy
[12:36] <sanity> no Java application should be able to crash OSX
[12:36] <bllx> worked ok tho for up to an hour before crash
[12:36] * tyesinclair (~tyesincla@170.252.64.1) has left #freenet
[12:36] <sanity> bllx: could you get a stack dump?
[12:36] <bllx> err i could try
[12:37] <bllx> i think the guy who reported it posted some info
[12:37] <bllx> stack dump in console log?
[12:39] <toad_> bllx: should be
[12:40] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-100x25-2.png
[12:40] <toad_> hmmm
[12:40] <toad_> debatable gain from offlinert
[12:40] <toad_> i suppose we need to run it longer
[12:41] <bllx> it means freezing my poor ole box again
[12:41] <mikeDOTd> bllx: what kind of mac do you have?
[12:42] <toad_> bbiab
[12:42] <bllx> 1.25 GHz G4
[12:42] <bllx> freenet's always run fine on this un and ole G3 400
[12:42] <mikeDOTd> yeah, i was going to say that should be more than enough
[12:44] <bllx> so, after I reboot after freenet freezes my box, I should check in console.log or system.log and check last entries?
[12:46] <toad_> what is console.log?
[12:46] <bllx> btw, after advice, I reduced bandwidth allowed in freenet.conf, and it seemed to stay up longer but still crashed. Other os X users tried tweaking nice for freenet but no joy either
[12:47] <mikeDOTd> . o O ( write once, run anywhere my ass )
[12:47] <bllx> err Console is an app that seems to monitor processes in real time and keep logs
[12:47] <bllx> ok here goes...
[12:47] <toad_> hmmm
[12:48] <toad_> about 30 seconds per cycle for 200 nodes...
[12:48] <bllx> yep, still using up to 90%+ CPU
[12:49] <toad_> 90%+ CPU should not cause a crash
[12:49] <toad_> unless your computer is severely underventilated
[12:49] <toad_> in which case, take it back
[12:50] <d-ArkAngel> it's not the only person I've seen talking about this kind of problem. Not sure if it was related to a mac version tho.
[12:51] <d-ArkAngel> in the end we solved thier problem in the same way I got mine to work happily on my C3, I've set it with a small enough bandwidth limit to prevent it getting that high with CPU usage.
[12:51] <toad_> well
[12:51] <toad_> permanently pegged cpu causes big problems
[12:51] <toad_> but a G4 1.25GHz is reasonably fast
[12:51] <d-ArkAngel> true
[12:52] <bllx> i tried that Ark; and yes its a common os X prob with current Apple Java
[12:52] <d-ArkAngel> is there no sun JRE that will run it?
[12:52] <bllx> d-ArkAngel: know how I get a stack dump in os X?
[12:53] <d-ArkAngel> sorry, I've no clue, I'm not an OS X'er
[12:53] <toad_> bllx: it's a common OS/X problem with other java apps?
[12:53] <bllx> not as far as I know
[12:54] <bllx> --its already been written up on freenet help page as a reported bug
[12:54] <toad_> hmmm
[12:54] <toad_> 200x25 is sloow...
[12:55] <d-ArkAngel> bllx, how much bandwidth does your node have?
[12:55] * Hory (~Miranda@81.196.25.110) has joined #FreeNet
[12:55] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-200x25.png
[12:57] <bllx> you mean what did I change it to in .conf? or my dsl pipe-which is 512/256
[12:58] <d-ArkAngel> try these in your freenet.conf file.
[12:58] <d-ArkAngel> inputBandwidthLimit=38400
[12:58] <d-ArkAngel> outputBandwidthLimit=8192
[12:59] <bllx> yeh you told me that last time: I did it and it kept my node up a little longer but still froze after a while
[12:59] <toad_> can you send me your freenet.conf?
[12:59] <d-ArkAngel> have you tried dropping those number more to see if there's a point at which it'll balance at?
[12:59] <toad_> also maybe try setting aggressiveGC=30?
[12:59] <toad_> that'll make a cpu spike every 30 seconds...
[12:59] <toad_> nah
[13:00] <toad_> don't do that
[13:00] <bllx> toad_: its not the .conf, its an os X java/freenet issue, I assure you. Many others have identical prob
[13:00] <toad_> bllx: well then there's nothing, nil, zero, nada, null, we can do about it, because it's apple's fault
[13:01] <toad_> bllx: there should be some conf change we can make to make it use less resources and ward off the crash
[13:01] <d-ArkAngel> time I went home
[13:01] <d-ArkAngel> catch you guys later
[13:01] <toad_> bllx: how are you running it?
[13:01] <bllx> i hear ya toad
[13:01] <bllx> how? err, sh start-freenet.sh
[13:02] <toad_> possible it's an issue with that...
[13:02] <bllx> ?
[13:02] <bllx> is there another way?
[13:02] <toad_> no i mean
[13:02] <toad_> the content of it
[13:02] <toad_> what command line does it use?
[13:02] <bllx> you mean what shell?
[13:02] <bllx> bash
[13:03] <toad_> no
[13:03] <toad_> what does it say when you run start-freenet.sh?
[13:04] <bllx> well i get 2 error msgs, but I was told they dont matter
[13:04] <bllx> INFO: Native CPUID library jcpuid not loaded,
[13:04] <bllx> and
[13:04] <bllx> INFO: Native BigInteger library jbigi not loaded, reason:
[13:04] <toad_> it should tell you what command line it's starting java with
[13:04] <toad_> if not it's an old version of start-freenet.sh
[13:04] <toad_> get a newer one
[13:04] <bllx> reasons both times cos it doesnt know os type Mac OS X
[13:04] <bllx> yes its an old one..i only run update.sh
[13:05] <toad_> or it doesn't know the cpu type
[13:05] <bllx> to update
[13:05] <hirvox> bllx:yet the latest freenet-ext.jar
[13:05] <hirvox> get
[13:05] <bllx> no, it doesnt know CPU type, but Im told that s not important
[13:05] * d-ArkAngel (~Rob@213-131-104-86.onyx.net) has left #freenet
[13:05] <toad_> hmmm
[13:05] <bllx> update.sh gets the latest freenet-ext.jar
[13:06] <bllx> doesnt it
[13:06] <toad_> well purple isn't necessarily a big gain on red
[13:06] <toad_> at 100 nodes
[13:06] <toad_> and blue doesn't seem much better than green...
[13:06] * sanity (~ian@81-178-94-146.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[13:06] <bllx> but maybe i need new start and stop scripts
[13:06] <bllx> getting...
[13:07] * sanity (~ian@81-178-94-146.dsl.pipex.com) has joined #freenet
[13:08] * toad_ thinks he will go to aikido tonight...
[13:08] <toad_> that's in about an hour
[13:09] <toad_> the BigServer still isn't up
[13:09] <toad_> so i just have d-arkangel's one and mine
[13:10] <toad_> the offline-RT changes don't seem to have helped enormously on 100x25: http://amphibian.dyndns.org/simulations2-epass-0.025-100x25-2.png
[13:10] <toad_> but maybe they need more time
[13:10] <toad_> or perhaps it makes a better difference on 200x25
[13:10] <toad_> and it's all rather slooow...
[13:11] <toad_> also it seems to slow down with success
[13:11] <toad_> hmmm
[13:11] <toad_> maybe i should profile...
[13:20] <bllx> toad_: have dloaded freenet-latest. Do you recommend I just replace the start and stop scripts with latest? or other freenet files too
[13:20] <bllx> maybe i should replace the lot
[13:25] <toad_> replace the lot...
[13:29] <bllx> cc
[13:34] <bllx> v weird: 2 boxes running identical os X v.; one will un-tar freenet-latest no prob, the other reports tar: A lone zero block at 47168
[13:40] <bllx> toad_: got new start n stop scripts in there
[13:40] <bllx> starting freenet reports: Command line: java -Xmx128m freenet.node.Main
[13:40] <toad_> probably won't make any difference...
[13:40] <toad_> ah ok
[13:40] <bllx> and, Sun Java 1.4.2 detected.
[13:40] <toad_> recommend edit the file change 128 to 192 or 256 (how much ram do you have?)
[13:40] <bllx> but the same msgs re not recognising CPU or OS
[13:40] <bllx> 768
[13:41] <toad_> ok
[13:41] <toad_> change it to at least 192, probably 256
[13:41] <bllx> ok, in freenet.conf ?
[13:41] <toad_> in start-freenet.sh
[13:41] <bllx> ah...nerver edited that before
[13:42] <toad_> should be 2 occurrences
[13:42] <toad_> well it's to test a theory :)
[13:42] <bllx> ok
[13:45] <bllx> afk few mins
[14:07] * guido^pe (~unknown@dsl-213-023-245-024.arcor-ip.net) Quit (Read error: 113 (No route to host))
[14:08] * guido^pe (~unknown@dsl-082-082-156-206.arcor-ip.net) has joined #freenet
[14:13] <bllx> toad_: i changed 128 to 256 in 2 lines at very end of start-freenet.sh
[14:13] <toad_> aha
[14:14] <toad_> i think i've discovered why simulations are so slow
[14:14] <toad_> yup, i have...
[14:15] <bllx> -sound right?
[14:15] <toad_> hmm, or have i?
[14:16] <toad_> i may have...
[14:17] <hobx_> somebody say my name?
[14:17] <toad_> Connections added: 47, tried 5276, force tried 0, passed estimators: 47
[14:17] <toad_> Cycle took 18430ms
[14:18] <toad_> seems to be wobbling around it...
[14:18] <toad_> hobx: not recently
[14:18] <bllx> toad_: done. Starts now with: Starting Freenet now: Command line: java -Xmx256m freenet.node.Main
[14:18] <hobx_> stupid thing colors the tabs arbitrarily
[14:18] <bllx> instead of 128
[14:22] <bllx> still hi cpu use
[14:25] <bllx> shee-eet: frozen. Faster to freeze with 256 set this time...5 mins
[14:30] <bllx> ok Ill try freenet unstable, see if it also has probs with Apples Java 1.4.2
[14:38] <toad_> it won't make any difference
[14:38] <toad_> there isn't that much difference between the two
[14:45] <bllx> running unstable now
[14:45] * Hory (~Miranda@81.196.25.110) Quit (Read error: 54 (Connection reset by peer))
[14:46] <toad_> hmmm
[14:46] <toad_> maybe logDEBUG not getting set in the copy constructor is the problem...
[14:47] <toad_> brb
[14:51] * tav is now known as tav|offline
[14:53] * tav|offline is now known as tav
[14:59] <bllx> well no freezin so far with unstable freenet on os X
[15:04] * spaetz (~spaetz@217-162-197-136.dclient.hispeed.ch) has joined #freenet
[15:04] * spaetz (~spaetz@217-162-197-136.dclient.hispeed.ch) has left #freenet
[15:05] * spaetz (~spaetz@217-162-197-136.dclient.hispeed.ch) has joined #freenet
[15:05] <toad_> hi
[15:05] <hobx_> an oldtimer pops in for a very brief visit
[15:05] <spaetz> hehe, yep
[15:05] <hobx_> oh look, he returns
[15:05] <toad_> so it would seem
[15:05] <spaetz> hehe, jsut had to rearrange my irssi windows
[15:06] <spaetz> Hi Oscar, Hi Toad
[15:06] <toad_> hmmm
[15:06] <toad_> why is freenet.node.rt.FastSlidingBucketsKeyspaceEstim
[15:06] <toad_> ator.getBucketValue
[15:06] <toad_> at the top of the simulation profile?
[15:06] <spaetz> Toad, sorry I've seen a couple of virii mails from "me" which I didn't cancel in time (I get notified)
[15:07] <toad_> it just consists of testing a final boolean, getting a value out of an array, and calling a method on it
[15:07] <toad_> spaetz: hmm
[15:07] <spaetz> Sorry that you have to do that
[15:07] <spaetz> I still think that my mutt isn't guilty of spreading virii :-)
[15:08] <toad_> i thought i'd added you to always-moderate?
[15:08] <toad_> or whatever it's called...
[15:08] <spaetz> yep, you did. But I get notified whenever a new mail is in your "moderation queue"
[15:09] <spaetz> I'm exited to see the new simulator stuff happening. Don't want to disturb you. Keep of developing :)
[15:09] * kers (~kers@h180n1fls33o873.telia.com) has joined #freenet
[15:09] <spaetz> Keep on...
[15:09] <toad_> hmmm
[15:10] * spaetz still cannot type properly
[15:10] <toad_> i think i've probably gone as far as i reasonably can with optimizing from profile, which is sad, because it's still really slow :|
[15:13] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-200x25.png
[15:13] <toad_> hmmm
[15:13] <spaetz> hobx, do you code again on Freenet or do you just hang around?
[15:13] <toad_> so is offlinert a benefit or not?
[15:13] <toad_> spaetz: he hangs around
[15:13] <toad_> spaetz: and pontificates
[15:13] <toad_> spaetz: and occasionally make useful criticism
[15:13] <toad_> such as that which started the current round of simulations
[15:13] <spaetz> hehe, old habits die hard ;)
[15:13] <toad_> a very useful criticism in that particular instance
[15:14] <spaetz> yep, I agree. Back then hobx was the only one whoe ever managed to do simulations
[15:14] <toad_> spaetz: you don't have to have a spare dual opteron around somewhere?
[15:14] <toad_> or anyone else?
[15:14] <toad_> I have shells on two
[15:14] <toad_> but one is down
[15:15] <spaetz> hehe, pick my PIII 500Mhz or my C3 1Ghz, I can dedicate them :)
[15:15] <toad_> heh
[15:15] <spaetz> Nahh, I have an Athlon1800+ and an Athlon 2400+ which you could use
[15:15] <toad_> could i use both?
[15:15] <hobx_> Pontificates. I like that.
[15:15] <toad_> i could run one simulation on each at a fair speed...
[15:15] <hobx_> Can I be the freenet pope?
[15:15] <spaetz> toad, sure
[15:16] <spaetz> One is my desktop machine, the other my girl friends laptop
[15:16] <toad_> okay, msg me shell details
[15:16] <spaetz> Both running Gentoo. You can have them
[15:16] <toad_> until somebody decides they need to actually use them :)
[15:16] <spaetz> What do you need? ssh and shell enough?
[15:16] <toad_> yeah, ssh
[15:16] <toad_> rsync would be nice
[15:16] <toad_> over ssh of course
[15:17] <spaetz> rsync should be installed, given that Gentoo rsyncs a lot
[15:17] <toad_> 100MB disk quota maybe
[15:17] <toad_> good
[15:17] <spaetz> ok, let me set up things BRB
[15:21] * toad_ starts a 400x25@22
[15:21] <toad_> first cycle took 9.5 seconds, 97% success
[15:21] <toad_> and falling... cycle time rising of course
[15:22] <toad_> uh oh
[15:22] <toad_> Success ratio on last 10K reqs: 0.9703092783505155 (288 0s, 9412 1s, 9700 total)
[15:22] <toad_> Success ratio on last 10K reqs: 0.9643654822335025 (702 0s, 18998 1s, 19700 total)
[15:22] <toad_> Success ratio on last 10K reqs: 0.93395 (1321 0s, 18679 1s, 20000 total)
[15:22] <toad_> Success ratio on last 10K reqs: 0.8883 (2234 0s, 17766 1s, 20000 total)
[15:22] <toad_> Success ratio on last 10K reqs: 0.86605 (2679 0s, 17321 1s, 20000 total)
[15:22] <toad_> Success ratio on last 10K reqs: 0.86165 (2767 0s, 17233 1s, 20000 total)
[15:22] <toad_> hmmm maybe it's on a turning point
[15:22] <toad_> it is!
[15:23] <toad_> Success ratio on last 10K reqs: 0.86185 (2763 0s, 17237 1s, 20000 total)
[15:24] <toad_> now it starts the slow and chaotic climb...
[15:24] <toad_> latest is 0.8776
[15:25] <toad_> i think i see what's happening now
[15:25] <toad_> the dip isn'
[15:25] <toad_> the lowest point isn't logarithmic
[15:25] <toad_> it's polynomial or worse
[15:25] <toad_> but then it rises, and the settling value may well be...
[15:26] <toad_> hmm, chaos comes...
[15:28] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-400x25.png
[15:35] * nextime (~nextime@ns0.nexlab.net) has joined #freenet
[15:37] <toad_> hi nextime
[15:38] <toad_> hmmm
[15:39] <toad_> 0.08 in 120,000 reqs = the full distance (approximately) in 240,000 requests... 240k req/ 400 nodes = 240,000/400 = 60,000/100 = 600 requests per node
[15:39] <toad_> not bad
[15:40] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-400x25.png
[15:40] <toad_> in another 200,000 requests or so, it should be into the comfort zone
[15:40] <toad_> i.e. 95%+
[15:41] <toad_> now lets see how the 200-node sims are doing...
[15:44] * hobbs (hobbs@hobbs.user) has joined #freenet
[15:44] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-200x25.png
[15:45] <toad_> not too badly, at htl 16 with offlinert turned off...
[15:45] <sanity> toad: you should email tech every now and then explaining what you are learning (not just pointing to the graphs, they tend to be near-impossible to interpret without context)
[15:45] <hobbs> toad_: Is there a one-sentence executive summary answer to: why shouldn't nodes consider themselves overloaded when they're completely saturating the other nodes in their RT?
[15:46] <toad_> hobbs: yes, we tried it, it sucked
[15:46] <toad_> hobbs: the more theoretical answer is that it would result in network freezedown, unless we can make sure that requests don't multiply
[15:46] <toad_> something which we should do anyway
[15:46] <hobbs> heh. Okay. Because I've been trying running a node for a little while recently, and it's _always_ in that state. :)
[15:46] <toad_> but which at present is not true...
[15:46] <toad_> hobbs: even with queueing?
[15:47] <toad_> sanity: well, one thing i've learned: the offline-routing-table idea doesn't seem to make much difference
[15:47] <hobbs> toad_: Well, I've noticed that it's not so completely useless anymore.
[15:48] <hobbs> toad_: just that on the whole I get a lot more requests in than downstream nodes are able to satisfy, as long as I'm reading it right.
[15:49] <hobbs> anyway. Just curious :)
[15:49] <toad_> this is one wierd monster we're building: http://amphibian.dyndns.org/simulations2-epass-0.025-400x25.png
[15:49] <toad_> is that going to keep going up or not? and why the different waveform?
[15:50] <toad_> hobbs: well, how many nodes are backed off in your RT?
[15:50] <toad_> and have a look at the queue success statistic
[15:50] * FallingBuzzard (~srademach@207.152.112.129) Quit (Read error: 104 (Connection reset by peer))
[15:50] <toad_> that gives a good idea whether you're receiving too many queries for your routes
[15:51] <hobbs> toad_: I've never managed to get over 30 contacted nodes; at any point, there are always between n-2 and n backed off nodes out of n.
[15:51] * sanity (~ian@81-178-94-146.dsl.pipex.com) Quit (Read error: 232 (Connection reset by peer))
[15:51] <toad_> hobbs: 30? ouch
[15:51] * sanity (~ian@81-178-94-146.dsl.pipex.com) has joined #freenet
[15:51] <hobbs> And usually it's n.
[15:51] <toad_> no wonder you have problems... are you heavily firewalled?
[15:52] <hobbs> toad_: outgoing I shouldn't be firewalled at all.
[15:52] <hobbs> toad_: Incoming I'm probably unreachable
[15:52] <toad_> hobbs: and incoming?
[15:52] <toad_> nodes that can't receive conns often have problems...
[15:53] <hobbs> toad_: is the diagnostic you mentioned requestQueueingSuccess?
[15:53] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-400x25.png
[15:53] <toad_> hmmm
[15:53] <toad_> i have a horrible feeling that it's levelling out at 0.92...
[15:54] <toad_> anyone got any reasonably serious computational horsepower i could borrow?
[15:55] <toad_> it takes too long running much more than 1 simulation at a time here...
[15:55] <toad_> any CPU over about 1.5GHz would help
[15:55] <toad_> if you can give me a shell
[15:56] <hobbs> toad_: I've got 1.8GHz 512MB, should be able to rig it so you can ssh in. But only for a couple hours.
[15:57] <spaetz> toad: Feel free to use spaetz.dyndns.org:24. It's going to run the next few hours if you need it.
[15:57] <hobbs> actually, cancel that. If I walk home you can have it for a good while.
[15:58] <toad_> spaetz: msg me password etc?
[16:05] <hobbs> toad_: maybe I spoke too soon. I reseeded it and have been running for a few hours, and while I've only got about 21 nodes to talk to right now, it finally settled down and has at least two or three non-backed-off nodes at any time ;)
[16:05] * cbreak (~cbreak@dhcp-32-094.via-eth.ch) has joined #freenet
[16:08] * eMBee (~mbaehr@sirius.iaeste.tuwien.ac.at) Quit (burroughs.freenode.net irc.freenode.net)
[16:08] * Ash-Fox (Hal-9000@aag85.neoplus.adsl.tpnet.pl) Quit (burroughs.freenode.net irc.freenode.net)
[16:10] * Ash-Fox (Hal-9000@aag85.neoplus.adsl.tpnet.pl) has joined #FreeNET
[16:13] <bllx> well freenet-unstable has been up and running over 2 hrs on os X box with apples java 1.4.2_2, something that latest stable build cant do
[16:14] <cbreak> hmm... you are lucky...
[16:14] <cbreak> Maybe the KPs relate with load, which is lower on unstable.
[16:15] <bllx> yeh probably
[16:15] <bllx> its still got too high cpu usage
[16:16] <bllx> probably averaging 25% on a 1.25GHz G4 PPC
[16:16] <cbreak> go to the environment page. lot's of opening connection threads?
[16:16] <bllx> yes lots ...if you mean the shell i started freenet in
[16:16] <cbreak> here's 25% on a 500 MHz Dual :P
[16:16] * kers (~kers@h180n1fls33o873.telia.com) Quit (Read error: 60 (Operation timed out))
[16:17] <bllx> things like Unable to find native library for fec8 for platform mac os x-ppc
[16:17] <cbreak> no
[16:17] <cbreak> thats just a warning, nothing important.
[16:17] <bllx> zorton: IOException: no peerhandler
[16:17] <bllx> at freenet.MuxConnectionHandler.<init>(MuxConnectionHandler.java:152)
[16:18] <cbreak> I mean THREADS on the Environment page
[16:18] <bllx> srry
[16:18] <bllx> <--or that one above
[16:18] <cbreak> don't know that... :/
[16:18] <bllx> o ok
[16:19] <cbreak> freenets demands on cpu power is changing, at startup it needs a lot more since it has to negotiate a lot of connections.
[16:19] <hobbs> I think that means you got a message from a node that your node doesn't recognize. But I'm guessing from context.
[16:19] <hobbs> Sort of an oddity
[16:19] <cbreak> also, if you are donwloading with fproxy, decoding fec in java uses a lot of computational resources.
[16:19] <bllx> ok
[16:20] <bllx> but this is a commoon os X java prob
[16:20] * eMBee (~mbaehr@sirius.iaeste.tuwien.ac.at) has joined #freenet
[16:20] <cbreak> javas cpu demands are a common any platform java problem.
[16:20] <bllx> now Ive got this thing i sometimes get, where the Freenet home page wont load
[16:21] <bllx> i mean the web interface
[16:21] <bllx> but freenet is ok still running
[16:22] <hobbs> blix: Are you loading other pages on freenet at the moment? Perhaps one with a million images?
[16:22] * sanity (~ian@81-178-94-146.dsl.pipex.com) Quit (Read error: 54 (Connection reset by peer))
[16:22] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-400x25.png
[16:22] <toad_> hmmm
[16:22] <toad_> not too promising; 200 nodes only need 16 HTL
[16:22] * sanity (~ian@81-178-94-146.dsl.pipex.com) has joined #freenet
[16:22] <bllx> o wait a min: thats a Firefox prob. Safari will load the web interface page
[16:23] <toad_> still, we'll see...
[16:23] <bllx> yes loadiing loads
[16:23] * kers (~kers@h180n1fls33o873.telia.com) has joined #freenet
[16:23] <hobbs> blix: then all it is is that your browser is refusing to open more than a certain number of connections to freenet at once
[16:23] <cbreak> bllx: Most browsers only connect a limited amount of times to the same server.
[16:24] <hobbs> blix: Which makes perfect sense when dealing with remote webservers, but with freenet, the server is local, and also tends to take a very long time to answer
[16:24] <hobbs> blix: So what happens is that if you're opening TFE or something, then it will use all of those connections trying to load images, and it will just queue up your request to get the front page (or whatever)
[16:25] <bllx> ok thanks a lot i get it
[16:25] <hobbs> Most browsers have an option somewhere or other to change that number.
[16:25] <bllx> so...the environment page
[16:26] <bllx> cbreak: the environment page has quite a lot of threads on it , but i dont know how many is normal
[16:27] <hobbs> toad_: I should go read so I can find out what these pretty graphs are about
[16:27] <cbreak> my node uses around 50 or around 150, depending weather it's in the mood to torture me...
[16:28] <bllx> under 'Pooled Thread Consumers'?
[16:29] <bllx> or, YThread goes up to 350, but many are missing
[16:30] <cbreak> Pooled threads in use
[16:30] <bllx> if i add up all the classes...
[16:31] <bllx> 139
[16:32] * d-ArkAngel (~robert@cpc2-midd2-5-0-cust16.midd.cable.ntl.com) has joined #freenet
[16:32] <cbreak> you are running standard mac os config? If so: Note that the default file descriptor limit for a process is 256, so you could get "to many open files" errors. Printing those to terminal uses considerable resources :)
[16:33] * d-ArkAngel (~robert@cpc2-midd2-5-0-cust16.midd.cable.ntl.com) Quit (Client Quit)
[16:34] <bllx> yes I am
[16:34] * d-ArkAngel (~robert@cpc2-midd2-5-0-cust16.midd.cable.ntl.com) has joined #freenet
[16:34] <bllx> yes Ive heard about this...
[16:34] <toad_> hmmm
[16:35] <toad_> 800x25@28:
[16:35] <toad_> Success ratio on last 10K reqs: 0.952994923857868 (926 0s, 18774 1s, 19700 total)
[16:35] <toad_> Success ratio on last 10K reqs: 0.8975 (2050 0s, 17950 1s, 20000 total)
[16:35] <toad_> Success ratio on last 10K reqs: 0.77975 (4405 0s, 15595 1s, 20000 total)
[16:35] <toad_> Success ratio on last 10K reqs: 0.70965 (5807 0s, 14193 1s, 20000 total)
[16:36] <toad_> granted x25 is a bit low for 800 nodes...
[16:36] <cbreak> you now simulate time?
[16:36] * sanity (~ian@81-178-94-146.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[16:36] <toad_> cbreak: huh?
[16:36] <cbreak> or what does that 1s mean?
[16:37] * sanity (~ian@81-178-94-146.dsl.pipex.com) has joined #freenet
[16:38] <hobbs> Considering that it says "success ratio", I'm betting on "zeroes and ones" as in successes and failures :)
[16:38] <toad_> Success ratio on last 10K reqs: 0.716 (5680 0s, 14320 1s, 20000 total)
[16:38] <toad_> yay, it's turning
[16:38] <toad_> it's got a long way to climb though..
[16:38] <d-ArkAngel> lol
[16:38] <d-ArkAngel> maybe it's just big random oscillations :-)
[16:38] <toad_> Success ratio on last 10K reqs: 0.7192 (5616 0s, 14384 1s, 20000 total)
[16:38] <bllx> os X's own activity monitor reports java as using 178 threads atm
[16:38] <toad_> okay, a theory presents
[16:39] <toad_> the node will eventually recover to a decent level, given enough request time
[16:39] <toad_> if we have a roughly log HTL
[16:39] <toad_> HOWEVER
[16:39] <cbreak> bllx: Freenet recycles threads
[16:39] <toad_> so HTL sorta scales
[16:39] <toad_> BUT
[16:39] <toad_> it will take approximately linear time to recover on startup!
[16:40] <toad_> Success ratio on last 10K reqs: 0.7157 (5686 0s, 14314 1s, 20000 total)
[16:41] <d-ArkAngel> which means slashdot = death
[16:41] <d-ArkAngel> for weeks.
[16:41] <toad_> it's far worse than that...
[16:41] <d-ArkAngel> so while routing is log, learning is not.
[16:42] <toad_> that's the suspicion
[16:42] <toad_> having said that the 200x25 graph casts some suspicion on routing being log
[16:42] <d-ArkAngel> dose the sim implement RT, or just conencted peers?
[16:42] <toad_> d-ArkAngel: hmm?
[16:43] <toad_> Success ratio on last 10K reqs: 0.71555 (5689 0s, 14311 1s, 20000 total)
[16:43] <toad_> uh oh...
[16:43] <toad_> is it settling at 0.71 ?!
[16:43] <bllx> is it easy to change the file descriptor max in os X?
[16:43] <d-ArkAngel> just wondering if adding lots of new nodes won't be quite so bad because even if all your connections get pulled off to nodes that know nothing, you'll stll have nodes in your RT that are known good for something to you, so some of the network might hold routability...
[16:43] <toad_> d-ArkAngel: hmmm
[16:43] <toad_> well duh
[16:43] <cbreak> yes.
[16:43] <toad_> best case learning time is n log n!
[16:43] <toad_> of course
[16:43] <toad_> me stupid
[16:44] <toad_> Success ratio on last 10K reqs: 0.70925 (5815 0s, 14185 1s, 20000 total)
[16:44] <hobbs> toad_: I'm hoping that ! is emphatic and not mathematical :)
[16:44] <d-ArkAngel> lol
[16:44] <toad_> :)
[16:44] <cbreak> bllx: insert "ulimit -n 512" or "ulimit -n 1024" in your startup-freenet.sh before the "nice -n 10 -- java..." line.
[16:45] <bllx> cool: thanks
[16:45] <hobbs> toad_: anyway, since I missed the beginning of this whole mess -- I get a feeling I know how to read the graphs, but what are the actual conditions of the simulation?
[16:46] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-400x25.png
[16:46] <toad_> hmmm
[16:46] <toad_> hobbs: pretty much ideal world, for non-fully-connected network
[16:46] <toad_> all nodes are initialized with random connections
[16:46] <toad_> when i tried gradually adding nodes, it didn't work very well
[16:46] <bllx> and that increases it just for freenet, not all java apps?
[16:46] <toad_> having said that, it might work better with estimator passing...
[16:47] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-400x25.png - looks like green is loosely following red
[16:47] <hobbs> toad_: so it's just set the network rolling and then graph how well it learns to route over time?
[16:47] <cbreak> bllx: that increases it for all applications started in the same shell. if you insert it in startup-freenet it only affects apps started in the script.
[16:47] <toad_> which suggests we need more than 22 HTL for 400 nodes - at least with an RT of 25...
[16:47] <hobbs> toad_: and the parameters are -- network size and RT size?
[16:48] * kers (~kers@h180n1fls33o873.telia.com) Quit (Read error: 110 (Connection timed out))
[16:48] <toad_> hobbs: there are lots of parameters...
[16:48] <toad_> but the axes are probability of success against time
[16:48] <bllx> aha: tx
[16:48] * kers (~kers@h180n1fls33o873.telia.com) has joined #freenet
[16:48] <toad_> Success ratio on last 10K reqs: 0.7217 (5566 0s, 14434 1s, 20000 total)
[16:48] <toad_> Success ratio on last 10K reqs: 0.72755 (5449 0s, 14551 1s, 20000 total)
[16:48] <hobbs> toad_: well the numeric ones in the filename there, AxB
[16:48] <toad_> hmm, it's rising...
[16:48] <toad_> yes
[16:49] <toad_> <nodes>x<rt size>@<htl>
[16:49] <toad_> Success ratio on last 10K reqs: 0.7299 (5402 0s, 14598 1s, 20000 total)
[16:49] <toad_> yay, it's rising...
[16:49] <cbreak> it looks like fluctuating.
[16:49] <toad_> hmmm
[16:49] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-400x25.png
[16:49] <toad_> i think that that is rising
[16:49] <toad_> but it may take millions of requests to reach a stage of being consistently over 95%
[16:50] <toad_> cbreak: yeah, it fluctuates a lot
[16:50] <toad_> hmmm maybe not
[16:51] <toad_> it's risen by maybe a quarter to a half of a unit in the last several hundred thousand requests
[16:51] <spaetz> help window
[16:51] <toad_> so about 8 more units
[16:52] <toad_> so hopefully it will reach the needed level in another 2 million requests or so
[16:52] <toad_> IF it doesn't slow down further
[16:52] <toad_> so that'd be maybe 3 million reqs total
[16:52] <toad_> over 400 nodes is 7,500 requests per node
[16:52] <toad_> that's not too terrible...
[16:53] <cbreak> 300 Requests per node per (node in its rt)
[16:53] <toad_> but 200 nodes seems to do it a lot faster...
[16:53] <toad_> 200 nodes at htl 16 does it in maybe 200,000 reqs total
[16:53] <toad_> so that's maybe 1,000 reqs per node
[16:54] * spaetz (~spaetz@217-162-197-136.dclient.hispeed.ch) has left #freenet
[16:54] <toad_> hmm
[16:54] * spaetz (~spaetz@217-162-197-136.dclient.hispeed.ch) has joined #freenet
[16:54] <toad_> i think i've probably got the target htls wrong
[16:55] <toad_> htl for 200 is probably a bit too high
[16:59] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-400x25.png
[16:59] * kers (~kers@h180n1fls33o873.telia.com) Quit (Read error: 104 (Connection reset by peer))
[16:59] <toad_> might be promising...
[17:00] * tav is now known as tav|offline
[17:00] * tav|offline is now known as tav
[17:00] <hobbs> toad_: it flies like a paper airplane. Stall... dip... swoop... stall
[17:01] <cbreak> what does estimator passing and offline rt do? I guess one passes estimators around and the other keeps estimators of disconnected nodes?
[17:01] <toad_> yup
[17:01] <toad_> bbiab
[17:01] <hobbs> This just in: the new freenet simulator is to consist of a gymnasium filled with paper airplanes and fans.
[17:02] <toad_> lol
[17:02] <toad_> bbl
[17:02] <cbreak> I know that game. GLIDER
[17:05] <hobbs> nice gliding paper airplanes are fun
[17:06] * hobbs (hobbs@hobbs.user) Quit ("Leaving")
[17:08] * ejhuff (~ejhuff@dsl-096.floodcity.net) Quit (Success)
[17:08] <spaetz> toad_: I'm leaving for now (bed). Feel free to continue to use the XP. Can I shutdown the Laptop? You don't seem to use it...
[17:09] <spaetz> ahh, you are using it. OK, I'll leave it on then.
[17:09] <spaetz> Good night
[17:14] * jay (~jcl@198.38.10.200) has joined #freenet
[17:16] <toad_> okay even the 800 node thingy is definitely learning
[17:16] <toad_> it's a question of whether the learning per node is anything close to log
[17:16] <toad_> if it's not, we're in trouble
[17:16] <toad_> learning per node to achieve a reasonable psuccess, that is
[17:16] <toad_> jay: hi, having fun with simulations
[17:22] * toad_ sets download-simlog on crontab so don't lose more than an hour's simulations...
[17:23] * ornge (~ornge@dialup-d2i-62-65-89-237.home.se) has joined #freenet
[17:23] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-400x25.png - it continues to rise, albeit slowly...
[17:25] <toad_> theory: if HTL is over X, then psuccess will eventually reach the target. if HTL is higher, it reaches it faster, but it will reach it anyway given a long time to learn. X is log with # nodes. time to learn in hits per node may or may not be.
[17:25] <toad_> well it's probably polylog, it's almost a "resource"...
[17:25] <toad_> Success ratio on last 10K reqs: 0.74905 (5019 0s, 14981 1s, 20000 total)
[17:25] <toad_> Success ratio on last 10K reqs: 0.76155 (4769 0s, 15231 1s, 20000 total)
[17:25] <toad_> Success ratio on last 10K reqs: 0.75855 (4829 0s, 15171 1s, 20000 total)
[17:25] <toad_> Success ratio on last 10K reqs: 0.7548 (4904 0s, 15096 1s, 20000 total)
[17:25] <toad_> that's on the 800x25...
[17:26] <toad_> which is slowly rising out of the ooze
[17:28] * krud (~unknown@pD9E5C7F9.dip.t-dialin.net) has joined #freenet
[17:28] <krud> hi
[17:29] <spaetz> toad_: is the 800x25 viewable? it's not on your server right?
[17:30] <krud> I run the current stable version, but my browser always says "document contains no data" when I try to access the node .. any ideas?
[17:30] <cbreak> browser?
[17:30] <cbreak> which one?
[17:30] <krud> cbreak: firefox, but i also tried konqueror (kde)
[17:31] <cbreak> are you trying to load a lot of concurent freenet pages?
[17:32] <krud> cbreak: i cannot access the start page at 127.0.0.1:8888
[17:32] <cbreak> and your node has started completely?
[17:32] <krud> the log say its started
[17:33] <krud> mainport and the other stuff
[17:33] <cbreak> hmm...
[17:34] <krud> i have these debug exceptions
[17:34] <krud> java.lang.Exception: debug
[17:34] <krud> at freenet.support.io.NIOInputStream.read(NIOInputStream.java:319)
[17:34] <krud> dont know what it means
[17:34] <cbreak> debug ones are the least important
[17:35] <cbreak> How long are you running freenet? It takes a few minutes to start up.
[17:36] * ejhuff (~ejhuff@dsl-096.floodcity.net) has joined #FreeNet
[17:36] <krud> oh damn, i have a typo in the ip adress
[17:36] <cbreak> try localhost :)
[17:36] <cbreak> freenet doesn't accept connections from remote hosts anyway by default...
[17:36] <cbreak> (on fproxy)
[17:37] <krud> cbreak: i configured it, but got the ip wrong
[17:37] <cbreak> use ssh port forwarding
[17:38] <krud> YEAH ... it works now :-)
[17:38] <cbreak> ssh -L 8888:127.0.0.1:8888 remote.host
[17:39] <cbreak> is more secure, less public, and more 1337 :)
[17:39] <krud> cbreak: its in my local network so i can just configure the node to allow access from localnet, no need for ssh ;-)
[17:40] <cbreak> hmm... until someone finds an open access point :) (which you probably don't have...)
[17:41] <cbreak> I use ssh to access my node from all over the world... (well, actualy just from my university, but it WOULD work elsewhere...)
[17:41] <toad_> anyone know how to tell gnuplot to give names to the axes?
[17:41] <toad_> spaetz: not yet
[17:42] <krud> cbreak: thats what i do too if i'm "outside"
[17:53] <spaetz> toad, yeah: isn't it "title="? Let me recheck
[17:59] * Sugadude (~Sugadude@the.dogs.bollo.cx) Quit (Remote closed the connection)
[18:00] <spaetz> gnuplot title: set y2label "Temp. in ?C"
[18:00] <spaetz> that would be the right hand side y axis
[18:05] <spaetz> IMHO: the graph could use some grid lines. Otherwise it is hard to see at which level it is...
[18:05] <spaetz> set grid {{no}{m}xtics} {{no}{m}ytics}
[18:06] <toad_> bbl
[18:07] <toad_> food!
[18:07] * spaetz (~spaetz@217-162-197-136.dclient.hispeed.ch) Quit ("finally sleeping")
[18:08] * pittaman (~mistery_b@d51A410CA.kabel.telenet.be) Quit (Client Quit)
[18:10] <d-ArkAngel> food sounds like a good idea
[18:11] <d-ArkAngel> one thing before I too head to the kitchen. Have you tried using offline RT without estimator passing?
[18:11] <d-ArkAngel> given they're both trying to compensate for the connection churn...
[18:11] <d-ArkAngel> just a thought.
[18:12] * d-ArkAngel has a food related thought
[18:12] <d-ArkAngel> bbl
[18:12] * krud (~unknown@pD9E5C7F9.dip.t-dialin.net) Quit ("leaving")
[18:13] <cbreak> I wonder if Estimator Passing could be used by "eve" to influence a nodes specialisation area...
[18:17] * Sugadude (random@badfish.securityminded.net) has joined #freenet
[18:24] * d-ArkAngel (~robert@cpc2-midd2-5-0-cust16.midd.cable.ntl.com) Quit (Remote closed the connection)
[18:26] * tav is now known as tav|offline
[18:43] * d-ArkAngel (~Rob@cpc2-midd2-5-0-cust16.midd.cable.ntl.com) has joined #freenet
[18:43] <d-ArkAngel> mmmm, tasty
[18:54] * jay (~jcl@198.38.10.200) Quit ()
[19:11] <i2p_iip> <gott> toad
[19:11] <d-ArkAngel> he's AFK eating
[19:13] <i2p_iip> <gott> tell him that mihi has stopped development on FIW
[19:13] <i2p_iip> <gott> because he says Freenet doesn't work
[19:14] <i2p_iip> <gott> tell him this is unacceptable
[19:14] <i2p_iip> <gott> tell him I require a currently maintained Freesite insertion tool
[19:14] <i2p_iip> <gott> tell him if I do not receive this, I will be miffed
[19:14] <d-ArkAngel> I guess you'll end up being miffed then. In the short term at least.
[19:17] <i2p_iip> <gott> I DON'T LIKE BEING MIFFED
[19:17] <d-ArkAngel> well write and maintain a freenet insertion tool ;-)
[19:18] <cbreak> the fcptools are maintained. not neough features?
[19:18] <i2p_iip> <gott> The Destroyer of Worlds does not do peasant work.
[19:18] <i2p_iip> <gott> Weren't FCPTools *NIX only ?
[19:19] <cbreak> they seem to run on any posix platform.
[19:21] <i2p_iip> <gott> This is not acceptable.
[19:21] * pupok (~r00t@81-178-94-146.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[19:21] * sanity (~ian@81-178-94-146.dsl.pipex.com) Quit (Connection reset by peer)
[19:21] <i2p_iip> <gott> The Destroyer of Worlds requires a Windows solution.
[19:22] <cbreak> hahahahaha!
[19:22] <i2p_iip> <gott> cunt
[19:22] * d-ArkAngel fears no destroyer of worlds with Willy G at the helm
[19:23] <i2p_iip> <gott> Incorrect.
[19:23] <d-ArkAngel> it's totally true. I have no fear. I might be wrong not to fear, but I don't
[19:25] <cbreak> gott, you might be successfull with trying to compile the fcptools under some abstraction layer.
[19:25] <cbreak> how's that called again?
[19:26] * sanity (~ian@81-178-94-146.dsl.pipex.com) has joined #freenet
[19:29] * hobbs (hobbs@hobbs.user) has joined #freenet
[19:30] * pupok (~r00t@81-178-94-146.dsl.pipex.com) has joined #freenet
[19:31] * ornge (~ornge@dialup-d2i-62-65-89-237.home.se) Quit ("Reconnecting")
[19:32] * ornge (~ornge@dialup-d2i-62-65-89-237.home.se) has joined #freenet
[19:37] * ornge (~ornge@dialup-d2i-62-65-89-237.home.se) Quit ("leaving")
[19:50] <hobbs> gotta love chanlogs. They let me see that since I left three hours ago there have been maybe 40 lines of conversation, and half of them were gott :)
[19:50] <i2p_iip> <gott> correct
[19:50] <hobbs> gott: what's your story?
[19:51] <d-ArkAngel> He's the destroyer of worlds, and this function apparently involves complaining ;-)
[19:52] <hobbs> d-ArkAngel: last time I walked into a channel after some absence, gott asks me out of nowhere "hobbs: what's your story?" :)
[19:52] <hobbs> anyway, hey d-ArkAngel .
[19:53] <d-ArkAngel> hey hobbs
[19:53] <hobbs> hrm.. Dark Angel. That show was good for about one season.
[19:53] <d-ArkAngel> well they stole my name :-) I've been using d-ArkAngeln got about 6 years now :-P
[19:53] <hobbs> I actually sort of figured.
[19:54] <hobbs> but it made me think
[19:54] <d-ArkAngel> and I'm a pale faced geek not a geneticaly enhanced hotty ;-)
[19:54] <i2p_iip> <gott> The Destroyer of Worlds demands obediance and sexual deviation.
[19:54] <hobbs> I've been hobbs since about 1999, and somehow or other I almost never run into anyone else using it.
[19:55] <hobbs> Even though I'm sure there are plenty of people around who have it as a last name
[19:55] <d-ArkAngel> Isn't that what Ming The Merciless demmanded? And look what happened to him!
[19:57] <hobbs> 'tis a sad day for freenet. I can't get True Names anymore.
[19:57] <i2p_iip> <gott> No, I am purely unique.
[19:57] <d-ArkAngel> True Names?
[19:57] * Hory (~Miranda@81.196.25.110) has joined #FreeNet
[19:58] <hobbs> i2p_iip: you are a beautiful and unique snowflake.
[19:58] <guido^pe> the novel. http://127.0.0.1:8888/SSK%40FaixQi7FkK-~RFGsa64m6FKreHIPAgM/truenames//
[19:58] <hobbs> d-ArkAngel: _True Names_ by Vernor Vinge. Good book, appropriate reading for freenetters.
[19:59] <d-ArkAngel> hmmm, it seems my freenet node has crashes :-(
[20:00] <hobbs> anyway... there's a shortage of genetically enhanced hotties in this world.
[20:00] <d-ArkAngel> true
[20:00] <d-ArkAngel> there will always be a shortage no matter how many there are
[20:01] <d-ArkAngel> hmmm "Java heap space java.lang.OutOfMemoryError: Java heap space"
[20:02] <d-ArkAngel> me thinks there's a memory leak somewhere
[20:04] * guido^pe (~unknown@dsl-082-082-156-206.arcor-ip.net) Quit (Remote closed the connection)
[20:15] * Hory (~Miranda@81.196.25.110) Quit ("Visit http://CyberLore.net for recommendations on the best games, websites and freeware. Don't settle for anything else!")
[20:18] * jay (jcl@ool-18bf6dac.dyn.optonline.net) has joined #freenet
[20:19] <bllx> if anyone's interested, freenet-unstable doesnt crash my os X box with java 1.4.2_2, where latest stable build does crash it
[20:19] <bllx> up 6 hours now no probs
[20:19] <cbreak> hmm... most interesting.
[20:20] <bllx> prob cos less cpu load in unstable build, someone said
[20:21] <hobbs> blix: If that's all it was, I'd think you'd run into something eventually, though, within 6 hours
[20:21] * pupok (~r00t@81-178-94-146.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[20:21] <bllx> yes, maybe
[20:21] <bllx> but stable freenet never lasted more than 2 hrs
[20:21] * sanity (~ian@81-178-94-146.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[20:21] * pupok (~r00t@81-178-94-146.dsl.pipex.com) has joined #freenet
[20:21] <jay> bllx: try using the latest build of 1.4.2 with stable
[20:22] <bllx> i did jay
[20:22] <d-ArkAngel> are you still getting the CPU not regnoised message on startup?
[20:22] <hobbs> blix: well that's what I'm saying
[20:22] <jay> _02 isn't the latest
[20:22] <bllx> yes Ark
[20:22] <jay> 1.4.2_05 is the latest
[20:22] <bllx> jay: its the latest from Apple...or do you mean the dev beta 1.5 build
[20:23] <jay> oh erm..
[20:23] <bllx> im more worried by trying 1.5 beta java than unstable freenet
[20:23] <bllx> hobbs: yes yr right i prob could get probs over time
[20:24] <jay> bllx: where did you get the jdk from.. apple's website?
[20:24] <hobbs> blix: that's also not what I'm saying, but okay :)
[20:24] <bllx> jay: i dont use the sdk
[20:24] <jay> the jre/jdk jave whatever.. it's all the same
[20:24] <bllx> but the apple dev site offfers the java beta i think
[20:25] <d-ArkAngel> 1.5 gave quite a nice performance increase in linux, it might be a good idea. (it was stable enough with 1.5b2 and 1.5RC on linux too
[20:25] <bllx> 1.4.2_2 comes direct from apple to the desktop via the Software Update app
[20:25] <cbreak> Java is installed by default on Mac OS X, and can be updated with System Update.
[20:25] <cbreak> bllx: sure it isn't 1.4.2_5 Update 2?
[20:25] <bllx> thats a 10-4 cb
[20:25] <jay> bllx: and there isn't a more recent build available for your machine?
[20:25] <bllx> cbreak: err im not sure dont think so
[20:26] <bllx> not non-beta
[20:26] <bllx> jay
[20:26] <jay> bllx: could try the latest 1.4.1.. that works as well
[20:26] <i2p_iip> <FillaMent> Wow... freenet doesn't like running with only 128MB RAM... too bad my server doesn't like freenet running with more
[20:27] <cbreak> bllx: type "java -version" in a terminal.
[20:27] * sanity (~ian@81-178-94-146.dsl.pipex.com) has joined #freenet
[20:27] <bllx> jay: yes it does but its hard to downgrade
[20:28] <jay> bllx: ok.. im still becoming familiar with the OSX specific issues
[20:29] <bllx> oh yr right cb it is 1.4.2 05
[20:29] <bllx> java version "1.4.2_05"
[20:29] <bllx> Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_05-141.3)
[20:29] <bllx> Java HotSpot(TM) Client VM (build 1.4.2-38, mixed mode)
[20:29] <bllx> (if its ok to paste)
[20:29] <jay> too late now.. but no prob
[20:29] <bllx> its just Apple's designation, 1.4.2_2
[20:29] <bllx> its their second revision released to apple boxes
[20:30] <bllx> but really its 1.4.2_05 obviously
[20:30] <cbreak> its the version of the installer :)
[20:30] <bllx> oh
[20:30] <cbreak> what you see in SW Update I mean.
[20:30] <bllx> yes right
[20:30] * lolo-laptop (~lostlogic@68.251.84.226) Quit ("Client exiting")
[20:31] <jay> gott: fcptools runs on windows.. though there are better GUI-based programs at the moment for win
[20:31] <bllx> so that java -version description above is Apple's latest non-beta release
[20:31] <bllx> in os X
[20:31] <jay> bllx: that's the latest in general
[20:32] <jay> for osx, win, lin, etc.
[20:32] <jay> bllx: what kind of crash happens with stable?
[20:33] <bllx> jay: it freezes the entire box spontaneously
[20:33] <jay> freeze then unfreeze, or final freeze?
[20:33] <bllx> you have to hold down power buttoon to turn it of, then boot
[20:33] <jay> ah ok
[20:33] <bllx> the worst kind
[20:33] <cbreak> want a panic.log?
[20:33] <bllx> havent been able to find a log with the right date yet
[20:34] <jay> cbreak: if he's not using the native bigint code it could cause sig 11's
[20:34] <bllx> ill try panic.log
[20:34] <jay> but im not sure if bigint even works on osx
[20:34] <cbreak> bllx: If it writes logs, they are in /Library/Logs/panic.log I think.
[20:34] <jay> native i mean
[20:34] <cbreak> bigint does not suport PPC now. Not in Freenet.
[20:35] <cbreak> The Error is: "Unresolved kernel trap(cpu 1): 0x300 - Data access DAR=0x0000000000000014"
[20:35] <jay> that may be it.. i saw this problem with 5096 and the old freenet-ext.jar file (w/out the new native code)
[20:35] <bllx> my most recent panic.log entry is Aug
[20:36] <jay> after i began using the new freenet-ext.jar the sig 11's ceased
[20:36] * pupok (~r00t@81-178-94-146.dsl.pipex.com) Quit (Read error: 104 (Connection reset by peer))
[20:36] * sanity (~ian@81-178-94-146.dsl.pipex.com) Quit (Connection reset by peer)
[20:36] * hobx (~hobx@h196n1fls21o1077.bredband.comhem.se) Quit (Remote closed the connection)
[20:36] <jay> bllx: but you can't do this since it doesn't exist for ppc
[20:37] <cbreak> I still think it's apples fault. No User Space App should be able to panic a system, or to freeze it.
[20:37] <bllx> im using a bran new latest freenet stable, with unstable 2 files added
[20:37] <jay> cbreak: well sig 11's aren't supposed to happen either on linux in a java app
[20:37] <bllx> when it was freezing in stable i was using latest build 5096 or whatever, but updated via update.sh, so not all files were latest
[20:38] <hobbs> anyone around running distributionServlet? Maybe I'll have better luck that way?
[20:39] <hobbs> (and willing to share, that is)
[20:39] * hobx (~hobx@h196n1fls21o1077.bredband.comhem.se) has joined #freenet
[20:40] <cbreak> unstable?
[20:40] <hobbs> sure
[20:40] <cbreak> ok, let me allow 8891 for external access...
[20:41] <toad_> hi
[20:42] <bllx> uh oh, something bads happening on the box running freenet...i think it may go down
[20:42] * sanity (~ian@81-178-94-146.dsl.pipex.com) has joined #freenet
[20:42] <bllx> major slowdown, freenet at 90% cpu
[20:42] <bllx> at least its not immed total freze like with the stable freenet
[20:43] <bllx> ok well prob enough of this...the issue is unresolved but noted
[20:43] <bllx> thanks for help
[20:43] <cbreak> http://dhcp-32-094.via-eth.ch:8891/ONfN~MRfI8Y/
[20:44] <cbreak> (I wonder if this thing works...)
[20:44] <toad_> re nicknames... i've been toad since about 1992
[20:44] <hobbs> cbreak: I'm getting it, just slowly
[20:44] <jay> bllx: the cpu behaves better with the native bigint lib.. which doesn't exist at the moment for ppc :(
[20:44] <bllx> thats a java lib?
[20:44] <toad_> <bllx> if anyone's interested, freenet-unstable doesnt crash my os X box with java 1.4.2_2, where latest stable build does crash it
[20:44] <toad_> <bllx> up 6 hours now no probs
[20:44] <toad_> that's really wierd
[20:44] <cbreak> hobbs: Yes, Output Bandwith Limiting is quite strict, to allow me using the net with freenet :)
[20:45] <jay> bllx: it's technically a static library
[20:45] <jay> C based
[20:45] <toad_> there isn't that much between them...
[20:45] <hobbs> cbreak: right, and it applies there too. I forgot. :)
[20:45] <jay> i've been jay since birth
[20:45] <jay> re nicks
[20:45] * pupok (~r00t@81-178-94-146.dsl.pipex.com) has joined #freenet
[20:45] <hobbs> cbreak: It would take 2-3 hours for me to get the zip
[20:46] <cbreak> you where born with a nick? THAT'S cool.
[20:46] <jay> heh
[20:46] <toad_> <i2p_iip> <FillaMent> Wow... freenet doesn't like running with only 128MB RAM... too bad my server doesn't like freenet running with more - sorry, we will deal with resource usage issues.. eventually :)
[20:46] <jay> i used braindead for a while but ppl just assumed the worst in chatting with me
[20:46] <i2p_iip> <gott> toad
[20:46] <hobbs> cbreak: hrm... just improved some, now it's settled down at around 2K/s
[20:46] <i2p_iip> <gott> mihi is no longer developing FIW
[20:47] <i2p_iip> <gott> because he says Freenet doesn't work
[20:47] <toad_> <cbreak> The Error is: "Unresolved kernel trap(cpu 1): 0x300 - Data access DAR=0x0000000000000014" - WITHOUT the native code? that would make it a sun bug :(
[20:47] <toad_> somebody ought to compile a library for os/x...
[20:47] <cbreak> hobbs: ok, gave it priority. If this limiter works...
[20:47] <jay> toad_: i can try it on sf's complile farm
[20:48] <hobbs> toad_: more than a java bug if it manages to lock the machine hard :)
[20:48] <bllx> well, for the record, it just it did just freeze again :( after almost 7 hrs
[20:48] <i2p_iip> <gott> toad, I demand that you fix the interpersonal problems people are having with Freenet
[20:48] * d-ArkAngel (~Rob@cpc2-midd2-5-0-cust16.midd.cable.ntl.com) Quit (Read error: 60 (Operation timed out))
[20:49] <toad_> gott: how?
[20:49] <toad_> outside my area of expertise
[20:49] <jay> send them all free pizzas and cold beer
[20:49] * toad_ downloads simlogs...
[20:50] <toad_> jay: out of our meagre funds?
[20:50] <i2p_iip> <gott> make them feel more loved and more cherished as Freenet contributors
[20:50] <bllx> toad_: you get that? although freenet-unstable lasted a lot longer on os X here than stable, it *did* crash in the end
[20:50] <toad_> also some of them are anon :)
[20:50] <i2p_iip> <gott> no matter what they do
[20:50] <toad_> bllx: :(
[20:50] <i2p_iip> <gott> example
[20:50] <toad_> bllx: maybe it's a problem of not having native code for os/x?
[20:50] <i2p_iip> <gott> stop insulting newsbyte when he helps freenet
[20:50] <bllx> even after giving it 256MB in start-freenet.sh and limiting bwidth in .conf to reduce cpu use
[20:50] <toad_> anyone can compile the relevant libs for os/x? i suppose you'd have to write the cpuid code too...
[20:50] <hobbs> cbreak: nah, no help. You want to try just extracting a seednodes and sending it my way?
[20:51] <cbreak> ok :)
[20:51] <bllx> yes toad, as jay said
[20:51] <bllx> the bigint.lib onee yeh
[20:52] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-400x25.png
[20:52] <toad_> hmmm
[20:52] <jay> gott: how does newsbyte help freenet?
[20:53] <cbreak> he manages freenethelp.org
[20:53] <jay> he doesn't help in any technical sense as far as can tell
[20:53] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-800x25.png
[20:53] <toad_> he runs freenethelp.org
[20:53] <toad_> that is useful
[20:53] <jay> that i know of
[20:53] <bllx> anyone else heard the Shatner album that just came out?
[20:54] <toad_> okay, on the first graph... is green rising or isn't it?
[20:54] <cbreak> William Shattner? As in "Kirk"
[20:54] <bllx> yep
[20:54] <toad_> seems to be a fractal pattern - cycles within cycles...
[20:54] <hobbs> toad_: on the whole, looks like it's rising slowly overall
[20:54] <bllx> called 'Has Been'; I recommend it
[20:55] <bllx> bought it from the iTunes music store ;)
[20:55] <hobbs> toad_: but it's harder to see behind the "darker" red and blue
[20:55] <toad_> hobbs: estimate for the number of requests for it to be (more or less) consistently over 95%?
[20:55] <toad_> any of them, really...
[20:55] <cbreak> toad_: You could fit a function to it, and get better interpretable results.
[20:56] <toad_> that'd be a pretty wierd function
[20:56] <toad_> blue might be settling
[20:56] <toad_> hopefully not...
[20:56] <cbreak> something like a linear least squares fit to a function with the pattern f(x)=a*x+b
[20:56] <toad_> no way is the trend linear
[20:56] <hobbs> toad_: The mean looks like it'd be there within another million, probably less. but for it to be over at line, say 90% of the time looks like it'll take a lot longer
[20:57] <hobbs> *that line
[20:57] <cbreak> you could see if the line is increasing or decreasing. Or take a more complex function...
[20:57] <bllx> the os X freenet bug is FreeBSD too: its described at
[20:57] <bllx> http://wikiserver.freenethelp.org:14741/TopTen#WikiFootnote_4
[20:57] <toad_> blue, green already cross it at points
[20:57] <toad_> I want it over 95% 90% of the time...
[20:58] <hobbs> toad_: If this is a model of fairly perfect conditions, what accounts for such large swings towards inefficiency? Any guesses yet?
[20:59] <toad_> hobbs: estimator passing, perhaps
[20:59] <toad_> has a regressive effect?
[20:59] <toad_> as well as being the only thing that makes it go up
[21:00] <toad_> the fact that we always replace estimators when returning estimators could be a cause of the wiggling
[21:00] <toad_> unfortunately that's a pretty basic safety feature, given we have estimator passing
[21:00] <toad_> i dunno
[21:00] <i2p_iip> <gott> you sicken me
[21:00] <toad_> other things maybe
[21:00] <i2p_iip> <gott> It cost him money to register freenethelp.org
[21:00] <i2p_iip> <gott> and you belittle it, jay
[21:00] <i2p_iip> <gott> you are distasteful
[21:01] <hobbs> toad_: Intuition says if you can keep the "wiggling" down the growth would show through better. But yeah, of course not at the cost of network safety.
[21:01] <toad_> http://amphibian.dyndns.org/simulations2-epass-0.025-200x25.png
[21:01] <toad_> WOAH
[21:01] <toad_> observe blue, purple
[21:01] <toad_> i wonder if that's just a fourth order cycle?
[21:02] <jay> gott: that doesn't give him the right to comment on development ideas he doesn't understand, nor even cares to understand
[21:02] <i2p_iip> <gott> jay: yes it does
[21:02] <jay> gott: i have nothing personal against him
[21:02] <i2p_iip> <gott> 'judge not lest ye be judged', right toad ?
[21:03] <jay> gott: really? IMO a thorough understanding of the research behind freenet gives one the right to comment
[21:03] <i2p_iip> <gott> In my opinion, you can go to hell
[21:04] <jay> gott: who's judging who now?
[21:04] <i2p_iip> <gott> anyway, I need an actively maintained windows freesite insertion tool
[21:04] <i2p_iip> <gott> I am not christian, therefore I don't have to follow its restrictions
[21:04] <cbreak> jay: You are lukcy he is gott and not teufel :)
[21:04] <i2p_iip> <gott> you, however, are
[21:04] <toad_> who's teufel?
[21:04] <jay> cbreak: i am not lucky that gott is anything
[21:04] <i2p_iip> <gott> FIW was a very good tool
[21:04] <i2p_iip> <gott> it no longer is
[21:05] <i2p_iip> <gott> because it is not maintained
[21:05] <i2p_iip> <gott> therefore, I need a good tool
[21:05] <i2p_iip> <gott> toad, you are more trustable
[21:05] <i2p_iip> <gott> what would you suggest I use to insert freesites ?
[21:05] <cbreak> teufel is devil, satan, mephisto, how ever you want to call that concept.
[21:05] <jay> cbreak: im aware
[21:06] <toad_> okay, here's the idea:
[21:06] <bllx> cbreak: I have a panic.log entry for the freenet induced freeze i just got in os X...any use?
[21:06] <toad_> with estimator passing, the estimator gets passed down the storedata chain towards the requestor
[21:07] <cbreak> I can't read them, maybe you could send them to apple... I would archive them anyway.
[21:07] <toad_> it gets replaced along the way if anyone has a reference to the node in question
[21:07] <bllx> roger
[21:07] <toad_> now
[21:07] <toad_> we COULD perhaps make the oscillations less troublesome by not replacing it, or only replacing it when the replacement is more experienced
[21:08] <toad_> but if we do we undermine one of the main control mechanisms limiting the damage that can be done by bogus estimators being passed
[21:08] <hobbs> toad_: isn't that a matter of trust, though?
[21:08] <hobbs> right
[21:08] <cbreak> can you average an estimator, like, merge two?
[21:08] <toad_> cbreak: not easily
[21:11] <hobbs> toad_: how about replacing it probabilistically? It's more likely to propagate further, but if a lot of nodes along the way also hold references, it's still likely to get replaced quickly. (depending on the value of p, anyway)
[21:12] <hobbs> toad_: </speculation>
[21:12] <i2p_iip> <gott> toad
[21:12] <toad_> hobbs: well, i'm not sure that helps given the attack model
[21:12] <toad_> attack model: Mallory owns two nodes, A and B.
[21:12] <toad_> request succeeds on B
[21:13] <i2p_iip> <gott> what do you suggest as a good freesite insertion tool for windows
[21:13] <toad_> so it sends a StoreData pointing to A, with exaggerated estimators, but not quite exaggerated enough to be rejected
[21:13] * jay (jcl@ool-18bf6dac.dyn.optonline.net) Quit (".")
[21:13] <toad_> there are several things to prevent this from causing serious damage:
[21:14] <toad_> 1. probability estimators take the first 200 bits per RunningAverage instead of the full 1000 bits
[21:14] <toad_> 2. estimator replacement.
[21:14] <toad_> 3. absurdly specialized estimators are rejected
[21:15] <hobbs> toad_: If it's an unknown node, it wouldn't get replaced anyway. If you do it probabilistically, then it gets more likely to be replaced as more other nodes hold refs, but it doesn't instantly get killed at the first one.
[21:16] <toad_> hobbs: a) true. b) hmmm
[21:16] <toad_> possibly yeah
[21:16] * cbreak has to go to bed...
[21:16] <toad_> on average we add 20ish connections on each cycle...
[21:16] <cbreak> bye :)
[21:16] <toad_> a cycle is 2,500 requests...
[21:16] * cbreak (~cbreak@dhcp-32-094.via-eth.ch) Quit ("leaving")
[21:16] <hobbs> I'm not making any speculations as to suitable values of p, though.
[21:16] <i2p_iip> <gott> toad
[21:16] <i2p_iip> <gott> what do you suggest as a good freesite insertion tool for windows
[21:16] <toad_> gott: no idea
[21:17] <toad_> FIW maybe
[21:17] <i2p_iip> <gott> have you not heard any of what I have been saying ?
[21:17] <toad_> which is open source; even if its maintainer abandons it you can always hack it yourself
[21:17] <i2p_iip> <gott> FIW is no longer maintained
[21:17] <toad_> <toad_> which is open source; even if its maintainer abandons it you can always hack it yourself
[21:17] <hobbs> gott: just because an app goes unmaintained doesn't make it worthless. Unless the world changes around it and breaks it
[21:17] <hobbs> gott: In which case someone will likely pick it up and fix it, because lots of people use FIW
[21:18] <hobbs> gott: And probably at least one of them is a competent coder.
[21:18] <toad_> Average hops taken to find data: 2.7603610997127617
[21:18] <toad_> Connections added: 27, tried 1113, force tried 0, passed estimators: 27
[21:18] <toad_> Routed: 0 newbie, 0 random, 5962 by estimators
[21:18] <toad_> okay so
[21:18] <toad_> 5,962/200 = 60/2 = 30
[21:19] <toad_> so we're adding just under as many conns as we are allowed
[21:19] <i2p_iip> <gott> cunt
[21:19] <toad_> one parameter we can tweak is the number of hits before a peer ceases to be newbie
[21:19] <i2p_iip> <gott> I require an actively maintained freesite insertion tool
[21:19] <toad_> right now it's at 200
[21:19] <toad_> gott: so make one, or pay for one
[21:21] <hobbs> anyway. It's the time of the season for postseason. :)
[21:21] <hobbs> And the time of the night for dinner, for that matter.
[21:21] * hobbs (hobbs@hobbs.user) Quit ("Leaving")
[21:21] <i2p_iip> <gott> no
[21:34] <toad_> bbl
[21:37] * jay (~jcl@ool-18bf6dac.dyn.optonline.net) has joined #freenet
[22:09] * sanity_ (~ian@81-178-85-135.dsl.pipex.com) has joined #freenet
[22:15] * Sugardude (~Sugadude@the.dogs.bollo.cx) Quit (Remote closed the connection)
[22:23] * plixed (~plixed@p50823A59.dip.t-dialin.net) Quit ()
[22:26] * sanity (~ian@81-178-94-146.dsl.pipex.com) Quit (Read error: 110 (Connection timed out))
[22:27] * pupok (~r00t@81-178-94-146.dsl.pipex.com) Quit (Read error: 110 (Connection timed out))
[22:28] * sanity_ (~ian@81-178-85-135.dsl.pipex.com) Quit (Read error: 110 (Connection timed out))
[22:29] * Sugardude (~Sugadude@althing.cs.dartmouth.edu) has joined #freenet
[22:32] * Sugadude (random@badfish.securityminded.net) Quit (Read error: 238 (Connection timed out))
[22:32] * Sugadude (debian-tor@166.70.93.2) has joined #freenet
[23:11] * bllx (~leot@host217-44-61-160.range217-44.btcentralplus.com) Quit ("BitchX: more nutritious than a six-pack.")
[23:29] * jay (~jcl@ool-18bf6dac.dyn.optonline.net) Quit ("Leaving")
These logs were automatically created by Jay Oliveri with his gimp hapi on irc.freenode.net.
