Maverick

I wonder if PoliX has tried setting it on and checked if it influences performance.

Doc Flabby, L.C. and PoLiX, Please restart your directory server if you notice excess bandwith usage which is likely to be caused by my directory client misfires. I don't want to cause any excess bandwith costs. Apologies for the inconvenience (sigh)

It's still quite weird though. Is it such a performance hog that it must be cached so extremely like once a hour? We used to have no problems with it in previous version, right?

That guy doesn't understand that posting videos on a video website with only sound isn't very interesting. You actually listened to that whole thing?

I fear that every time my directory client closes the connection without finishing (either because of a timeout or its process got killed by the server), the directory server keeps sending server lists (as no ACK packets are received by the server). Plus each time the directory client runs, it grabs a random port and doesn't reuse the same port so it can't finish the existing process with the server. If this prediction is what really happens, I wouldn't really know how to fix this on my end. I can test running the directory client at the same port each time but this might just break it when it suddenly starts receiving server list packets without asking for it. EDIT: Querying directory servers disabled again. The querying of the 7 directory servers took to long when 2 or 3 directory servers are down (zone population count and directory server querying takes longer then 1 minute in total). EDIT2: Maybe I'll set the directory client to always use the same local port and make a seperate program that opens this local port and just sends a 0x07 disconnection packet to the directory server after the directory client has run. If the directory client gets killed, the seperate program will close the connection and make the directory server stop sending packets (theoretically).

L.C., ds.hlrse.net seems to be down? I just checked http://bw.krslynx.com and the outbound traffic rate seems to be climbing again?

I found out what was wrong with my Directory Client. Basically, my directory client kept sending 0x01 Server List Requests to the Directory Server because there was a bug of not clearing the receive-buffer. Luckily, nothing much else was wrong in the protocol implementation. It was hard to hunt down the bug though. This the protocol I've implemented; CLIENT SERVER ------------------------------------ -------------------------------------------------- Encryption request 0x01 -------> <------ 0x02 Encryption response Server list request (reliable) 0x03 -------> Synchronization request 0x05 -------> <------ 0x03 Reliable Packet Reliable ACK 0x04 -------> <------ 0x0E Cluster <------ 0x03 Reliable Packet (contains 0x0A Massive Chunk) Reliable ACK 0x04 -------> (0x03 is received multiple times for the complete server list) Disconnect 0x07 -------> <------ 0x07 Confirm Disconnection Everything seems to work fine on CatId's, Snrrrub's and Doc Flabby's Central directory server... well .. more or less; Snrrrub's directory server seems to crash if my directory client closes the connection during the server list download. CatId's and Doc Flabby's Central directory server doesn't confirm the disconnection so my client waits for the connection to hit the Time Out and then closes the connection. Luckily this still works though :-) I'm going to restart the process of polling the directory servers every minute. Please let me know if you see any problems popping up again. I hope it doesn't as I've tested things thoroughly this time

Such a shame to see everything going to pieces

I checked the source of my directory client again with the help of Doc Flabby. He found out that a List in my application wasn't thread-safe while the program was using multiple threads to access it. I made the List thread-safe and then booted up a local Snrrrub Directory Server to see if there is something out of the ordinary. Immediately after sending the client key to the Directory Server, the server kept sending zone lists indefinately (in the log it showed as repeatedly sending a zone list to a client). Shutting down the client even made the Directory Server crash. It's obvious something was going very wrong. I researched the protocol and read Catid's documentation on the protocol after a tip from Doc Flabby. Doc Flabby also said I should use a packet sniffer like WireShark to see what was going on. I used WireShark on the Continuum client when it downloads a zone list from the directory server and found out that the Continuum client sends ACK packets on the 0x0a packets (there were more differences to the protocol the client uses but this was the biggest one). There are two possible causes to this problem; the documented subspace protocol for a directory client is missing large pieces of information about acknowledging 0x0a packets, or there are big differences to using the Subspace protocol over the Continuum protocol (something I didn't expect). On the latter possibility it would be an easy fix to set my directory client to identify itself as using the subspace protocol. Nevertheless, my directory client basically DOS'ed Snrrrub's and Doc Flabby's directory servers (I haven't tested it on Priitk's directory server yet though) - a client should never be able to do such a thing (and even if a client can, the server should stop transmitting packets once the client has stopped doing it). Lynx's directory server is currently even transmitting about 1.2 MBit/s! Anyhow, my apologies for the inconvenience caused. I will continue work and test my directory client more thorougly on Snrrrub's, Doc Flabby's and Priitk's Directory Server.

meh I'll stop all directory server polling and get someone to look at the source code. EDIT: Data sources disabled. Cacti will stop executing the directory client each minute now. EDIT2: I'll take another look at the source code. I'm sure that the directory client is properly disconnecting after getting the 0x07 disconnection packet from the directory server though.

Alright, hopefully things will go well this time.

Alphabet game?

Alright, PoLiX. Lynx, do you still have the dreamhost server IP banned on your end?

sscentral.sscuservers.net is back online I've made a few small changes to my directory client to garantuee that a directory server is only queried at a maximum of once a minute.

I can't find the problem. Unfortunately Eclipse removed the Local History so I couldn't look back to what was done on the 24th. I've checked the packets being sent and received and there is nothing out of the ordinary. The directory server returns a correct Disconnection Request (0x07) packet after it has done sending the directory list and my directory client closes the connection. Changing my directory client to identify itself as Continuum or Subspace doesn't seem to matter anything to the packets being received. Communicating with ds1.krslynx.com results in 155 bytes send, 10049 bytes received, totalling in 10204 bytes transferred (about 10 kB). (Note: tested it on my own computer at home, not on the server.) I've tried running Snrrrub's directory server at my own computer to see if the directory server would give any strange error messages but nothing was reported in the (debug) logging. EDIT: Might it be possible that this URL is requested alot and results in bandwith usage? http://forums.trenchwars.org/showthread.php?38966-Directory-Servers It uses Doc Flabby's script for querying if a directory server is online and also runs on the DreamHost server. I doubt this script would be giving any problems as it's caching the page for 30 minutes (changed it to 60 minutes now) though.

Every minute. Not really. I can send you the source code for inspection but first I want to try to debug it myself Several things may be causing this behaviour. My directory client does receive an unknown packet (0x90) upon connecting to the directory server but everything works fine. The directory client also identifies itself as using the Continuum protocol while it communicates using the known Subspace protocol (this was needed to communicate with Doc Flabby's directory server). This identification mismatch shouldn't really be a problem as no encryption is used anyway but I intend to restore this back to Subspace identification to see if it helps. I hope to find out what I changed on the 24th february. If it was this continuum / subspace identification then this might be the culprit.

I'm really puzzled by this. The only thing I can think of is my directory client not closing the connection properly somehow (although it does send disconnection packets and closes the UDP socket when it's done) which causes the directory server to keep sending packets (the zone list most likely). If this takes longer then a minute, this will result in a repeating effect as my directory client would again open up a connection to retrieve the zone list.

Lynx, do you know since when the bandwith consumption has been like this?

I'm querying all the directory servers every minute for the population statistics; (drpepper.dreamhost.com is the server it's running on) I have a basic directory client that doesn't do anything more then just send the appropiate packets (encryption request, zone list request), receives the zone directory list and saves it to a file. I seriously doubt it would use 150 GB although it may add up over time. If it causes problems, I will stop the querying process. EDIT: Quick and dirty calculations; Each time a zone list is requested and returned, it takes around 8000 bytes (that's the size of the file that's stored with the results) Every day, that would take [24*60*8000] 11520000 bytes = 10,9 MB . Every month that would be (31 days) 340 MB. That's still a substantial amount and load on the directory server (which would only be queried seldomly by normal clients) but shouldn't be a real problem (I hope). L.C., PoLiX, can you check if your directory server is also generating such bandwith consumption ASAP? I will stop all directory server querying if it does. We don't want the few directory servers that we have to go down because of my population statistics

I have seen it on the television in the Netherlands quite a few times now, it's fun and informative show to watch.

yeah, if people aren't warned beforehand, most people tend to be quite credulous / gullible.

A one sided high five doesn't exist, always need two persons for a high five Definately a wave.

Scammers are getting more and more smarter by the day

Hey L.C., I've tryed the same directoryservers to query for SSCV Dragonball Z using my own directory client; IP PORT POP SCORING VERSION TIMESTAMP NAME DESCRIPTION ============================================================================================================================================ sscentral.com 74.86.4.98 21000 4 257 134 1267777114146 SSCV Dragonball Z ss://ds1.hlrse.net Fast pace action with intense battles in a multitude of carefully constructed bases. Fight with your team to capture the seven dragonballs and defend your base to the death with your friends. ds.hlrse.net 74.86.4.98 21000 4 257 134 1267777234557 SSCV Dragonball Z ss://ds1.hlrse.net Fast pace action with intense battles in a multitude of carefully constructed bases. Fight with your team to capture the seven dragonballs and defend your base to the death with your friends. nanavati.net 74.86.4.98 21000 1 257 134 1267777328487 SSCV Dragonball Z ss://ds1.hlrse.net Fast pace action with intense battles in a multitude of carefully constructed bases. Fight with your team to capture the seven dragonballs and defend your base to the death with your friends. ds1.krslynx.com 74.86.4.98 21000 4 1 134 1267777278111 SSCV Dragonball Z ss://ds1.hlrse.net Fast pace action with intense battles in a multitude of carefully constructed bases. Fight with your team to capture the seven dragonballs and defend your base to the death with your friends. ssdir.playsubspace.com 74.86.4.98 21000 1 1 134 1267777418076 SSCV Dragonball Z ss://ds1.hlrse.net Fast pace action with intense battles in a multitude of carefully constructed bases. Fight with your team to capture the seven dragonballs and defend your base to the death with your friends. ds1.hlrse.net 74.86.4.98 21000 1 1 134 1267777461621 SSCV Dragonball Z ss://ds1.hlrse.net Fast pace action with intense battles in a multitude of carefully constructed bases. Fight with your team to capture the seven dragonballs and defend your base to the death with your friends. I see no problem with the zone's directory entries. (My directory client seems to have problem interpreting the scoring boolean though.) It's possible that the Continuum client doesn't receive the zone because the description contains "ss://ds1.hlrse.net" . The client will try to resolve the IP of the zone by resolving the DNS name after "ss://" . More information here, here and here

Yes, the directory servers that most zones have configured should be first. SSCentral.sscuservers.net is most likely the most used one, unfortunately it's down for quite some time now. This is my list; my @ssdirs = ( 'sscentral.sscuservers.net', 'sscentral.com', 'ssdir.playsubspace.com', 'nanavati.net', 'ds1.hlrse.net', 'ds1.krslynx.com', 'ds.hlrse.net', 'dirserver.ssnecentral.net' ); Samapico, to add the output of the script into the Directory Servers topic, you need to add an HTML iframe into it somehow. On the TW Forums, I've used a custom BBCode for this purpose. This is the HTML code I've used on the TW forums: <iframe src="http://directoryservers.sshq.net/" title="Online Directory Servers" width="100%" height="400px" frameborder="0"></iframe> If Invision Board allows posting HTML in a single topic while dissallowing it in the rest of the forum, you probably won't need a custom tag. PS.: I've installed the script at stats.sshq.net so I can modify it later to use the population statistics to determine if a directory server is online or not.