Bot Owners Unite

[roswell_r]

I propose Bot Owners should participate in a joint project to help reolsve this issue.

 

After finding out that any sort of work around to the buddy list limit would result in server spam or would be classed as abuse and thus would get banned I thought of some other ideas. One of which was using Proxies. This doesn't sit very well on the conscience for me and probably will cause Entropy and Devs grief and also waste their time tracking who was using them. But its not needed if we help each other out by having multiple bots to track the online status of players.

 

Supposedly the minimum interval for downloading the Online Players list from Entropys site is 5 Minutes. If we had several bots grabbing the list every 5 Minutes at different intervals and uploading it to a central site so that all bots may access it we would have a more up to date list of online players.

 

This could be acoomplished by knowing which bots will get the Online List at what time. Since we live in all different countrys the EL game time would be a nice central time to base this off.

 

The Bot would download the online player list and send it to a central server. Included in the list would be a date and time stamp of the HTML file from Entropy's site. So that you wouldn't need to send the list if it was the same date and time for some reason. Other bots could grab this list from a central server more frequently and do the same.

 

Im not sure if ive explained it correctly but if we had 2 bots doing this then we'd have a online players list 2 and a half minutes up to date, if we had 4 then we'd have a online list just over a minute up to date.

 

Is there any interest for this?

 

Edit: Forgot to mention you can use the online player list ot check what buddies are online. Its obviousl though.

Edit2: The fact that we might end up going to this much effor to get around this problem might motivate the developers to fix the problem =)

Edited July 16, 2006 by roswell_r

[ttlanhil]

or better yet, a single bot that's allowed to check more often, and sends deltas (a list of players that have logged off since and a list of players who have logged on since) to other bots that register for it?

it's still not real time, and won't be without server support, but it could get a lot closer without need for server devs to do much if other bot users used it (and would still cut down on the web requests)

[roswell_r]

or better yet, a single bot that's allowed to check more often, and sends deltas (a list of players that have logged off since and a list of players who have logged on since) to other bots that register for it?

it's still not real time, and won't be without server support, but it could get a lot closer without need for server devs to do much if other bot users used it (and would still cut down on the web requests)

 

Nice one and before you start using it you grab the online player list so that you have the current set.

[ttlanhil]

Nice one and before you start using it you grab the online player list so that you have the current set.

no, you ask the registrar bot for the complete list on start-up and register for updates. as long as the registrar is online, no other bot has need to read the online players page itself

[LabRat]

Why not just ask Entropy or Learner to do a cut down version of the web page that is purely the player names separated by a space, as that would mean much less stress on the server bandwidth:

 

<a href="http://eternal-lands.solexine.fr/~radu/view_user.php?user=xxxxxxxxx">xxxxxxxxx</a>

 

The above takes up (um I think) 91 bytes, and that is for one single player, multiply that by approximately 700 players and bots online, each serving of that page takes up 62k bytes or thereabouts.

 

Compare that with just 10 bytes (roughly) by just sending the names followed by 0x20, and you get all the information needed in just over 6k, a 90% reduction in bandwidth, and surely that is not to be sneezed at, and possibly Entropy will be amenable to flexibility in the once per 5 minutes rule.

 

Actual figures when I did this:

Complete web page: 33,324 bytes

Stripped down page: 5,071 bytes

Total reduction: 28,253 bytes, an 85% reduction.

[dns_server]

i don't think it is that much of a problem myself in the scheme of things as long as it is not too offen, it is probly easyer to just hit the page every 5-15 minutes.

 

If it is really something you would want it would not nessisarily be too hard for my bot to be the master bot in this system, it could send out a full list of online players and diff's without too much work.

 

The way i would implement something like this is first your bot would send a request.

/iknow ListOnline

 

the response would be space seperated names, it will send multiple messages if it needs to.

/botname ONLINE:dns iknow whatever

/botname ONLINE:another line of messages

 

Then the bot would be registered with the bot to recieve differences, these would be every 5-10 minutes, it your bot misses a message it will be removed from the list.

/botname ONLINEREM:someon went offline

/botname ONLINEADD:somone went online

[roswell_r]

Why not just ask Entropy or Learner to do a cut down version of the web page that is purely the player names separated by a space, as that would mean much less stress on the server bandwidth:

 

<a href="http://eternal-lands.solexine.fr/~radu/view_user.php?user=xxxxxxxxx">xxxxxxxxx</a>

 

The above takes up (um I think) 91 bytes, and that is for one single player, multiply that by approximately 700 players and bots online, each serving of that page takes up 62k bytes or thereabouts.

 

Compare that with just 10 bytes (roughly) by just sending the names followed by 0x20, and you get all the information needed in just over 6k, a 90% reduction in bandwidth, and surely that is not to be sneezed at, and possibly Entropy will be amenable to flexibility in the once per 5 minutes rule.

 

Actual figures when I did this:

Complete web page: 33,324 bytes

Stripped down page: 5,071 bytes

Total reduction: 28,253 bytes, an 85% reduction.

 

That page doesn't tell whether a user is online or not. And using the online players list would be more smaller in size.

 

Im suprised nobody is annoyed about the limit of the buddy list. How are you's coping with it or how are you's getting around it?

[LabRat]

I *was* talking about the players online page, the line I showed is an example of the information that is sent per name.

 

How do I get around the buddy list? I just delete buddies I no longer chat to on a daily basis.

[LabRat]

[roswell_r]

Sorry misread, thought you were saying (http://eternal-lands.solexine.fr/~radu/view_user.php?user=) gets player online status. But you were talking about stripping css & anchors from the online player list to reduce its size.

[LabRat]

Yeah, I figured.

[Mar(c)]

/iknow ListOnline

 

/botname ONLINE:dns iknow whatever

/botname ONLINE:another line of messages

 

/botname ONLINEREM:someon went offline

/botname ONLINEADD:somone went online

This sounds like a increase of bandwith for the EL server, I thought that that was the main problem..

 

IMO, al we need is two or three mirror servers (on different continents), who are allowed to get the Players Online page (or a players online text file like LabRat proposed), and mirror it on the bot's/cron's server (and make the other file on the fly):

www.elmirrorserver.org/online_players.htm

www.elmirrorserver.org/online_players.txt

 

All bot owners and guild site owners (and also individual players) can get the players online list from the nearest mirror server "as often as they want"...

[Learner]

/iknow ListOnline

 

/botname ONLINE:dns iknow whatever

/botname ONLINE:another line of messages

 

/botname ONLINEREM:someon went offline

/botname ONLINEADD:somone went online

This sounds like a increase of bandwith for the EL server, I thought that that was the main problem..

 

IMO, al we need is two or three mirror servers (on different continents), who are allowed to get the Players Online page (or a players online text file like LabRat proposed), and mirror it on the bot's/cron's server (and make the other file on the fly):

www.elmirrorserver.org/online_players.htm

www.elmirrorserver.org/online_players.txt

 

All bot owners and guild site owners (and also individual players) can get the players online list from the nearest mirror server "as often as they want"...

Yes, any central point that people are trying to access should NOT be accessed thru the server.

[roswell_r]

I messaged entropy about this and he said that was the idea, but hasn't had the time to do it.

 

Ive made a simple php script that strips the online player list to just a txt file names delimited by space.

 

Hopefully he will reply again =D

[LabRat]

But using a php script to strip the HTML out still means reading the original web page, total bandwidth saving would be 15% *more* all told.

 

It still needs to be serverside to be of any real benefit, as I can hardly see Entropy allowing database access to the game so other sites can cache it, and that would still entail the server doing the same work anyway:

 

BOT: Get players online page from BACKUP_SERVER

BACKUP_SERVER: connect to EL_DATABASE

BACKUP_SERVER: read all players online

BACKUP_SERVER: send players online page to BOT

 

would amount to more bandwidth (the bot is a middleman and therefore receives and sends the same information) and the same database usage as:

 

BOT: Get players online page from EL_SERVER

EL_SERVER: already connected to EL_DATABASE

EL_SERVER: read all players online

EL_SERVER: send players online page to BOT

[roswell_r]

Something MAY happen in august =D

[Placid]

Most of these suggestions to me seem to just be shifting the bandwidth problem to in-game. Thus not actually removing the issue at all...

[roswell_r]

Finding a solution for the time been would be better then nothing.

[LabRat]

Most of these suggestions to me seem to just be shifting the bandwidth problem to in-game. Thus not actually removing the issue at all...

No matter what solution could be offered, at some point the game server *must* transmit the information to the tier two backup servers, there is really no way around this unfortunately. At least my solution provided a major bandwidth reduction from the server.

 

While I think about it, there is also room for a further 25% improvement in bandwidth of the stripped down page if you use only 6 bits to send the name details (A-Z, a-z, 0-9 " " and _ take up 64 characters, requiring 6 bits to represent any of those characters) and pack the entire page, sending as a binary stream instead of text, and decompressing on the fly.

 

I could write the deconstruct/construct routines for both client and server side if required, as I am sure pretty much anyone else could too.

[WizzKidd]

I like the idea of more up to date information. I think maybe the best idea is to have someone request permission to check the page every 30 seconds. Then on a server of their own, create an easily consumable page with the info. People could then connect to that page as often as they need to get the list of online players.

 

I have a server which I would be happy to provide this service through. Should I ask Entropy if this would be acceptable?

[LabRat]

I like the idea of more up to date information. I think maybe the best idea is to have someone request permission to check the page every 30 seconds. Then on a server of their own, create an easily consumable page with the info. People could then connect to that page as often as they need to get the list of online players.

 

I have a server which I would be happy to provide this service through. Should I ask Entropy if this would be acceptable?

Many of us have servers Some of us even have servers that have more than 20% uptime (well that counts me out with all my recent system crashes).. Maybe I better make a 386 linux box :/

[Placid]

Most of these suggestions to me seem to just be shifting the bandwidth problem to in-game. Thus not actually removing the issue at all...

No matter what solution could be offered, at some point the game server *must* transmit the information to the tier two backup servers, there is really no way around this unfortunately. At least my solution provided a major bandwidth reduction from the server.

Yes, your's did, hence the reason I said most

[WizzKidd]

I like the idea of more up to date information. I think maybe the best idea is to have someone request permission to check the page every 30 seconds. Then on a server of their own, create an easily consumable page with the info. People could then connect to that page as often as they need to get the list of online players.

 

I have a server which I would be happy to provide this service through. Should I ask Entropy if this would be acceptable?

Many of us have servers Some of us even have servers that have more than 20% uptime (well that counts me out with all my recent system crashes).. Maybe I better make a 386 linux box :/

 

I guess I should have added that my bot is also poised to do this as I currenlty consume that list and could output it easily. But hey, if somebody else wants to do it.. please do, it's one less thing for me to be responsible for in game.

[ttlanhil]

Most of these suggestions to me seem to just be shifting the bandwidth problem to in-game. Thus not actually removing the issue at all...

if it's done by in-game PMs then yes, but it doesn't have to be...

I like the idea of more up to date information. I think maybe the best idea is to have someone request permission to check the page every 30 seconds. Then on a server of their own, create an easily consumable page with the info. People could then connect to that page as often as they need to get the list of online players.

page? bleh, HTML isn't for rapidly changing content. why not, instead, simply send messages over sockets? send a bit/byte for if the player goes online/offline followed by the name each time it changes. and on a new connection, the full current list as sign-ons (so bots connecting to it don't know how long someone has been on for... but that's potentially incorrect anyway since someone may log off and back on before the next check of EL's page)

granted, as long as you're scraping the EL webpage for this it'll be done in pulses, but it'd be a bandwidth freindly way to do it

(anyone capable of writing a bot that works on EL shouldn't have a problem parsing that stream... at least I'd hope not)

ed: actually, even the online/offline bit can be done away with... each name itself is a toggle. if you haven't seen that name before, then they're now online. if you have it in your list, they just went offline. it's the minimal delta, to get it any smaller would take name completion, encoding, or compression

Edited July 19, 2006 by ttlanhil

[dejan]

Why not simply give us a (XML RPC) web service for this?