Jump to content
Eternal Lands Official Forums
Fedora

Some Server Changes

Recommended Posts

@Ent: due to my and Bluap's ongoing client patches, some minor server changes are needed.

 

 

ADD_ACTOR_COMMAND msg:

Roja agreed on having cast/summon animations implemented as emotes with the possibility of having different animations based on spell id. Atm this is not possible, the server does not send the corresponding actor commands.

I can link the animation to the SEND_SPECIAL_EFFECT msg, but it's not so clean and moreover not every spell/summon has a special effect.

Looking at the code, one possible way is to link animations to ADD_ACTOR_COMMAND. Since the command is one byte only, the spell id can be packed in the first 7 bits and the 8th set to 1. This way the current client will still work, adding spell animations will require only minor changes in the next_command() function and would also leave 50 or so available placeholders for new commands.

For the summon animation, I can link it to the local chat text...but if the ADD_ACTOR_COMMAND is going to be modified, why not use a specific command? Maybe reusing the "cast=10" to "summon=10" in client_serv.h

 

Or we can simply have 5 new commands: cast_summon, cast_heal, cast_defensive, cast_attack, cast_misc and play animations based on spell groups.

 

 

Object Ids

As stated in the corresponding thread, saving a snapshot of the current inv and restoring it from sto/bags requires object ids (well not, but would simplify the code a lot), while atm the client knows only image_id, position and flags (and description if requested). This would also simplify code for bots. I haven't any good idea on how this can be done nicely, maybe Bluap can add on the subject.

 

 

If you tell us how you want things done, we can finish the patches.

Ty ;)

Edited by Fedora

Share this post


Link to post
Share on other sites

The best way is to have new commands, otherwise it would be an ugly hack.

Regarding the object id, that's not a problem, but probably should be done when closer to an update.

Share this post


Link to post
Share on other sites
Regarding the object id, that's not a problem, but probably should be done when closer to an update.

May be we can chat about possible implementations. As I suggested in the thread fedora mentioned. If new commands were used, we could possibly maintain compatibility with the existing client. I may have a go at the GUI side sooner anyway, so people can have a look and give some feedback/suggestions.

Share this post


Link to post
Share on other sites

As a bot coder, I'd love to have real object ID's, even though it would mean rewriting (and testing!) part of the code.

I think this might reduce server load as well, since object ID's allow caching of descriptions (which is only possible in a limited way now)

 

May I suggest posting about it also in the bot owners forum when it gets implemented (or rather as soon as the protocol is decided), so we as bot coders can start working on the changes in time to get them ready before the feature gets live?

Share this post


Link to post
Share on other sites
The best way is to have new commands, otherwise it would be an ugly hack.

Regarding the object id, that's not a problem, but probably should be done when closer to an update.

 

Which one of the two proposed alternatives for ADD_ACTOR_COMMAND you want me to use? (or a third one?)

 

 

May be we can chat about possible implementations. As I suggested in the thread fedora mentioned. If new commands were used, we could possibly maintain compatibility with the existing client.

 

So, to start the chat I'll suggest a possible implementation :)

 

I think that object ids are not only an useful addition for GUI features or bot code, but can be used to introduce some desired but lacking game features. I'm referring to the many threads about Unique Objects Ids. Well almost unique :)

 

So, if the obj id is a 32 bit integer we can use, for example, the first 24 bits as the id number and the remaining 8 bits as flags. These flags can store the "quality" of the object to reflect the skills of the mixer (together with useful additional info as the category and what skill are they mixed with). The quality (let's say 0 to 4, 0=no quality, 1..4=low,normal,high,outstanding) can be set at the time of the object creation, i.e. based on the mixer skill or on the selling NPC or the monster drop.

 

Objects will stay in inv, trade win and bags based on their image_id and pos, so no big changes are needed in the client code. The only thing to change a bit is the storage code...but not that much.

 

Indeed I think the current storage structure can be preserved. If the server code is something like:

 

item storage[300];

 

it can be changed to:

 

item storage[300][5];

 

where storage[x][0] is the storage as it is now with 300 slots and objects stacked regardless of quality, while storage[x][1..4] stores the quantities for each quality type of that object.

 

By double clicking on a item, let's say a srs, the client could send a INSPECT_STO_OBJ(id,pos) and would get the content of storage[x][1..4] allowing the display of my oustanding,high,normal and low srs, which I can then move to inv. If I don't want to inspect, the server fills my inv with objects of the quality it wants.

 

The coding should all amount to some bit masking here and there. Once quality is implemented it can be used in many different ways, from giving bonuses to better items, to higher NPC buy price, to increased durability...

 

 

The client code would not require big changes, and I volunteer for the obj quality storage thing if it gets implemented.

I also spoke with some bot owners/coders and I think they would be happy with object ids even without backward compatibility. If so, what is required is just to send the object id together with image_id and pos in the server messages about items.

 

Last thing, when is the next update planned?

Share this post


Link to post
Share on other sites
Which one of the two proposed alternatives for ADD_ACTOR_COMMAND you want me to use? (or a third one?)

Sorry, I meant use it as it is now, and add new commands for the emotes.

 

 

May be we can chat about possible implementations. As I suggested in the thread fedora mentioned. If new commands were used, we could possibly maintain compatibility with the existing client.

 

So, if the obj id is a 32 bit integer we can use, for example, the first 24 bits as the id number and the remaining 8 bits as flags. These flags can store the "quality" of the object to reflect the skills of the mixer (together with useful additional info as the category and what skill are they mixed with). The quality (let's say 0 to 4, 0=no quality, 1..4=low,normal,high,outstanding) can be set at the time of the object creation, i.e. based on the mixer skill or on the selling NPC or the monster drop.

The server uses a short. I guess some flags can be added, maybe 2-3, but not more than that. We already have over 1K item types, and it is possible to get over to 8K in some future, although that is doubtful. However, it is always good to have room for expansion.

 

Indeed I think the current storage structure can be preserved. If the server code is something like:

 

item storage[300];

 

it can be changed to:

 

item storage[300][5];

 

where storage[x][0] is the storage as it is now with 300 slots and objects stacked regardless of quality, while storage[x][1..4] stores the quantities for each quality type of that object.

That can't happen, because you have to remember that all this data is saved in a file, so a conversion would be needed. And some pretty major changes need to be done to ALL the code regardign items, not only storage, but bags, trade, hyperspace bags, etc. Too much effort.

 

So let's stick to how things are done now, and I can modify the send item protocol to also send the true item id along with the image id. Probably for bots, only the item id, once all the bot owners switch to the new system.

 

Last thing, when is the next update planned?

I don't know, maybe January? Or possibly December, depending on what Roja wants to add to the new client.

Share this post


Link to post
Share on other sites
<snip>

So let's stick to how things are done now, and I can modify the send item protocol to also send the true item id along with the image id. Probably for bots, only the item id, once all the bot owners switch to the new system.

<snip>

 

For bots, the item ID would indeed be sufficient (at the moment, they have no use for image IDs, but who knows about the future).

 

But why 2 protocol changes? Unless I'm missing something, all bots will break on the first protocol change, so will have to switch anyway. And, as things stand now, the client will always need the item ID before the description, so why not send it asap?

 

I'm guessing here, but I'd say the possibility to cache the information for each item (true item ID + image ID + description) could reduce server traffic enough, that the extra short/int required to send the image ID straight away is more than compensated (caching item data should be possible in client as well). Also, making no difference between bots and players should simplify the server code (unless there are server side security issues that require the separate protocols, ofc).

 

One thing that would be good to know is, if true item ID's can change between server restarts (I mean ID for a given item, new items will mean new ID's ofc...). If not, data can be cached across bot restarts, further reducing server traffic.

 

Note: I could help recoding that part of the client, if you want. In that case I would like a pointer to how to submit patches :P

Share this post


Link to post
Share on other sites
For bots, the item ID would indeed be sufficient (at the moment, they have no use for image IDs, but who knows about the future).

Bots "currently" use image IDs to display the correct image on web pages.

 

But why 2 protocol changes? Unless I'm missing something, all bots will break on the first protocol change, so will have to switch anyway. And, as things stand now, the client will always need the item ID before the description, so why not send it asap?

Agreed, the bots will break. Surely we'll have a little lead time so we can change/test this before it goes live. (I know, I know, don't call you s(h)urely.)

 

I'm guessing here, but I'd say the possibility to cache the information for each item (true item ID + image ID + description) could reduce server traffic enough, that the extra short/int required to send the image ID straight away is more than compensated (caching item data should be possible in client as well). Also, making no difference between bots and players should simplify the server code (unless there are server side security issues that require the separate protocols, ofc).

Agreed

 

One thing that would be good to know is, if true item ID's can change between server restarts (I mean ID for a given item, new items will mean new ID's ofc...). If not, data can be cached across bot restarts, further reducing server traffic.

Agreed, it would be preferred if this stayed 'constant' for the life of the item.

 

 

Something I didn't notice (it's probably obvious to everyone else and that's why) is that there seems to be a need to be an additional protocol. For caching purposes there'd need to be a "GET_ITEM_NAME" which doesn't require a position, but an item id. Otherwise, there'd be no way to cache names for items not in character/bot's inventory/storage. Or, LOOK_AT_* could be changed to use item id instead of position.

 

On a side note, the client is already kind of a ram/cpu hog, will caching items make it even worse? ... should probably be considered when trying to implement this.

Share this post


Link to post
Share on other sites
For bots, the item ID would indeed be sufficient (at the moment, they have no use for image IDs, but who knows about the future).

Bots "currently" use image IDs to display the correct image on web pages.

Didn't know about that usage...

 

<snip>
One thing that would be good to know is, if true item ID's can change between server restarts (I mean ID for a given item, new items will mean new ID's ofc...). If not, data can be cached across bot restarts, further reducing server traffic.

Agreed, it would be preferred if this stayed 'constant' for the life of the item.

 

Preferred, yes, necessary, no: it's not that difficult (nor different from the current situation) to request descriptions after a bot restart, just nice if it could be avoided. It would require writing data to permanent storage (file) sometime before stopping the bot, complicating the code in that respect. (The bot needs to be able to request a description for an unknown ID anyway)

 

 

Something I didn't notice (it's probably obvious to everyone else and that's why) is that there seems to be a need to be an additional protocol. For caching purposes there'd need to be a "GET_ITEM_NAME" which doesn't require a position, but an item id. Otherwise, there'd be no way to cache names for items not in character/bot's inventory/storage. Or, LOOK_AT_* could be changed to use item id instead of position.

Neither would be really necessary, you can cache a description based on item position as well (we do it actually on IRON's bots for image ID's we know to be unique).

 

Just for clarification, I'm looking at things from a pure 'client bot' perspective, i.e. a trade bot that operates as (restricted) player character. If you have associated web pages and such, your requirements may differ (as seen for the image IDs)

 

On a side note, the client is already kind of a ram/cpu hog, will caching items make it even worse? ... should probably be considered when trying to implement this.

The client programmers should know more about this, but my guess is that a lot of memory and CPU is grabbed by the graphics part of the client (a bot we use needs about 3% of the memory of the full client). Of course, a cache requires memory, but not that much, and it should save some CPU time (replacing a network request to be decoded by a simple lookup)

Share this post


Link to post
Share on other sites
Which one of the two proposed alternatives for ADD_ACTOR_COMMAND you want me to use? (or a third one?)

Sorry, I meant use it as it is now, and add new commands for the emotes.

 

Ok, I added the following to client_serv.h:

   cast_spell = 80,
 //total of 32 commands reserved for spells. 
 //Server sends cast_spell+spell_id or cast_summon
 cast_summon = 112

 

So you need to send "cast_spell+spell_id" or cast_summon.

If 32 commands (so 11 more spells available) are not enough, just tell me how many to reserve :wub:

 

If it's ok, I can submit the patch.

From what I see, the current client should have no problem handling these new commands, so for the sake of better debugging, you can safely modify the test server.

Share this post


Link to post
Share on other sites

Here is my thought for a new protocol to enable the load/save of inventory items. I was hoping for something simple that would not require the current inventory/storage protocol to be modified. My major assumption is that the server already holds a unique id for each object in the game and that these IDs can be exchanged between the client and server. I think all we would need are three new client/server messages.

 

When the client wants to save the inventory contents it sends this message to the server:

GET_INVENTORY_OBJECT_IDS

 

The server responds with this message:

HERE_YOUR_INVENTORY_OBJECT_IDS, <number of objects> <object id>, <quantity>, <object id>, <quantity>, ...

 

The client can now save the object IDs and quantities referenced by a player chosen name.

 

There would be two ways to use the saved object lists. 1) To replace the current inventory contents. 2) To append to the current inventory contents. Both could be actioned using a single new message from the client to the server.

 

GET_OBJECTS_FROM_STORAGE, <append or replace>, <number of objects> <object id>, <quantity>, <object id>, <quantity>, ...

 

The server would do all the work to append or replace the chosen objects. If we required that the storage window already be open, the client could do some initial checks but the server would still have to do the final check. It would be nice if the position in the inventory window could be saved too but that might prove too complicated to do using this simple protocol and would not be possible using the append function anyway.

 

OK, I'm wearing my most robust flame-proof clothing......

Share this post


Link to post
Share on other sites
Here is my thought for a new protocol to enable the load/save of inventory items. I was hoping for something simple that would not require the current inventory/storage protocol to be modified. My major assumption is that the server already holds a unique id for each object in the game and that these IDs can be exchanged between the client and server. I think all we would need are three new client/server messages.

 

Imo, since we are modifying the server, we should aim for simple and efficient changes. I wouldn't worry to much about bots, they will be rewritten whatever protocol change we can decide. I'll go for a radical solution: whenever an item is sent, send the obj id together with the other data. The same for every request done by the client regarding an item.

 

This way we can cache item description, cache the storage, add user-defined storage categories (just a list of obj ids), implement the inv snapshot, equip snapshot, bag snapshot or whatever else with very few client changes. Probably we'll need only a GET_OBJ_FROM_STO_BY_ID <id><quantity>. Or just modify the current message to avoid including the position.

 

As pointed out by revi, this way we can greatly reduce server traffic and the cache (storage/description) can be flushed out and reconstructed only after a server update (by checking #stats for example) to be in synch.

 

Hope this makes sense, it's late here :P

Share this post


Link to post
Share on other sites
OK, I'm wearing my most robust flame-proof clothing......

You totally forgot about inventory slot number and I don't see any trading messages.... :P

 

 

Why don't we just the current messages as they are and decode the image IDs in the message, like: bit 1 to 12 = image id and 13-16 special modifier.

So we have 4096 image ids in 16 variations.

 

Example: CoL & damaged CoL

currently:

CoL and dmg. CoL: image Id = 323 = 0x0143

new:

CoL Id = 323 + 0*2^12 = 0x0143

dmg. CoL Id = 323 + 1*2^12 = 0x1143

 

con: we need more images for the books

pro: most bot codes should be unaffected

 

But if you think this has to many limitation why not just add an byte or two to the image IDs and build an unique ID?

 

Am I missing something?

Share this post


Link to post
Share on other sites
Which one of the two proposed alternatives for ADD_ACTOR_COMMAND you want me to use? (or a third one?)

Sorry, I meant use it as it is now, and add new commands for the emotes.

 

Ok, I added the following to client_serv.h:

   cast_spell = 80,
 //total of 32 commands reserved for spells. 
 //Server sends cast_spell+spell_id or cast_summon
 cast_summon = 112

 

So you need to send "cast_spell+spell_id" or cast_summon.

If 32 commands (so 11 more spells available) are not enough, just tell me how many to reserve :P

 

If it's ok, I can submit the patch.

From what I see, the current client should have no problem handling these new commands, so for the sake of better debugging, you can safely modify the test server.

 

That would be fine, I think you can commit it.

Share this post


Link to post
Share on other sites

Maybe we should split this into two topics, one to discuss either message?

 

@bluap: can't all of what you're suggesting be accomplished by simply adding the item ID to the current protocols as discussed? It also seems you basically just made the new protocols be the old ones with item id, so what's the difference? (unless you're thinking backwards compatability... then I guess I get it, but still seems to be a lot of duplication...) I guess it's a coin toss, add three new ones or change what we have to work the way we need/want them to.

 

@Alberich: your suggestion just makes me dizzy :P In my experience, building intelligence into IDs (image or otherwise) is probably not a good idea, for a number of reasons... What if one needs to change to a different image, which often happens, seems like this would require recoding/recompiling every time? What happens when you exceed the number of bytes based on your formula? Why should we need to do math to derive two numbers from one number when we could just get the two numbers? Why make unique image ids when they're not unique (and you can accomplish uniqueness by looking at the item id)? I'd have to say that this seems like less work in the short run, but way too many pitfalls and extra work in the future. Also, I think bluap left things out because they're already covered in existing protocols (which wouldn't require changing by his suggestion...)

 

I wouldn't worry to much about bots, they will be rewritten whatever protocol change we can decide.

I would, lol :) Someone has to rewrite/change them :P

 

I'll go for a radical solution: whenever an item is sent, send the obj id together with the other data. The same for every request done by the client regarding an item.

Why is this radical? It makes pretty good sense to me.

 

Neither would be really necessary, you can cache a description based on item position as well (we do it actually on IRON's bots for image ID's we know to be unique).

Yes, I was referring to something that wasn't in inventory or storage (hence no position ID to send when asking.) This is only for caching, of course. If I know the itemd ID (however I know it...) then why not a protocol to tell me the rest of the info based on that ID? I'm just thinking bots here, and them wanting stuff that they'ver never actually seen and I can't say I've thought this through totally, so maybe it isn't needed, as you're stating...

Probably we'll need only a GET_OBJ_FROM_STO_BY_ID <id><quantity>. Or just modify the current message to avoid including the position.

Yeah, that's what I was thinking, but maybe I've overcooked the thought?

Share this post


Link to post
Share on other sites
You totally forgot about inventory slot number and I don't see any trading messages.... :)

<snip>

Yes, we did :P

 

As for your suggestion of coding alternatives in the current image ID, I think we shouldn't couple items and images too strongly, for a number of reasons (simplicity being one, and an important one)

Also, radu said in an earlier post here that the server uses a short (16-bit?), and there would only be 2-3 bits available for flags, or at most 8 variations; that would mean a lot of extra book images (or several ID's for the same image...)

 

The simplest solution I can see would be to just add an unique item ID whenever an imageID is sent:

- the client gets the image asap, needed for the user interface (trading works with images, and descriptions are on request)

- the bots have the unique identifier they need, so don't have to request a description too often (we don't dare cache trade item descriptions atm, to much risk of errors (=scam) or too complicated to implement).

- the only changes that are absolutely needed are in the routines that decipher the messages, and that should be fast and easy to do (at most one routine for each message type, right?)

 

The 'fast and easy to do' bit is getting important if we have an end of year deadline: the client, and preferably the bots, have to stay working over a server change, so we have at best 2 months left to get things right, overhauling all the internals correctly would probably take longer.

 

I realise that this might mean that a lot of bots will not use the maximum of the possibilities offered by the new system (although I might be wrong there). If that's a problem, announce a second change when this change get announced: "image ID's will be removed from some messages in the future (so better prepare yourselves now)" :P

 

Side note (and clearly guessing):

I suppose the server works internally with unique item ids in inventory and storage management etc. Then each client request for an inventory already requires a lookup based on item ID (to get the image ID), so the routine doing the lookup could return a 32-bit value (itemID + imageID*2^16) instead of a 16-bit value, with a good chance that the compiler catches any spots where the change in return value has not been taken into account...

Share this post


Link to post
Share on other sites

I suppose how it's implemented doesn't reall matter, as long as it works. I would caution against the idea of "we better get it done fast". A wise advertising firm once said: "Pay me now, or pay me later!" I'd rather see it take a bit longer, and do it right, than rush to a quick and dirty solution, but then that's just me.

 

Whatever change is made, please just give some lead time (I know, I asked before, but it's important) so that we can at least make sure we test this before going live.

Share this post


Link to post
Share on other sites

Now that we have UIDs, here is the patch for the client to read them. It is #def'd with ITEM_UID (not included in make.conf).

 

 

I tried it and seems to work, but pls test it :)

Share this post


Link to post
Share on other sites
Now that we have UIDs, here is the patch for the client to read them. It is #def'd with ITEM_UID (not included in make.conf).

I tried it and seems to work, but pls test it :D

I get an error when I apply the patch:

patching file bags.h
patching file items.c
patching file items.h
patching file multiplayer.c
patch: **** malformed patch at line 162: @@ -732,11 +740,17 @@

Also, looks like you left in some debug in multiplayer.c, forcing the call to put_text_in_buffer().

Share this post


Link to post
Share on other sites
oops, my bad.

I'll upload the corrections as soon as I'm back home from xmas week.

It's OK. I've work around the issue. Using your patch as a basis, I've added some protocol versions detection and I'm now testing a single build that will work with and without the item_uid enabled.

 

Edit:

OK, Fedora's patch committed. Modified to detect the server message from the "#item_uid" command and switch modes as required. Item uid when not set are 65535.

Edited by bluap

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×