Jump to content
Eternal Lands Official Forums
Entropy

Special effects

Recommended Posts

Trollson:

 

My benchmarks show it performing faster than powf, but about the same as my powf cache's general case function. If I hadn't already written that cache, I'd probably go with that exp/log method, but since it already exists, I'll probably just stick with it.

 

You're right, I probably could get more speed using the exp/log method if I cache log. It'd be easier to cache than powf, and since log grows so slowly, I could well use a log cache small enough to readily fit into the cpu cache, reducing cpu cache misses. However, it'd be a lot of effort, and it's probably not worth the time at this point.

 

Thanks for all your help! :)

Share this post


Link to post
Share on other sites

It's been a few days since I could test the client but the recent changes made have resolved the valgrind problems I was seeing during destruction of the wind effects :omg: I also tried using the --smc-check=all valgrind option and I still get loads of GL warnings, never suggested these were due to the effects code though.

 

Have you tried making clean and then rebuilding? Sometimes sources get out of sync with .o files.

Yep, I'm always doing that.

Share this post


Link to post
Share on other sites

Just come across the fish smoking fire in South Redmoon. He's a screenshot, to me, the smoke looks a little over zealous; it was quite subtle before. :D My FPS drop to almost nothing as I get closer too. The location is 218,264.

Share this post


Link to post
Share on other sites

Holy heck, that's quite some smoke there!

 

Which smoke effect was that? I need to know which one to scale down (there are five of them). Oy, I really need to get my test char mm to I can go places like that :D

 

Checking in the infinite loop fixes.

Edited by KarenRei

Share this post


Link to post
Share on other sites

* Shrinking redmoon smoke size

* Effect obstruction work underway

* (Grumble, grumble) infinite loop fix that I missed :D (thanks!)

Edited by KarenRei

Share this post


Link to post
Share on other sites

Hi, the smoke in South Redmoon looks much better.

 

Unfortunately, I think you have a very serious memory leak in I believe TeleporterEffect. In the portal room, for example, the memory usage just keeps climbing by 10s of MB every few seconds. First time this happened my machine ran out of memory and was unusable until the EL client crashed. I rebooted as who knows what else had failed (though good old Debian Linux does appear to have recovered). I then used the ulimit command to limit EL's memory so I could monitor things. I can repeatedly run out of memory by just sitting in the portal room. I have to go to bed now but I'll try and chase this up tomorrow, unless you can sort it out before then.

 

On the plus side, the teleport and fountain effects looks great by the way :D

 

edit: OK I didn't go to bed.

 

I think the problem is in TeleporterEffect::request_LOD() of effect_teleporter.cpp. When you clear out capless_cylinders array using erase, you are just deleting the pointers, not the objects. Adding a "delete capless_cylinders[0];" before the erase appears to do the trick. However, it might be better to loop deleting each object then just clear() the vector in one go. Haven't checked elsewhere for similar issues. This time I am going to bed. :)

Edited by bluap

Share this post


Link to post
Share on other sites

Latest patch:

 

* Fixed teleporter LOD-change memory leak (incl. temporary teleport effects)

* Almost ready to enable the code for object obstructions; still needs a little more work.

 

After that, I plan to hook in glowing swords, then map editor improvements to allow for clouds, fireflies, blowing objects, and setting various effect properties. Then I have a few more effects to write on request from Roja (waterfalls, different color flames, etc), and then it's just the few things that I can't implement without more info from the server (for example, spells with multiple targets). Then I'll be switching into maintenance mode. :P

Share this post


Link to post
Share on other sites

I think it would be a better idea to switch to maintanance mode before all the new stuff, and add new things only if the current ones are working perfectly.

We want the update to be in May, so that means we'll have to have another feature freeze soon.

 

The good thing is, you will have more time after this update to perfect the new stuff, and add even more things if you want.

Share this post


Link to post
Share on other sites

Sure thing.

 

When bug reports are slow in coming, I could work on a branch that starts implementing new features, and then merge it back in after the release. Whenever bug reports are coming, however, I would drop that and take care of them in the main branch right away.

Share this post


Link to post
Share on other sites

Hi Karen!

 

Sorry for stepping in that late, but I have another plea for you. The current hook-in of eye_candy in add_particle_sys() is kinda a hack. I'd suggest to get rid of this function entirely and use something like

 

void add_particle_sys (enum EffectType type, float x_pos, float y_pos, float z_pos[, unsigned int dynamic])

 

then do a switch statement and based on #ifdefs call create_particle_sys or eye candy code. The return code of add_particle_sys is ignored on all 6 calls,* so using a void function is appropriate. Of course, these 6 calls have to be adjusted too but with grep it's done in a blink. Thank you!

 

With regards

Lachesis

 

*) Strictly, they're 12 if you count #ifdef branches. Also, add_particle_sys_at_tile will need to undergo similar changes, but that's no problem either.

Edited by Lachesis

Share this post


Link to post
Share on other sites

Lachesis:

 

The problem with that suggestion is the proposal to use "enum EffectType type", which the maps don't provide. When a map calls for a special effect, we get a filename for a particle sys:

 

  char *cur_particles_pointer=(char *)&cur_particles_io;
#ifdef  ZLIB
  gzread(f, cur_particles_pointer,particles_io_size);
#else   //ZLIB
  fread(cur_particles_pointer,1,particles_io_size,f);
#endif  //ZLIB

#ifdef EL_BIG_ENDIAN
  cur_particles_io.x_pos = SwapFloat(cur_particles_io.x_pos);
  cur_particles_io.y_pos = SwapFloat(cur_particles_io.y_pos);
  cur_particles_io.z_pos = SwapFloat(cur_particles_io.z_pos);
#endif


#ifdef  NEW_FRUSTUM
  add_particle_sys (cur_particles_io.file_name, cur_particles_io.x_pos, cur_particles_io.y_pos, cur_particles_io.z_pos, 0);
#else
  add_particle_sys (cur_particles_io.file_name, cur_particles_io.x_pos, cur_particles_io.y_pos, cur_particles_io.z_pos);

 

Thus, for now at least, I have to replace calls to use .part files with the creation of eye candy effects with appropriate arguments. Unless you want to deal with the server and update every last EL map... Not to mention, special effects take arguments. And they're different depending on what kind of effect you want.

 

When I finish with obstructions and swords (and any delays due to feature freezes), as mentioned, I'll need to work on the map editor. If Ent doesn't feel like having the map format modified and all of the maps changed (which I wouldn't blame him for), I could work around that; I could put an effect type and all arguments into the filename in a way that won't interfere with old-style effects, using text encoding of binary data instead of plaintext so it won't take up much space.

 

For example, instead of:

"./particles/something.part"

 

It would be something like:

"ec://1A422F2C09838DFF"

 

Which would mean something like, for example, "Effect 0x1A" (#26), argument 1=0x422F, argument 2 = 0x2C09838D, argument 3 = 0xFF". Or whatever arguments and argument lengths effect #26 needs. The 80 chars for the filename should be more than enough space, even for effects that need their boundaries specified. Naturally, the encoding of arguments would be done by a function instead of just dumping the arguments' binary data in hex, so as to not cause problems with, for example, big vs. little endian systems.

 

For now, though, what I have to work with is maps that send filenames of .part files. Thus, for now, I have to parse those filenames into eye candy effects. You may consider it a hack, but really, what other options do I have?

Edited by KarenRei

Share this post


Link to post
Share on other sites

I know that, and it's ok to use this kind of "translation" inside map_io.c, but add_particle_sys also is called with a string literal as particle file, and I would like to have that kind cleant. The current version classifies as so-called spaghetti code, because there is no obvious reason why a call to

add_particle_sys("particles/foo.part", ...)

should do anything else than loading particles/foo.part. It's a map-format related problem, so I'd like to keep the kludge inside map_io.c. The map format rarely changes and the kludge is probably going to stay there for a long time, so I would like as little confusion to be caused in the meantime as possible. You can define the enum at map_io.h as you pleases.

 

Note that there also exists a symbol table "class" defined in symbol_table.h that you can use for an easy and fast conversion from strings to numbers (even when there's a huge number of strings with little differences) without using lots of if's. For example usage, you can check books/symbols.c and books/parser.c

 

P.S.

Alternatively to using an enumeration, it would also be possible to remove the eye candy code from add_particle sys, and keep it as it is apart from that, then #ifdef mask each call to it, switching to ec_create_* if EYE_CANDY is defined. Then, the "translation" in map_io.c would only be done if EYE_CANDY is defined, leaving the old code exactly as it is. You can still use an enum here like in books/parser.c in order to avoid the many, hard-to-read if's. This may be a cleaner method, maybe you like that one better?

Edited by Lachesis

Share this post


Link to post
Share on other sites

Ah, well that's something that I can deal with. :) I could relocate that section; I just can't eliminate it (for now, at least).

 

As for symbol table, since there's only a few cases in question here, I think a few strncmps are the wiser choice, for simplicity and maintainability. If it took more comparisons, well, this is another example of where C++ would be really nice, and why I wrote the eye candy core in C++. Want to store strings and see if any match? Just use a std::map and assign keys. To check a string, check to see if it can find a key that matches it. Just a few lines of code that you know will Just Work and is very portable, since the std:: classes are tested ad nauseum.

 

 

I'm not sure what you're talking about in your PS. The code in particles.c is already #ifdef'ed. Could you clarify?

Share this post


Link to post
Share on other sites

Yeah I know but it's really simple and the code looks very neat and readable :) And I would like it to be used by others than me as well. :) I invented it because it was a performance issue in the book parser, and there were many other approaches in the client before, like using many if's (command parsing) or hashes (key.ini parsing). But in new code, I'd like this variant to be used, because of it's readability and scalability and also in order to promote the code amongst client developers so that they don'T reinvent the wheel.

Share this post


Link to post
Share on other sites

Performance test on my system:

 

FPS: 3

FPS (poor_man=1): 20, but occasionally dropping to < 5 FPS and not recovering for a at least minute or two.

FPS (no eye candy): 20

CPU usage at 3 FPS: < 5 %

CPU usage at 20 FPS: ca. 20 %

GPU: Nvidia Geforce FX 5200 (AGP 8x, 128 MB memory)

CPU: AMD Athlon XP 1800

RAM: 768 MB DDR 266

 

#glinfo:

[21:19:05] Video card: GeForce FX 5200/AGP/SSE/3DNOW!
[21:19:05] Vendor ID: NVIDIA Corporation
[21:19:05] OpenGL Version: 2.0.2 NVIDIA 87.76
[21:19:05] Supported extensions: GL_ARB_depth_texture GL_ARB_fragment_program GL_ARB_fragment_program_shadow GL_ARB_fragment_shader GL_ARB_half_float_pixel GL_ARB_imaging GL_ARB_multisample GL_ARB_multitexture GL_ARB_occlusion_query GL_ARB_pixel_buffer_object GL_ARB_point_parameters GL_ARB_point_sprite GL_ARB_shadow GL_ARB_shader_objects GL_ARB_shading_language_100 GL_ARB_texture_border_clamp GL_ARB_texture_compression GL_ARB_texture_cube_map GL_ARB_texture_env_add GL_ARB_texture_env_combine GL_ARB_texture_env_dot3 GL_ARB_texture_mirrored_repeat GL_ARB_texture_rectangle GL_ARB_transpose_matrix GL_ARB_vertex_buffer_object GL_ARB_vertex_program GL_ARB_vertex_shader GL_ARB_window_pos GL_S3_s3tc GL_EXT_texture_env_add GL_EXT_abgr GL_EXT_bgra GL_EXT_blend_color GL_EXT_blend_func_separate GL_EXT_blend_minmax GL_EXT_blend_subtract GL_EXT_compiled_vertex_array GL_EXT_Cg_shader GL_EXT_draw_range_elements GL_EXT_fog_coord GL_EXT_framebuffer_object GL_EXT_multi_draw_arrays GL_EXT_packed_depth_stencil GL_EXT_packed_pixels GL_EXT_paletted_texture GL_EXT_pixel_buffer_object GL_EXT_point_parameters GL_EXT_rescale_normal GL_EXT_secondary_color GL_EXT_separate_specular_color GL_EXT_shadow_funcs GL_EXT_shared_texture_palette GL_EXT_stencil_two_side GL_EXT_stencil_wrap GL_EXT_texture3D GL_EXT_texture_compression_s3tc GL_EXT_texture_cube_map GL_EXT_texture_edge_clamp GL_EXT_texture_env_combine GL_EXT_texture_env_dot3 GL_EXT_texture_filter_anisotropic GL_EXT_texture_lod GL_EXT_texture_lod_bias GL_EXT_texture_object GL_EXT_texture_sRGB GL_EXT_timer_query GL_EXT_vertex_array GL_HP_occlusion_test GL_IBM_rasterpos_clip GL_IBM_texture_mirrored_repeat GL_KTX_buffer_region GL_NV_blend_square GL_NV_copy_depth_to_color GL_NV_depth_clamp GL_NV_fence GL_NV_float_buffer GL_NV_fog_distance GL_NV_fragment_program GL_NV_fragment_program_option GL_NV_gpu_program_parameters GL_NV_half_float GL_NV_light_max_exponent GL_NV_multisample_filter_hint GL_NV_occlusion_query GL_NV_packed_depth_stencil GL_NV_pixel_data_range GL_NV_point_sprite GL_NV_primitive_restart GL_NV_register_combiners GL_NV_register_combiners2 GL_NV_texgen_reflection GL_NV_texture_compression_vtc GL_NV_texture_env_combine4 GL_NV_texture_expand_normal GL_NV_texture_rectangle GL_NV_texture_shader GL_NV_texture_shader2 GL_NV_texture_shader3 GL_NV_vertex_array_range GL_NV_vertex_array_range2 GL_NV_vertex_program GL_NV_vertex_program1_1 GL_NV_vertex_program2 GL_NV_vertex_program2_option GL_SGIS_generate_mipmap GL_SGIS_texture_lod GL_SGIX_depth_texture GL_SGIX_shadow GL_SUN_slice

 

make.conf (comments stripped):

UPDATE_CONF=no
FEATURES= \
	PNG_SCREENSHOT \
	NEW_TEX \
	OPTIONS_I18N \
	NEW_ACTOR_ANIMATION \
	AUTO_UPDATE COUNTERS \
	FONTS_FIX \
	COMMENT_BUFFER \
	AFK_FIX CUSTOM_LOOK \
	CUSTOM_UPDATE \
	SIMPLE_LOD SFX \
	USE_ACTOR_DEFAULTS \
	NOTEPAD \
	USE_INLINE \
	EYE_CANDY
PLATFORM=-march=athlon-xp -mfpmath=sse -O2 -ggdb -pg
XDIR=-L/usr/lib
CWARN=-Wall -Wno-sign-compare -Wno-pointer-sign
CXXWARN=-Wall -Wno-sign-compare
EXTRA_LIBS=-lalut
EXTRA_STATICLIBS=libs/libalut.a

Edited by Lachesis

Share this post


Link to post
Share on other sites

Hmm.

 

Grepping through the EL source code, it looks like nothing uses symbol table.

 

Looking at symbol table source code, it's uncommented, and I can't make any sense of it without studying it.

 

Looking at the symbol table h, the functions are described, but terms used in the description are never defined and no sample usage is given. For example, "the pointer to associate with the symbol" makes no sense without knowing what is implied by symbol, what you expect in a pointer, and what you mean by "associate". What's a committed entry? What is the "shadow array"? And most importantly, why would I want to take the time to learn all this?

 

I don't think that is an approach I would want to use. If I really had a large number of strings to compare (which I don't), I'd just open up eye_candy_wrapper.h and create a simple C wrapper around std::map: code that's well documented with tons of usage examples on the net. And I'd have two to four functions max. If I just wanted a list or two, I'd define them in the wrapper (something like "std::map<std::string, bool> our_map;") so that I don't need to worry about allocating or freeing pointers, and each only have two functions: void add_our_key(char* arg) and int check_our_key(char* arg). The entire length of add_our_key would be "our_map[std::string(arg)] = true;" The entire length of check_our_key would be "return our_map.has_key(std::string(arg));"

 

std::map is a binary search tree, so it guarantees searches in O(log N) time. I hate reinventing the wheel. :)

Share this post


Link to post
Share on other sites

C++ was not allowed in client when I was writing that code :)

Ok, the docs assume you know what a symbol table is. It's basicly something like std::map<const char *, PVoidOrInt>, where PVoidOrInt is a union of void * and int. However it's way faster when many strings start with the same characters and when many strings are inserted at once. The former is attributed to the fact that it uses multi key search instead of binary search, guaranteeing O(log N + L) time where N is the number of items and L is the length of the searched string, whereas std::map only guarantees O(L*log N) time. The second is attributed to the fact that it's optimized for inserting a large number of strings at first, and looking them up later, so instead of using a dynamical data structure just a sorted array is used.

The term symbol table stems from compiler development, where this kind of data structure is used in order to keep track of identifiers, or more precisely, so-called symbols. A symbol is simply a string, and the symbol table associates it with arbitrary data, just like an associative array or a PairAssociativeContainer does. Or just like a simple array associates integers with arbitrary data.

Edited by Lachesis

Share this post


Link to post
Share on other sites
Performance test on my system:

 

FPS: 3

FPS (poor_man=1): 20, but occasionally dropping to < 5 FPS and not recovering for a at least minute or two.

FPS (no eye candy): 20

CPU usage at 3 FPS: < 5 %

CPU usage at 20 FPS: ca. 20 %

GPU: Nvidia Geforce FX 5200 (AGP 8x, 128 MB memory)

CPU: AMD Athlon XP 1800

RAM: 768 MB DDR 266

 

Thanks for all that info. :)

 

Unfortunately, if the problem is not a CPU limitation, there's a lot less I can do. I'd be able to work best with case studies. If you were to find an egregiously bad situation (one of those ~3FPS situations), I may be able to deduce what's causing it (try it both with and without point particles). This will probably be a two-step process; you find a specific situation, I'll add in debugging statements to help with figuring out what's going in your situation, you'll re-run it and give me the results, and then I'll use that data to try and improve it.

 

However....

 

That doesn't mean I'll be able to remedy the problem, though. Your card is four years old. Certainly not ancient, but not modern, either. You may simply not have enough video card power to deal with fancy graphics -- for example, translucency. Some old cards simply cannot do translucency quickly (used to be, I could barely run Celestia unless I turned off galaxies (which were translucent); if I turned them off, everything went smoothly) . However, translucency is essential to all good particle systems, and it's used by all of the effects except blowing objects (leaves, petals).

 

Of course, it's hard to say what your problem is without seeing specifics. Which reminds me, I intended to put a checkbox to turn eye candy on or off so that people don't have to choose whether to build with or without it.

Share this post


Link to post
Share on other sites

I'm afraid I couldn't address the FPS drop to any particular event. If so, I gladly had posted it. There's no direct relationship to blending, either, I'm afraid. :) But I wouldn't focuse too much on that, EL already wastes a huge amount of resources (I can play Planeshift at 60 FPS for instance) because of the countless superfluous state changes.

Share this post


Link to post
Share on other sites

---

 

std::map would be O(log N+L) in the best case -- it's only O(L * log N) in the worst case (when all key strings are almost entirely identical data, running from the beginning). std::map does ><= comparisons on its keys (in this case, std::string). std::string stops comparisons as soon as it finds that one string is not a match. So, you only get O(L * log N) case in the following situation:

 

Keys:

AAAAAAAAAAAAA

AAAAAAAAAAAAB

AAAAAAAAAAAAC

AAAAAAAAAAAAD

AAAAAAAAAAAAE

AAAAAAAAAAAAF

AAAAAAAAAAAAG

 

Searching for:

AAAAAAAAAAAAC

 

In this case, it does essentially a full comparison on AAAAAAAAAAAAD, then essentially a full comparison on AAAAAAAAAAAAB, and finally on AAAAAAAAAAAAC. In this specific case, it's O(L log N). However, against a more realistic dataset:

 

Keys:

misc/somethingorother

misc/thingamabob

rocks/rock_big1

rocks/rock_small2

rocks/rubble8

trees/leaves

trees/stump11

trees/tree21

trees/tree3

 

Searching for:

trees/leaves

 

The search starts with rocks/rubble8 and fails immediately. It the goes to trees/stump11, passes up to the s, then fails. It then goes to trees/leaves and succeeds. This is general case -- somewhere in between best and worst.

 

Anyways, if that performance isn't good enough, one can just hash the value of the strings before loading them into the map. This means writing a fast hash function, but not like that's hard, unreliable, or unclear. Also, not like there aren't already a ton of public domain hash functions out there, some -- like superfast hash -- which complete lightning-fast. Certainly seems a lot simpler than learning what's going on in symbol_table. :P The code is not self explanatory, at least to my eyes -- and certainly not as tested as std::map. :)

 

Ed: Of course, not like it's that big of a deal -- we've kind of gotten offtopic here. :)

 

Re: FPS

 

Well, if you find a specific case you'd like me to improve, please post it; I'd be glad to help. :)

Edited by KarenRei

Share this post


Link to post
Share on other sites

Latest patch:

* Fixed teleporter LOD-change memory leak (incl. temporary teleport effects)

* Almost ready to enable the code for object obstructions; still needs a little more work.

Just a comment on the way you have fixed the leak. Ignore me if you wish :) You are calling the erase() method for each object pointer in the vector. Each time erase() is called, the vector moves things around to close the gap. As you intend to completely empty the vector, calling the clear() method just once would be better in my opinion. For example, in TeleporterEffect::request_LOD() you now have:

 while (capless_cylinders.size())
 {
std::vector<Shape*>::iterator iter = capless_cylinders.begin();
delete *iter;
capless_cylinders.erase(iter);
 }

This might be simpler and would be quicker if it was changed to:

 for (size_t i=0; i<capless_cylinders.size(); i++)
  delete capless_cylinders[i];
 capless_cylinders.clear();

 

This function is called very often on my set up at least so it might be worth considering a change that is, in my opinion, simpler and quicker. The overhead of a possible unnecessary clear() is most likely minuscule. As you are mid discussion with others on far more complicated matters I hesitate to suggest this but, there, I've done it.

Share this post


Link to post
Share on other sites

Actually, quite a good suggestion :) I use clear elsewhere -- I just forgot to use it there. Thanks!

 

Don't feel bad about suggesting things; your suggestions have been great and saved me a lot of time! Recently, you've been the most prolific playtester (in terms of catching bugs and offering fixes) so please don't feel bad about doing what you've been doing!

Edited by KarenRei

Share this post


Link to post
Share on other sites
I'll add a note that I plan to strip out the wind from the current wind effect into a separate object that can be accessed by other effects, including non-eye-candy effects. This would include smoke that blows in the wind, which I think would look pretty nice.
I'm just finishing off the client-server wind settings. that includes
extern int wind_speed;  //strength of wind, based on server's setting and local randomization. range of 1..100
extern int wind_direction;	  //wind direction, based on server's setting and local randomization. 0 and 360 are north

I don't know if these would be sufficient for what you want to do (I know the moving around obstacles part isn't included... on the other hand, a function for doing that could be applied to the rain drops as well if the performance hit is acceptable), but the code should now be all set to receive instructions from the server (unless there are more changes needed)

 

also, which of your weather-ish effects would you want to be triggerable by the server? in two places (one each side of NEW_WEATHER) you can find

       } else if(data[0] == weather_effect_leaves){
               //EC_TAG
               return;

data[2] being the intensity (in the range of 0..100), if you want to hook in there... and you could probably use the same enum in client_serv.h and the if/else in weather.c to call other functions that would suit... also depending on which ones entropy would like to set like this, of course

I'm not really an expert on it, but I've spent a bit of time working on it, so gimme a yell if I can help :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×