Optimizing The Renderer

[Wytter]

Hey all,

 

I fooled a bit around with optimizing the renderer, and came up with the following:

 

http://wytter.tfm.ro/elc/vbo.diff

 

I got performance increases of 20-50%, so actually quite noticable - furthermore it uses noticable less CPU power. That is - where I normally got ~80 FPS now I'm getting 120 etc. It will only be usable on newer graphics cards with updated drivers however, but the decrease in the number of state changes should be somewhat noticable as well

 

What I did was to add vertex buffer objects for all 3d objects. They are now located in video memory, hence it removes a huge memory bottle neck. Furthermore I've made sure that the number of state changes have decreased throughout the client.

 

I did not want to commit it yet however, since I am not exactly an OpenGL Expert - so please look over the patch and yell at me if I did anything wrong

 

Please give some input of whether it's working for you as well. It will tell you that it's using the GL_ARB_vertex_buffer_object extension when opening EL - if it doesn't appear, you're not using it.

 

I could've added actors to this as well, but since we're changing to Cal3D I didn't want to bother with it..

Edited May 7, 2005 by Wytter

[Learner]

At a minimum, reducing state changes along with model & texture changes can help everyone, even people with slower systems. I've been thinking that sorting the map maps by object name/type would even help speed things up. But, the sort can't affect the object ID's. This can even help VBO as well.

 

For VBO, does your patch allow it to still run on slower/older machines? Yes, I haven't looked at it.

[Wytter]

Yes, it just detects if the video card (or rather, the driver) supports VBO - if not, it'll just use the old method that reads data from the main memory.

 

And yes, it does sound like a good idea to render the same model types at the same time. Could easily be done using i.e. a BSP tree.

Edited May 7, 2005 by Wytter

[Entropy]

Not really. Many obejcts use multiple textures, so this won't help that much. Plus, I already optimized the objects so they render each texture at once, without swaping textures around in a single object.

[Learner]

The older, slower, and less memory that a video board or system has, the more this will make a difference. The fewer textures you swap through at a time for instance will allow slow video boards to more effectively use their local memory and use system memory or texture reloading less often. Drawing the same object multiple times in a row may be able to improve things like cache hit rates. This has potential enough to be investigated to not be discounted immediately. Plus, VBO usage can be even more effective when you do multiple usages of he same VBO in a row. This also allows another option for low memory video boards that do support VBO, by creating them (or some of them) just before they are needed, and then destroying when you are done.

[LabRat]

Results on 3 systems (unoptimised graphics options)

PC1 - junk PC for the wife

AMD Duron 1.3Ghz

512MB

GeForce4MX4-440 128MB

Windows XP Pro

el.exe 40FPS

CVS with patch 50-60FPS, occasionally 65+

 

PC2 - My programming PC

AMD Athlon 2.0GHz

512MB

ATI Radeon 9200se 128MB

Windows XP Pro

el.exe 65FPS

90-120FPS, occasionally 130FPS

 

PC3 - The Gaming PC

Intel Pentiom 4, 3.0GHz

1024MB

PCI Express

ATI X300 256MB

Windows XP Pro

Averaging 150FPS, increasing drastically (250 and better) when the processor cooling system kicks in (first time I heard it since the PC was bought)

 

Wytter - for such a small patch file to give such great results.. You need a 20K patch file and we will all be playing at 500FPS on a 286/12 with 128KB

[mikeman]

IMO, the best way to get performance&quality is to drop the "object" as the main entity in a map and switch to space partitioning entirely. That is, don't store the map as a list of objects, but compile the map as a list of polygons organised in BSP or octree, with each leaf being a VBO with its materials sorted. Traverse the tree, and if a leaf is visible, just render the VBO. You can't get any faster than that.

[Wytter]

Just updated the patch. Fixes crashing when switching resolutions.

 

Sounds interesting mikeman - could be a good solution for EL with static maps, but is it also a good approach when using dynamic maps?

 

I'm very glad to hear the patch is getting the same results with you Labrat

Edited May 7, 2005 by Wytter

[DarkWarriorBOW]

It IS faster, about 20 to 30% increase in FPS.

In low-detail areas, my fps shot up to 200+. I never had that before.

 

But.. it's also more unstable. in the end it actually crashed after screwing up royally. It totally lost track of vertex sizes and things blew out of proportion before crashing.

Also textures seemed to get misplaced on the Z-plane.

 

All in all, good work, but you better do some heavy tweaking before the release.

 

Here is a screenshot of the texture order misplacement :

 

 

Hardware :

 

Intel Pentium 4 HT (in HT mode) 3.06 Ghz

Windows XP Pro SP1

1024MB

ATI Radeon Mobility 7500, 64MB nonshared memory

[espenfjo]

Oh, lakeside is flooded

 

Cool research addon, where did you get that?

[Wytter]

I see - strange, it hasn't happened to me yet. Which patch version where you using? (The 10 or the 16kb one?) Could you provide some more details, such as how long it took before that happened etc.?

 

Oh, I see that you're using a costum client. That might be the source of the error - try using a normal CVS client and see if it happens then.

Edited May 7, 2005 by Wytter

[DarkWarriorBOW]

@ Espenfjo : LOL, check the patches on berlios

 

@Wytter : I am not sure if i can reproduce it. could have been an 'accident'.

I am using the 16kb patch, I assume that is the new one.

It took about 20 secs to crash and die, it didnt do that anymore after the first crash. but then again, i haven't given it enough time.

 

Texture misalignments and screwed up objects are common and everywhere at all times (look at the barrels in the screenshot).

 

Yes I am using a custom client. That means, the latest (todays actually) source with only the coordinates patch, tag color and research bar.

 

I dont see where it should screw up there. but ill humor you lol

gonna compile a clean CVS.

Edited May 7, 2005 by DarkWarriorBOW

[Wytter]

I've run with this all evening and without problems. Try with a clean CVS.

 

EDIT: Just crashed a few minutes after. Can it be caused by the cache cleaning up?

Edited May 7, 2005 by Wytter

[Brom]

Looks to be an overall improvement. I have not crashed it yet. but these are the results.

System:

 

AMD Athlon 2400+ (~2GHz)

512 DDR400 RAM

Nvidia Geforce FX 5200 (128MB)

Gentoo Linux

ewq222@Glowie elc $ uname -a
Linux Glowie 2.6.10-gentoo-r4 #1 SMP Tue Jan 11 04:23:36 MST 2005 i686 AMD Athlon(tm) XP 2400+ AuthenticAMD GNU/Linux

 

eh, if you want to send me a message latter, tell me how to fix my timestamp anyway.

Before using CVS client 14-17FPS not moving about the same when moving.

After your patch: 14-21FPS not moving and a ton more when moving.

 

Its sort of random I think, but it bounces rapidly from 20FPS to 80+FPS when moving, sort of strange.

ok, another test near raven showed bouncing from 25 FPS to 160FPS..yeah this is odd, but I would say it is an overall improvement.

 

Any more information you need let me know.

 

EDIT:Added glxinfo

ewq222@Glowie ewq222 $ glxinfo
name of display: :0.0
display: :0  screen: 0
direct rendering: Yes
server glx vendor string: NVIDIA Corporation
server glx version string: 1.3
server glx extensions:
   GLX_EXT_visual_info, GLX_EXT_visual_rating, GLX_SGIX_fbconfig,
   GLX_SGIX_pbuffer, GLX_SGI_video_sync, GLX_SGI_swap_control,
   GLX_ARB_multisample, GLX_NV_float_buffer
client glx vendor string: NVIDIA Corporation
client glx version string: 1.3
client glx extensions:
   GLX_ARB_get_proc_address, GLX_ARB_multisample, GLX_EXT_visual_info,
   GLX_EXT_visual_rating, GLX_EXT_import_context, GLX_SGI_video_sync,
   GLX_NV_swap_group, GLX_SGIX_fbconfig, GLX_SGIX_pbuffer,
   GLX_SGI_swap_control, GLX_NV_float_buffer
GLX extensions:
   GLX_EXT_visual_info, GLX_EXT_visual_rating, GLX_SGIX_fbconfig,
   GLX_SGIX_pbuffer, GLX_SGI_video_sync, GLX_SGI_swap_control,
   GLX_ARB_multisample, GLX_NV_float_buffer, GLX_ARB_get_proc_address
OpenGL vendor string: NVIDIA Corporation
OpenGL renderer string: GeForce FX 5200/AGP/SSE/3DNOW!
OpenGL version string: 1.5.1 NVIDIA 61.11
OpenGL extensions:
   GL_ARB_depth_texture, GL_ARB_fragment_program,
   GL_ARB_fragment_program_shadow, GL_ARB_fragment_shader, GL_ARB_imaging,
   GL_ARB_multisample, GL_ARB_multitexture, GL_ARB_occlusion_query,
   GL_ARB_point_parameters, GL_ARB_point_sprite, GL_ARB_shadow,
   GL_ARB_shader_objects, GL_ARB_shading_language_100,
   GL_ARB_texture_border_clamp, GL_ARB_texture_compression,
   GL_ARB_texture_cube_map, GL_ARB_texture_env_add,
   GL_ARB_texture_env_combine, GL_ARB_texture_env_dot3,
   GL_ARB_texture_mirrored_repeat, GL_ARB_transpose_matrix,
   GL_ARB_vertex_buffer_object, GL_ARB_vertex_program, GL_ARB_vertex_shader,
   GL_ARB_window_pos, GL_S3_s3tc, GL_EXT_texture_env_add, GL_EXT_abgr,
   GL_EXT_bgra, GL_EXT_blend_color, GL_EXT_blend_func_separate,
   GL_EXT_blend_minmax, GL_EXT_blend_subtract, GL_EXT_compiled_vertex_array,
   GL_EXT_Cg_shader, GL_EXT_draw_range_elements, GL_EXT_fog_coord,
   GL_EXT_multi_draw_arrays, GL_EXT_packed_pixels, GL_EXT_paletted_texture,
   GL_EXT_pixel_buffer_object, GL_EXT_point_parameters,
   GL_EXT_rescale_normal, GL_EXT_secondary_color,
   GL_EXT_separate_specular_color, GL_EXT_shadow_funcs,
   GL_EXT_shared_texture_palette, GL_EXT_stencil_two_side,
   GL_EXT_stencil_wrap, GL_EXT_texture3D, GL_EXT_texture_compression_s3tc,
   GL_EXT_texture_cube_map, GL_EXT_texture_edge_clamp,
   GL_EXT_texture_env_combine, GL_EXT_texture_env_dot3,
   GL_EXT_texture_filter_anisotropic, GL_EXT_texture_lod,
   GL_EXT_texture_lod_bias, GL_EXT_texture_object, GL_EXT_vertex_array,
   GL_HP_occlusion_test, GL_IBM_rasterpos_clip,
   GL_IBM_texture_mirrored_repeat, GL_KTX_buffer_region, GL_NV_blend_square,
   GL_NV_copy_depth_to_color, GL_NV_depth_clamp, GL_NV_fence,
   GL_NV_float_buffer, GL_NV_fog_distance, GL_NV_fragment_program,
   GL_NV_fragment_program_option, GL_NV_half_float, GL_NV_light_max_exponent,
   GL_NV_multisample_filter_hint, GL_NV_occlusion_query,
   GL_NV_packed_depth_stencil, GL_NV_pixel_data_range, GL_NV_point_sprite,
   GL_NV_primitive_restart, GL_NV_register_combiners,
   GL_NV_register_combiners2, GL_NV_texgen_reflection,
   GL_NV_texture_compression_vtc, GL_NV_texture_env_combine4,
   GL_NV_texture_expand_normal, GL_NV_texture_rectangle,
   GL_NV_texture_shader, GL_NV_texture_shader2, GL_NV_texture_shader3,
   GL_NV_vertex_array_range, GL_NV_vertex_array_range2, GL_NV_vertex_program,
   GL_NV_vertex_program1_1, GL_NV_vertex_program2,
   GL_NV_vertex_program2_option, GL_SGIS_generate_mipmap,
   GL_SGIS_texture_lod, GL_SGIX_depth_texture, GL_SGIX_shadow,
   GL_SUN_slice_accum
glu version: 1.3
glu extensions:
   GLU_EXT_nurbs_tessellator, GLU_EXT_object_space_tess

 

the other junk I did not think you wanted to see.

Edited May 7, 2005 by EWQ222

[Wytter]

OK, I think I found the source of the crash as I forgot deleting the allocated buffers for the clouds uv when switching maps.

 

Updated the patch.

Edited May 7, 2005 by Wytter

[DarkWarriorBOW]

Ok, humored you and compiled a clean CVS and patched it.

It instantly crashed on resizing the client 3 times in a row. did not try a fourth time.

Same problems still occur sorry.

 

oh, and the patch also breaks the use of alpha borders. (the shadows under the windows)

Edited May 7, 2005 by DarkWarriorBOW

[Wytter]

Yea, I know - I commented the alpha borders out, since I hate the current implementation.

 

Tried the updated patch?

Edited May 7, 2005 by Wytter

[Wytter]

Hmm, so it's crashing when resizing the window? That's strange... What driver are you using?

[DarkWarriorBOW]

I am using Catalyst 5.4 drivers.

 

Since i am using a laptop, and driver support from laptop vendors is as lame as it gets. (newest drivers for my laptop from Acer are still Catalyst 4.something) i use the Omega drivers. (www.omegadrivers.net)

 

All laptop users that can't find current video drivers for their laptop i can recommend to get their drivers from there.

Omega has both Detonator and Catalyst drivers that work on virtually ANY card, wether it be laptop or not.

 

laptop producers suck lol.

Edited May 7, 2005 by DarkWarriorBOW

[Wytter]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 182925802208 (LWP 25127)]
0x0000002a983a6043 in ?? ()
(gdb) backtrace
#0  0x0000002a983a6043 in ?? ()
#1  0x0000002a97bb9040 in ?? ()
#2  0x0000002a98006a30 in ?? ()
#3  0x000000000000003f in ?? ()
#4  0x0000002a97bb9040 in ?? ()
#5  0x0000002a96dbf148 in _nv000859gl () from /usr/lib/opengl/nvidia/lib/libGLcore.so.1
#6  0x0000002a96b8ce67 in _nv000820gl () from /usr/lib/opengl/nvidia/lib/libGLcore.so.1
#7  0x000000000043cdd3 in draw_3d_reflection (object_id=0x3e0aae0) at reflection.c:219
#8  0x000000000043d3fb in display_3d_reflection () at reflection.c:373
#9  0x000000000041ef59 in display_game_handler (win=0x337a4b0) at gamewin.c:553
#10 0x00000000004188f6 in draw_window (win=0x337a4b0) at elwindows.c:980
#11 0x0000000000418d10 in display_window (win_id=0) at elwindows.c:1106
#12 0x0000000000416e4f in display_windows (level=1) at elwindows.c:52
#13 0x0000000000415300 in draw_scene () at draw_scene.c:87
#14 0x000000000042d21d in start_rendering () at main.c:59
#15 0x000000000042d3f2 in main (argc=0, argv=0x0) at main.c:143

 

I've made an error somewhere... Just don't know where yet :-\...

[Wytter]

Found and fixed one bug, but I'm not sure if it's the last. Updated the patch.

[DarkWarriorBOW]

I don't know what you did, but i can't use the diff file anymore to patch into my clean CVS tree.

 

only the patching in 3d_objects.c works, the rest gets rejected..

 

C:\Documents and Settings\blahblahusernameblah\Mijn documenten\C++ Projecten\elc>patch

.diff.txt --binary

patching file 3d_objects.c

patching file alphamap.c

Hunk #1 FAILED at 162.

Hunk #2 FAILED at 165.

Hunk #3 FAILED at 169.

Hunk #4 FAILED at 176.

Hunk #5 FAILED at 232.

5 out of 5 hunks FAILED -- saving rejects to file alphamap.c.rej

patching file e3d.h

patching file gl_init.c

patching file gl_init.h

patching file map_io.c

Hunk #1 FAILED at 51.

1 out of 1 hunk FAILED -- saving rejects to file map_io.c.rej

patching file reflection.c

Hunk #1 FAILED at 198.

Hunk #2 FAILED at 219.

Hunk #3 FAILED at 220.

3 out of 5 hunks FAILED -- saving rejects to file reflection.c.rej

patching file shadows.c

Hunk #1 FAILED at 221.

Hunk #2 FAILED at 229.

Hunk #3 FAILED at 241.

Hunk #4 FAILED at 242.

Hunk #5 FAILED at 246.

Hunk #9 FAILED at 548.

Hunk #12 FAILED at 612.

7 out of 13 hunks FAILED -- saving rejects to file shadows.c.rej

 

did some heavy moving around in the code ?

Edited May 7, 2005 by DarkWarriorBOW

[Wytter]

Sounds more likely that your tree isn't clean and that you already have a patched source tree. Delete everything, download the patch again and then try over.

[DarkWarriorBOW]

Well, to humor you again, here you go again :

 

D:\CVS SOURCES\Eternal Lands\Kopie van elc>patch -i vbo.diff.txt --binary

patching file 3d_objects.c

Hunk #3 FAILED at 34.

Hunk #5 FAILED at 61.

Hunk #6 FAILED at 66.

Hunk #7 FAILED at 71.

Hunk #8 FAILED at 95.

Hunk #9 FAILED at 129.

Hunk #10 FAILED at 130.

Hunk #11 FAILED at 137.

Hunk #12 FAILED at 146.

Hunk #13 FAILED at 147.

Hunk #14 FAILED at 162.

Hunk #15 FAILED at 164.

12 out of 21 hunks FAILED -- saving rejects to file 3d_objects.c.rej

patching file alphamap.c

Hunk #1 FAILED at 162.

Hunk #2 FAILED at 165.

Hunk #3 FAILED at 169.

Hunk #4 FAILED at 176.

Hunk #5 FAILED at 232.

5 out of 5 hunks FAILED -- saving rejects to file alphamap.c.rej

patching file e3d.h

patching file gl_init.c

patching file gl_init.h

patching file map_io.c

Hunk #1 FAILED at 51.

1 out of 1 hunk FAILED -- saving rejects to file map_io.c.rej

patching file reflection.c

Hunk #1 FAILED at 198.

Hunk #2 FAILED at 219.

Hunk #3 FAILED at 220.

3 out of 5 hunks FAILED -- saving rejects to file reflection.c.rej

patching file shadows.c

Hunk #1 FAILED at 221.

Hunk #2 FAILED at 229.

Hunk #3 FAILED at 241.

Hunk #4 FAILED at 242.

Hunk #5 FAILED at 246.

Hunk #9 FAILED at 548.

Hunk #12 FAILED at 612.

7 out of 13 hunks FAILED -- saving rejects to file shadows.c.rej

 

D:\CVS SOURCES\Eternal Lands\Kopie van elc>

 

That is trying to patch it into a copy my master CVS source that i -always- keep clean in case of screw up and to diff my experimental version against so that i can keep track of newest developments without losing my own work.

 

Trust me on this one after 22 years of coding, i tend to know what I am doing

[Wytter]

Strange indeed. It might be because of unix/windows newlines though.

 

Try redownloading it. I just uploaded the file again from a fresh cvs dif > vbo.diff.