Materials properties patch committed

[KarenRei]

I've committed a patch to adding materials properties (ambient, diffuse, specular, emission, and shininess) and scene properties (ambient, diffuse, and specular), which can be applied to models, meshes, and textures, along with a sample material_metadata.xml config file. The config file covers almost all textures, but very few models and meshes, so could use some improvement. The patch also adds a contrast option to the menu.

 

NEW_LIGHTING is required to be enabled for these changes. FIX_NORMALS is recommended. Be sure to copy materials_metadata.xml (checked in with the source for now) to your EL datadir. Check it out to see what the format is like.

 

In contrast to the old NEW_LIGHTING, the new patch is far more flexible. Snow can have higher ambient so that shadows don't look as dark on it, and so on. Thus, we're able to have higher contrast (making scenes not look as flat) without having some things like snow shadows end up unusually dark.

 

Known issues:

* Some textures already have fake specular shines on them. This provides real specular shines. Obviously, the real ones should eventually supercede the fake ones.

* Many texture maps have a whole bunch of objects in them that ideally should have different material properties. The way to do this is to assign material properties to the objects themselves. I'm not familiar with a good way to view objects, though. I suppose I could have done it in the map editor. Anyways, until the materials_metadata.xml file is fleshed out more, many metal weapons and armors won't have specular shines, since their textures are often found all jumbled up with other textures and need to be set on a per-model or per-mesh basis.

* This patch only provides for material properties that apply to an entire texture, object, or mesh, as well as scene lighting properties; it does not provide support for ambient, diffuse, specular, and emission maps on objects. While the latter provide more flexibility, they also carry performance costs (these material properties are essentially free). Xaphier's work may eventually provide that capability. An ideal engine has support for both -- maps for when you need them, material properties for when you don't.

* There are some game map elements which are designed improperly (out of convenience), which show now. It's not a major problem, but it can be noticed. The main example I've seen is the marble collonade railings in Irsis. These are comprised of a bunch of independent railing sections put right up against one another rather than a continuous piece. Picture viewing them at an angle where the sun would be causing a specular shine on the near corner of the base of the railing. Well, as it's implemented, there are many pieces, each with a corner pointing at the sun. Due to the way OpenGL calculates specular (on a per-vertex basis), each corner of the bases of the railing get lit up, so there's a row of shimmers instead of just a shimmer on what should be the only real corner, the near one. Fixing this will require replacing these piecemeal rows of shiny objects with a continuous one. Or, making them non-shiny -- but realistically, marble should be shiny.

 

Try it out and let me know what you think.

 

- Karen

[Florian]

Undefined symbols:

_set_model_metadata

_set_hair_metadata

_set_legs_metadata

_set_part_metadata

_set_shirt_metadata

_set_skin_metadata

_set_weapon_metadata

_load_material_metadata

_set_scene_metadata

[KarenRei]

Hmm. What flags are you building with to get that? They're right there in textures.c. Could you perchance have OLD_TEXTURE_LOADER defined? Is that a flag people still use?

[Florian]

Hmm, now this is weird. I can see your changes in the cvs repository of course. And I deleted my textures.* files and did a cvs up. Still, undefined symbols and no functions in textures.c ...

Somehow my cvs does not download the latest files ...

 

this is my .cvsrc:

log -N

diff -uNdp

rdiff -u

update -Pd

checkout -P

release -d

 

did I forget something?

[KarenRei]

Try a clean build, perhaps? Get rid of your old .o's, etc

[Florian]

I completely removed my elc dir and did a clean checkout. Now it compiles and links again

[KarenRei]

Excellent.

 

Let me know how it works for you.

[Florian]

It works, but with new lighting turned on it looks, well, "unusual" would be the polite word ...

[KarenRei]

Screenshot?

Also, did you install the materials_metadata.xml file as directed?

Edited November 28, 2007 by KarenRei

[Florian]

new lighting off

http://www.superfloh.dyndns.org/el/newmaterial1.png

 

new lighting on

http://www.superfloh.dyndns.org/el/newmaterial2.png

 

Also, did you install the materials_metadata.xml file as directed?

erm, ok

looks different now

http://www.superfloh.dyndns.org/el/newmaterial3.png

 

dungeons look weird. Sorry I don't know the right words to describe the "wrong" lighting, I couldn't even find the right word in German.

 

screens:

http://www.superfloh.dyndns.org/el/elscreen007.png

http://www.superfloh.dyndns.org/el/elscreen008.png

Edited November 28, 2007 by Florian

[KarenRei]

newmaterial3.png looks mostly correct, except what's up with your peoples' faces and shirts? It looks like there's a line streaking across them. It seems lined up with the direction of the sun. Do you usually get that, or is that new? What sort of shadow settings do you have?

 

Apart from those bizarre streaks, it looks right. You'll notice everything looks less flat, the helmet is shiny, etc.

 

elscreen008.png looks wrong. Could you tell me where that is to I can visit it and see if I can reproduce that?

[Florian]

newmaterial3.png looks mostly correct, except what's up with your peoples' faces and shirts? It looks like there's a line streaking across them. It seems lined up with the direction of the sun. Do you usually get that, or is that new? What sort of shadow settings do you have?

 

#video_mode= 8

#full_screen = 0

#no_adjust_shadows = 0

#shadows_on= 1

#use_shadow_mapping= 1

#max_shadow_map_size = 2048

#clouds_shadows = 1

#show_reflection= 1

#show_fps=1

#limit_fps=0

#poor_man = 0

#use_vertex_array = 1

#use_mipmaps= 0

#use_point_particles= 1

#particles_percentage= 100

#name_text_size = 1

#name_font = 0

#chat_text_size = 1

#chat_font = 0

#render_fog= 0

#use_vertex_buffers= 1

#click_speed = 300

#ati_click_workaround= 0

#use_alpha_border = 1

#use_floating_messages = 1

#isometric = 0

#perspective= 0.3

#near_plane= 17

#anti_alias = 0

#gamma = 1.2

#render_skeleton = 0

#render_mesh = 1

#use_frame_buffer= 1

#use_old_clicker = 1

#shadow_map_size= 7

#use_compiled_vertex_array= 1

#windows_on_top = 1

#show_statbars_in_hud = 1

#follow_cam = 0

#extended_cam = 1

#follow_strength = 0.09

#const_speed = 7

#lin_speed = 3

#quad_speed = 9

#zcull = 50

#tiles = 1

#2D = 1

#obj = 1

#act = 1

#blend = 1

#special_effects= 1

#serverpopup= 1

#customupdate= 1

#use_3d_alpha_blend= 0

#enable_blood= 1

#use_eye_candy= 1

#min_ec_framerate= 15

#max_ec_framerate= 60

#use_lamp_halo= 0

#transparency_resolution_fix= 0

#use_light_columns= 1

#use_fancy_smoke= 1

#light_columns_threshold= 5

#use_new_lighting= 1

#max_idle_cycles_per_second= 40

#note_text_size= 0.8

#use_alpha_banner= 0

#opaque_window_backgrounds= 0

#continent_map_boundaries= 1

#anisotropic_filter= 16

#video_info_sent= 1

#water_shader_quality= 2

#lighting_contrast= 0.5

 

elscreen008.png looks wrong. Could you tell me where that is to I can visit it and see if I can reproduce that?

 

That's in "You are in White Stone [198,302]" which is of course not quite right.

It's Selain's temple, enter it at WS 244,341

[KarenRei]

There was one more question in there that you missed

 

"Do you usually get that, or is that new?"

 

I'll check out Selain's temple this evening and see if A) I can reproduce that, and if so, what's up with it.

[Florian]

"Do you usually get that, or is that new?"

I really don't know, I play EL with the windows client and use the mac client only for testing. I'll have a look.

[KarenRei]

Well, I've found out what's wrong. And wasted my evening trying to fix it. Apparently when they moved to the new model format, they didn't bother to migrate the FIX_NORMALS code along with it. FIX_NORMALS currently does nothing. The reason the grass looks funky is because its normals are all messed up, like many objects in the game. Normally you don't see it because EL is normally almost completely flat. But when you actually make EL not flat, it shows up, and in cases like this, it can look pretty bad.

 

I've been trying to restore the code (in a function that's been moved to io/e3d_io.c), but it's not nearly so easy this time around. Before, we had these nice neat arrays -- an array for vertices, an array for normals, an array for faces, and so on. Now, they're kind of jumbled all together, and I'm trying to wrap my head around it. In a given object, I'm assuming all of the normals in an object are ((float*)(normals_ptr + v_size * j))[0] through [2], where normals_ptr is cur_object->vertex_data + get_normal_offset(cur_object->vertex_options), and j ranges from 0 to cur_object->vertex_no. I'm picking faces by:

 

int face_list[VERTEX_FLOAT_COUNT];

if (indicies_size == 2)

{

for (k = 0; k < VERTEX_FLOAT_COUNT; k++)

face_list[k] = short_list[j * VERTEX_FLOAT_COUNT + k];

}

else

{

for (k = 0; k < VERTEX_FLOAT_COUNT; k++)

face_list[k] = int_list[j * VERTEX_FLOAT_COUNT + k];

}

 

where j ranges from 0 to cur_object->index_no / VERTEX_FLOAT_COUNT. I'm then assuming that the vertices at each face in the same range is ((float*)(vertex_ptr + v_size * face_list[0] through [2]))[0] through [2], where vertex_ptr is cur_object->vertex_data + get_vertex_offset(cur_object->vertex_options);.

 

It works on most objects, but not on all of them. For example, with feather1.e3d, it works fine, and gets the same results as feather1.e3d's (properly set) normals. But with arrow1.e3d, the data gotten by the above assumptions is bonkers. Vertex 0 is identical to vertex 1, vertex 2 is the same as 3, 4=5, 6=7, and so on. And I have no clue why. In arrow1.e3d's case, it thinks the coordinates of vertices 0 through 4 are:

 

0) 0.066864, 0.003510, 0.006104

1) 0.066864, 0.003510, 0.006104

2) -0.443512, 0.003510, 0.038361

3) -0.443512, 0.003510, 0.005890

 

And so on. Needless to say, this produces completely invalid normals, as it doesn't make any sense. So, if anyone has a clue what I'm misunderstanding that's causing things to break, I'd be much obliged.

 

As for the other graphical bug in that scene, the floor tile being lighter than the wall tile, that's an easy one: the floor tile looked like smooth cobblestone river rocks rather than rough rocks, so when I looked at it, I assigned it to use semigloss rock material properties. The walls, however, are a rough rock material. Thus, it's looking brighter in the light, as it's more reflective. As soon as I fix the other bug, I'll change the floor's material to be a more rough rock surface like the walls. But I need to get to sleep. Two hours ago.

Edited November 30, 2007 by KarenRei

[KarenRei]

Updated version committed, for anyone who wants to try it out. First, Xaphier has a normal-fixing script -- it's not under elc, but a separate package called el3d-file-tools. Run ./converter.py -d PathToYour3dObjectsDir. This will correct all files with broken normals.

 

Secondly, update and rebuild elc. I have it enable two-sided lighting if you use new_lighting so the "flat" objects like grass look right. Remember to build with -DNEW_LIGHTING. For those who haven't installed materials_metadata.xml (see first post), please do so.

 

As for the above example which looked wrong: the grass is fixed now, but the wall/floor clash is not. Unfortunately, this is a map bug. The walls are self-lit, but the floor isn't. Self-lit, the walls don't respond to the change in lighting, and so long as they're self-lit, they never will. There's nothing I can do about that apart from editing the map itself.

 

Please let me know of any other erroneous situations that come up.

[KarenRei]

Blah... had some trouble actually committing the changes They're *actually* in now

[Florian]

OK, I converted all e3d files and this is the result:

 

w/o new lighting: http://www.superfloh.dyndns.org/el/elscreen039.png

with new lighting: http://www.superfloh.dyndns.org/el/elscreen040.png

[KarenRei]

Yeach! What happens when you compile w/o NEW_LIGHTING as a flag? Seeing as you're getting funkiness whether it's turned on or off, seems like something else may be at play here.

Edited December 17, 2007 by KarenRei

[Florian]

w/o NEW_LIGHTING everything looks just fine and normal.

 

/EDIT

all flags:

-DELC -DOSX -DAFK_FIX -DALPHA_ACTORS -DATI_9200_FIX -DAUTO_UPDATE -DCLICKABLE_CONTINENT_MAP -DCLUSTER_INSIDES -DCOUNTERS -DCUSTOM_LOOK -DCUSTOM_UPDATE -DCXX_MISC -DEYE_CANDY -DFONTS_FIX -DFUZZY_PATHS -DIDLE_FIX -DMASKING -DMINES -DMINIMAP -DNEW_ACTOR_ANIMATION -DNEW_ACTOR_SCALE -DNEW_FILE_IO -DNEW_SOUND -DNEW_TEX -DNOTEPAD -DOGG_VORBIS -DOPTIONS_I18N -DPNG_SCREENSHOT -DPOPUP -DSFX -DSIMPLE_LOD -DUSE_INLINE -DUSE_SEND_VIDEO_INFO -DZLIB -DUSE_SHADER -DMISSILES -DNO_SDL_REINIT

Edited December 17, 2007 by Florian

[KarenRei]

Q: You do have the materials metadata xml file installed, right?

[Florian]

Q: You do have the materials metadata xml file installed, right?

A: material_metadata.xml lies next to extentions.xml, knowledge.xml and mines.xml in my data directory.

[KarenRei]

If I get a chance (kind of tight, as we're getting ready for xmas), I'll try to reproduce it tonight.

[KarenRei]

Reproduced and fixed. Thanks for reporting!

[Florian]

Looks much better now